Guru Meditation Reports

Nova contains a mechanism whereby developers and system administrators can generate a report about the state of a running Nova executable. This report is called a Guru Meditation Report (GMR for short).

Generating a GMR

A GMR can be generated by sending the USR2 signal to any Nova process with support (see below). The GMR will then be outputted standard error for that particular process.

For example, suppose that nova-compute has process id 8675, and was run with 2>/var/log/nova/nova-compute-err.log. Then, kill -USR2 8675 will trigger the Guru Meditation report to be printed to /var/log/nova/nova-compute-err.log.

Nova API is commonly run under uWSGI, which intercepts SIGUSR2 signals. In this case, a file trigger may be used instead:

[oslo_reports]
log_dir = /var/log/nova
file_event_handler = /var/log/nova/gmr_trigger

Whenever the trigger file is modified, a GMR will be generated. To get a report, one may use touch /var/log/nova/gmr_trigger. Note that the configured file trigger must exist when Nova starts.

If a log dir is specified, the report will be written to a file within that directory instead of stderr. The report file will be named ${serviceName}_gurumeditation_${timestamp}.

Structure of a GMR

The GMR is designed to be extensible; any particular executable may add its own sections. However, the base GMR consists of several sections:

Package

Shows information about the package to which this process belongs, including version information

Threads

Shows stack traces and thread ids for each of the threads within this process

Green Threads

Shows stack traces for each of the green threads within this process (green threads don’t have thread ids)

Configuration

Lists all the configuration options currently accessible via the CONF object for the current process

Adding Support for GMRs to New Executables

Adding support for a GMR to a given executable is fairly easy.

First import the module, as well as the Nova version module:

from oslo_reports import guru_meditation_report as gmr
from oslo_reports import opts as gmr_opts
from nova import version

Then, register any additional sections (optional):

gmr.TextGuruMeditation.register_section('Some Special Section',
                                        some_section_generator)

Finally (under main), before running the “main loop” of the executable (usually service.server(server) or something similar), register the GMR hook:

gmr_opts.set_defaults(CONF)
gmr.TextGuruMeditation.setup_autorun(
  version, conf=CONF, service_name=service_name)

The service name is used when generating report files. If unspecified, GMR tries to automatically detect the binary name using the stack trace but usually ends up with thread.py.

Extending the GMR

As mentioned above, additional sections can be added to the GMR for a particular executable. For more information, see the inline documentation under oslo.reports