Learn how to send data emitted by New Relic to Wavefront.

New Relic is a popular APM solution for monitoring applications. The Wavefront collector supports pulling metrics from New Relic at least every minute. Application summary data can be polled every 30s, but is not guaranteed by New Relic to be updated that often.

Requirements

  • The New Relic user must be an admin.
  • Python 2.7+

Installation

$ pip install wavefront_collector
$ mkdir -p /opt/wavefront/etc/

Configuration

  1. Download the sample configuration files from wavefront-integrations-tool and place them into /opt/wavefront/etc.
  2. Open /opt/wavefront/etc/newrelic-summary.conf and /opt/wavefront/etc/newrelic-details.conf.
  3. Set the key in the api section.
  4. Set the application_ids in the filter section.
  5. Set the host and dry_run values in the writer section when ready to test. Leave dry_run = True to test what would be sent to the proxy. While dry_run = True run the script in the foreground so you can see the output and press Ctrl-C to stop execution.
  6. Update other configuration options as desired.
  7. Save the files.

Running the Script

The main script is wf (which is a symlink to wave.py). It is installed by pip in /usr/local/bin on most systems.

Command Line Options

OptionDescriptionDefault
--config FILE Full path to the configuration file /opt/wavefront/etc/wavefront-collector.conf

Execute

$ wf --config /opt/wavefront/etc/wavefront-collector.conf

Service Script

You can also run this as a service. A service script (/usr/local/bin/wavefront-collector) is provided in the pip installer. This script can be run with:

$ wavefront-collector [(start|stop|status|restart)]

This script assumes that you have /opt/wavefront/etc/wavefront-collector.conf (and is updated according to earlier steps).

Log Files

The default log file location is /tmp/wavefront-collector.log. This can be changed by editing the /opt/wavefront/etc/wavefront-collector.conf file.

Caching

The responses from */metrics.json API calls (the API that gets a list of metric names that are available) are cached for a day in /tmp/wfnrcache. One file is stored here per API path. The file is named by the MD5 of the API path.

Configuration Options

The default configuration files provided should be sufficient, but all configuration options available are documented here. Configuration is retrieved from and stored in an INI-formatted file with multiple sections. Each section is described in more detail below.

This configuration file also acts as a fileconfig for the logger. See fileConfig definition for more details on how to configure logging.

Section: api

This section configures items related to the New Relic API.

OptionDescriptionRequired?Default
key New Relic API key. See Generate your API keys. Yes None
endpoint New Relic API endpoint No https://api.newrelic.com/v2
log_path Path to the log file that will store API requests No None

Section: filter

This section configures the data that will be collected.

OptionDescriptionRequired?Default
names Comma-separated list of specific names to retrieve No None
regex Comma-separated list of regular expressions to match against results of the metrics.json calls. White list matches are retrieved from data.json. No None
blacklist_regex Comma-separated list of regular expressions to not include. Black list trumps white list. No None
additional_fields Comma-separated list of metric names to retrieve in addition to the ones returned by metrics.json calls. By default, you probably will want to include HttpDispatcher,Errors,Memcached,External No None
application_ids Comma-separated list of New Relic application IDs to retrieve metrics from No All
start_time Start time for range based backfilling query (YYYY-MM-DDTHH:mm:ss+00:00) No None
end_time End time for range based backfilling query (YYYY-MM-DDTHH:mm:ss+00:00) No None

Section: options

This section contains various options to configure how the script is executed.

OptionDescriptionRequired?Default
include_application_summary Include the summary metrics from the application API (error_rate, etc) No True
include_host_application_summary Include the summary metrics from the /host/application API No True
include_hosts Include the host metrics from /host/data.json No True
include_server_summary Include the summary details from the /servers/data.json API No True
include_server_details Include the server metrics from the /servers/data.json API No False
min_delay The minimum number of seconds between the last run time and the current run time No 60
skip_null_values Do not include metrics with null values (0 is not null in this case) No False
default_null_value If including null values, then replace 'null' with this value No 0
max_metric_names Maximum number of metric names to request at one time when querying /data.json API No 25
workers The number of threads to run when making /data.json requests No 30
send_zero_every Send 0 values no more than every x seconds. Set to 0 to send 0 values on every iteration. No 0

Section: wavefront_api

This section contains configuration items to describe the Wavefront API interaction. This section is not used currently.

OptionDescriptionRequired?Default
key Wavefront API Key No None
endpoint Wavefront endpont No None

Section: writer

This section describes how to contact the Wavefront proxy.

OptionDescriptionRequired?Default
host The host/IP of the writer endpoint Yes 127.0.0.1
port The port of the writer endpoint Yes 2878
dry_run When True, don’t actually send data points to the endpoint configured. Instead, print what would have been sent on stdout. No True

Standard Configuration

  • wavefront-collector.conf
    • Runs the script as a daemon with the PID file stored in the current working directory (./newrelic.pid) and the stdout/stderr in ./newrelic.out.
    • Runs 2 threads: one for processing the summary and one for the details.
  • summary.conf
    • Gets the application summary every 30s (no details are retrieved so the summary metrics can be updated more frequently)
  • details.conf
    • Gets the application (host) details and delays 5m between each run.