Learn how to get data into Wavefront.

Wavefront customers use Wavefront to visualize their metric and instrument them with alerts. How you get your data into Wavefront depends on the use case, but you have many options. Wavefront supports many integrations, or you can instrument your application and send data directly to Wavefront.

Watch this video to listen to our Co-founder Clement Pang talk about getting data into Wavefront:

getting data into wavefront

The Big Picture

You can use Wavefront for any time series metrics, even if you’re already collecting those metrics with a custom metrics pipeline. The data flow is similar in all cases:

  1. The metrics originate with your infrastructure or application.
  2. The metrics are sent to Wavefront:
    • If a built-in integration exists for your data source, setting up the integration arranges for a collector agent to take the metrics from your source and send them to a Wavefront proxy. A single proxy can forward metrics from many sources to the Wavefront service.
    • For cloud services, you use one of the Cloud integrations – you don’t have to install or set up a Wavefront proxy.
    • If no built-in integration for your data source exists, you can set up one of the collector integrations (such as a collectd integration) or you can stream your data to the Wavefront proxy. The proxy accepts metrics in OpenTSB, Graphana, and Wavefront data format.

data into wavefront

Step 1: Understand Your Use Case

Wavefront can help you get insight into all levels of your toolchain. Here are the levels, and the associated use cases:

application layers

  • Infrastructure metrics (1) enable you to find bottlenecks, explore trends, and troubleshoot your environment. You may be interested in:
    • Metrics such as CPU loads, memory usage, and so on, from host operating systems.
    • Application-driven infrastructure data from NGNIX, MySQL, and similar sources.

    See the list of integrations.

  • Application metrics (2) include data you want to collect from custom code in your proprietary applications, for example, to monitor business activity or to collect customer data such as website interactions.

  • Cloud metrics (3) are various metrics that are generated by a suite of cloud computing services such as AWS or GCP. Wavefront lets you combine and correlate these metrics, for unified visualizing, alerting, and troubleshooting.

  • Special cases don’t readily fit into one of the use cases above. For example, you might already have a metrics pipeline in place, or you might want to pull metrics from log files to present them from analysis.

Step 2: Use the Simplest Integration Path for Your Use Case

The simplest integration path for any use case is to use an integration, if one is supported. Even without one, though, it’s straightforward to get your metrics into Wavefront.

Infrastructure Integration

For use cases that center around infrastructure data like CPU, memory, or database metrics, you can get your data into Wavefront by following these steps:

  1. Check whether a built-in integration for your use case exists. See the complete list of integrations.

  2. Use the Setup steps in the integration to get data flowing from your application to Wavefront. As part of the setup, the integration lets you install a Wavefront proxy or choose a Wavefront proxy if you already have one installed.

  3. If no built-in integration exists, set up a collector agent such as Telegraf or collectd to collect your metrics. Wavefront includes integrations for many custom collector agents. The collector agent can send your data to the Wavefront proxy.

    Some collector agents, such as Telegraf, include an output plug-in for Wavefront. You add a configuration file to Telegraf that specifies which data you want to get from which host, and select the proxy to send the data to.

    data flow

    Many of the built-in integrations use Telegraf. Wavefront supports different ways of using Telegraf, but also supports other agents. You can mix and match.

    • The Active Directory integration is an example for a Windows Telegraf input plugin.
    • The Mesos integration is an example for an environment where both primary and secondary server have to have Telegraf input plugins.
    • The JBoss AS integration is an example for a Linux Telegraf plugin that uses Jolokia.

Application Code Instrumentation

For use cases that involve metrics from proprietary applications, you can instrument your custom code and get data into Wavefront using one of the following techniques:

  • Use a Wavefront application instrumentation integration. Wavefront supports instrumentation libraries for several popular programming languages. For example, you can create a Java Wavefront Reporter to use DropWizard metrics.

    1. Click Integrations and search for Application Integration to display the available application instrumentation integrations.
    2. Choose the integration that best suits your use case.
    3. Follow the Setup steps to register metrics and configure metric reporters. As part of the setup, the integration lets you install a Wavefront proxy or pick one if you have proxies in your environment.
  • Use your own custom-coded instrumentation:

    1. Set up a Wavefront Proxy. If you have a proxy installed in your environment, consider using that - or you can install a new proxy. See Installing and Managing Proxies
    2. Send the metrics you’re interested in to the Wavefront proxy at port 2878 in a format that Wavefront supports. See Supported Data Formats

Note: You can use a Wavefront Application Instrumentation integration to send metrics from your applications.

Metrics from a Cloud Provider

Cloud integrations use a different setup process than other integrations. You log in to the cloud service, and give Wavefront read access to the metrics that you’re interested in. There’s no need to install or set up a Wavefront proxy.

Wavefront supports integrations for the most popular cloud services. This doc set has customization information for example, for AWS Metrics Integration, AWS ECS Integration, AWS Lambda Functions, and Pivotal Container Service.

Special Use Cases

Even if your use case doesn’t fit the examples above, you can still get your time-series metrics into Wavefront.

If your company already has a custom metrics pipeline, follow these steps:

  1. Convert your data into one of the supported data formats. Wavefront accepts data in Wavefront data format, Graphite data format, and OpenTSDB data format.
  2. Send the data to a Wavefront proxy. You can use one proxy or a set of proxies behind a load balancer in most situations.

If you have metrics recorded in log files, follow these steps:

  1. Use the Filebeat Log Data built-in integration.
  2. Customize the process as discussed in Log Data Metrics Integration

Note For some integrations that use collectd instead of Telegraf as the collection agent, we give you the setup instructions in the documentation. See Integrations Pages for all topics related to integrations.

Step 3: Use Wavefront for Visualization and Alerting.

Once data is flowing, you can use Wavefront for visualization and alerting.

  1. Many integrations include dashboards that collect commonly used metrics.
  2. You can customize existing dashboards or create charts from scratch and add them to your own dashboard.
  3. Optionally, you can add alerts to your charts and send them to alert targets such as PagerDuty or a custom Webhook target.