Advanced Proxy Configuration and Installation

You can configure proxies using a configuration file, and you can performing advanced installation management such as installing proxies in a container.

In addition to the proxy configuration properties discussed here you can also use proxy preprocessor rules. These rules allow you to manipulate incoming metrics before they reach the proxy, for example, you could remove confidential text strings or replace unacceptable characters.

Proxy Configuration Properties

The main Wavefront proxy configuration file is maintained in <wavefront_config_path>/wavefront.conf (<wf_config_path>/wavefront.conf). The configuration file offers many options for changing how the proxy processes your data. There are optional configuration files for rewriting metrics and parsing log data. The default values work well in many cases, but you can adjust them as needed. After changing a configuration option, restart the proxy service.

Paths

In this section, file paths use the following conventions and values:

• <wavefront_config_path>
  • Linux - /etc/wavefront/wavefront-proxy
  • Mac - /usr/local/etc/wavefront/wavefront-proxy
  • Windows - C:\Program Files (x86)\Wavefront\conf
• <wavefront_log_path>
  • Linux - /var/log/wavefront
  • Mac - /usr/local/var/log/wavefront
  • Windows - C:\Program Files (x86)\Wavefront
• <wavefront_spool_path>
  • Linux - /var/spool/wavefront-proxy
  • Mac - /usr/local/var/spool/wavefront-proxy
  • Windows - C:\Program Files (x86)\Wavefront\bin

General Proxy Properties and Examples

This section lists:

• General proxy configuration properties
• Metrics proxy configuration properties
• Tracing proxy configuration properties

See Histogram Configuration Properties for properties specific to histogram distributions.

Tracing Proxy Properties and Examples

Data Buffering

If the Wavefront proxy is unable to post received data to the Wavefront servers, it buffers the data to disk across a number of buffer files, and then tries to resend the points once the connection to the Wavefront servers is available again. If this buffering occurs, you’ll see lines like this in wavefront.log:

2013-11-18 18:02:35,061 WARN  [com.wavefront.daemon.QueuedSshDaemonService] current retry queue sizes: [1/0/0/0]

By default, there are 4 threads (and 4 buffer files) waiting to retry points once the connections are up; this line shows how many blocks of points have been stored by each thread (in this case, the first thread has 1 block of queued points, while the second, third, and fourth threads all have 0 blocks). These lines are only printed when there are points in the queue; you’ll never see a line with all 0’s in the queue sizes. Once the connection to the Wavefront servers has been established, and all the threads have sent the past data to us, you’ll see a single line like this in wavefront.log:

2013-11-18 18:59:46,665 WARN [com.wavefront.daemon.QueuedSshDaemonService] retry queue has been cleared

Logging

The Wavefront proxy supports two log files: proxy log and blocked point log. To keep the log file sizes reasonable and avoid filling up the disk with logs, both log files are automatically rotated and purged periodically. You configure the log file locations and rotation rules in <wavefront_config_path>/log4j2.xml. For details on log4j2 configuration, see Log4j Configuration.

Proxy Log

By default, proxy log entries are logged to <wavefront_log_path>/wavefront.log. The log file is rolled over every day and when its size reaches 100MB. When there are 31 log files, older files are deleted.

Blocked Point Log

You can log raw blocked points in a separate log from the proxy log. Logging of blocked points is disabled by default. To enable logging block points, edit the log4j2 configuration file and uncomment the blocked points file appender:

<!--
    <AppenderRef ref="BlockedPointsFile"/>
-->

By default, blocked point entries are logged to <wavefront_log_path>/wavefront-blocked-points.log and the block point log file is rolled over every day and when its size reaches 100MB. When there are 31 log files, older files are deleted.

Configuring a Proxy in a Container

You can use the in-product Docker with cAdvisor or Kubernetes integration if you want to set up a proxy in a container. You can then customize that proxy.

Proxy Versions for Containers

For containers, the proxy image version is determined by the image property in the configuration file. You can set this to image: wavefronthq/proxy:latest, or specify a proxy version explicitly. The proxies are not stateful. Your configuration is managed in your yaml file. It’s safe to use proxy:latest – we ensure that proxies are backward compatible.

Restricting Memory Usage for the Container

To restrict memory usage of the container using Docker, you need to add a JAVA_HEAP_USAGE environment variable and restrict memory using the -m or --memory options for the docker run command. The container memory contraint should be at least 350mb larger than the JAVA_HEAP_USAGE environment variable.

To restrict a container’s memory usage to 2g with Docker run:

docker run -d --name wavefront-proxy ... -e JAVA_HEAP_USAGE="1650m" -m 2g ...

To limit memory usage of the container in Kubernetes use the resources.limits.memory property of a container definition. See the Kubernetes doc

Customizing Proxy Settings for Docker

When you run a Wavefront proxy inside a Docker container, you can tweak proxy configuration settings that are properties in the wavefront.conf file directly from the Docker run command. You use the WAVEFRONT_PROXY_ARGS environment variable and pass in the property name as a long form argument, preceded by --.

For example, add -e WAVEFRONT_PROXY_ARGS="--pushRateLimit 1000" to your docker run command to specify a rate limit of 1000 pps for the proxy.

See the Wavefront Proxy configuration file for a full list.

Installing Proxies on Multiple Linux Hosts

Ansible is an open-source automation engine that automates software provisioning, configuration and management, and application deployment. The Wavefront Ansible role installs and configures the Wavefront proxy, which allows you to automate Wavefront proxy installation on multiple Linux hosts.

Note: In most cases, you install only one or two proxies in your environment. You don’t need a proxy for each host you collect data from. See Proxy Deployment Options.

For details, see the Setup tab in the Ansible built-in integration.