Learn about sending logs to Tanzu Observability.

You can send logs to the Wavefront proxy from your log shipper or directly from your application. The Wavefront proxy sends the log data to the Wavefront instance.

shows how data goes from the log shipper to the wavefront proxy and then to the Wavefront instance

Install a Wavefront Proxy

Our logging solution currently requires a Wavefront proxy and does not support direct ingestion. The Wavefront proxy accepts a JSON array payload over HTTP or HTTPS and forwards it to the Wavefront service.

Proxy System Requirements:
  • 2 CPUs
  • 4 GB memory
  • Additional proxy configuration settings: - name: JAVA_HEAP_USAGE value:2G - name: JVM_USE_CONTAINER_OPTS value: "false"
Proxy Kubernetes Requirements:
  • Request resources: 1 CPU and 2 GB memory
  • Limit resources: 2 CPUs and 4 GB memory
  • 1 GB heap memory

To install and configure a new proxy version 11.3 or later:

  1. Log in to your Wavefront instance and select Browse > Proxies.
  2. Click Add Proxy and follow the instructions on screen.
  3. Edit the wavefront.conf file to open the pushListenerPorts to receive logs from the log shipper.
    For example:
    • If you installed the proxy on Linux, Mac, or Windows, open the wavefront.conf file, uncomment the pushListenerPorts configuration property, and save the file. The port is set to 2878 by default.
    • If you installed the proxy on Docker, the command you use opens the pushListenerPorts and sets it to 2878.
  4. Optionally, uncomment or add other logs proxy configurations the wavefront.conf file.
  5. Optionally, configure preprocessor rules for logs in the preprocessor_rules.yaml file.
  6. Start the proxy.

Configure the Log Shipper

The log shipper sends your data to the Wavefront proxy. During Beta, we support the Fluentd log shipper, which scrapes and buffers your logs before sending them to the Wavefront proxy specified in the fluent.conf file.

If you want to use a different log shipper, contact technical support.

Prerequisite:

Add the VMware domain (*.vmware.com) to the allowlist in your environment. Because Tanzu Observability uses a VMware log cluster, you need to add the VMware domain to your allowlist to send log data successfully. If you want to narrow down the domain, contact your Tanzu Observability account representative.

Configure your log shipper:

  1. Install the log shipper. For example, install Fluentd.

  2. Configure the log shipper to send data to the Wavefront proxy.

    a. Add the hostname of the host where the proxy runs.

    b. Add the pushListenerPorts that you configured in the proxy.
    For example, edit the fluent.conf file to send data to a proxy as follows:

     <match wf.**>
       @type copy
       <store>
         @type http
         endpoint http://<proxy url>:<proxy port (example:2878)>/logs/json_array?f=logs_json_arr
         open_timeout 2
         json_array true
         <buffer>
           flush_interval 10s
         </buffer>
       </store>
     </match>
    
  3. As part of preprocessing, tag the logs with the application and service name to ensure you can drill down from traces to logs.
  4. (Optional) If you’re already using a logging solution, specify alternate strings for required and optional log attributes in the proxy configuration file. See also My Logging Solution Doesn’t Use the Default Attributes.

Limits for Logs

If logs exceed the maximum character limit for a message, tag, or value, the Wavefront proxy drops the logs. Ensure that your logs are within the given limits. See How Do I Track Data Blocked by the Wavefront Proxy?

Attribute Default Limit
Log message 20,000 characters
Log tags Low cardinality. Many of the recommendations in Optimizing Data Shape to Improve Performance apply.
128 characters per tag key
128 characters per tag value
100 tags per log

Learn More!