Request Tracing Service

The Request Tracing Service provides tracing facilities for multiple protocols and process communications done by the components of deployed applications.

The service helps users to detect application slowness and performance degradation by logging requests that exceed a given threshold. The trace data from long-running requests gives insight to solving bottlenecks and other performance issues.

The service is available in both Payara Server and Payara Micro, though the service configuration is different in Payara Micro

The following types of requests are traced if the service is enabled:

  • JAX-RS endpoints

  • JAX-RS client calls

  • MicroProfile REST Client calls

  • Servlets (HTTP requests)

  • SOAP Web Services

  • WebSockets

  • EJB timers execution

  • Inbound JMS messages received by an MDB

  • JBatch jobs creation

  • Tasks executed by injected Managed Executors.

  • Remote Invocations of EJBs

Starting from release 5.181, the Request Tracing Service was given an overhaul - traces are now in an OpenTracing-style format using Spans, instead of the original event stream format. Practically, this means that instead of having an ordered sequence of time stamped events constituting a trace (with separate entered and exited events), you now have a set of related traces (now called spans) with individual durations and attributes.

The OpenTracing specification, found here, gives a more detailed description of what Spans are and the general trace format. Alternatively you can view a short summary here.

The Request Tracing overhaul also brought in the concept of sample rates, affording you extra options to limit the amount of tracing done. In Payara 5.181, there are two offerings: flat probability, and an adaptive solution. The flat probability sampling simply applies a percentage chance that a trace will not be traced. The adaptive solution allows you to configure a target number of traces to sample over a given time period (e.g. sample 6 traces every 10 minutes).

An option to perform this sampling before or after the threshold checks is offered, allowing you to either determine if a trace should be sampled at its inception, or only after determining that a trace has breached your configured thresholds respectively.

OpenTracing Support

In-line with our support for MicroProfile OpenTracing, Payara Server and Payara Micro have support for OpenTracing.io code instrumentation.

In a practical sense, this means that you can instrument your code using either the new @Traced annotation, or by injecting and using an OpenTracing Tracer via CDI. For full details, see the MicroProfile OpenTracing and OpenTracing.io documentation. The OpenTracing Java GitHub page also has some useful information about using the OpenTracing API.

The new MicroProfile @Traced annotation replaces the Payara Server and Payara Micro specific annotation provided previously, which will require you to either remove or refactor any code instrumented with the old annotation. The new annotation’s canonical name is org.eclipse.microprofile.opentracing.Traced.

API Version Implemented

Payara Server & Micro Version OpenTracing.io API Version

5.36.0 - 5.65.0

0.33

4.1.2.183 - 4.1.2.191; 5.183 - 5.35.0

0.31

4.1.2.182; 5.182

0.30

Features Not Currently Supported

We currently do not offer the following OpenTracing features:

  • Conversion to or from OpenTracing 0.31 or newer APIs