Eclipse MicroProfile Fault Tolerance API

Provided version of the API: MicroProfile Fault Tolerance 3.0

Background

The Fault Tolerance API was created to help separate execution logic from execution. The execution can be configured with a number of fault tolerance policies.

A full overview for the reasoning behind the API can be found in the README for the source code repository.

Additions to the Fault Tolerance Spec

In addition to the functionality detailed in the spec, the following is also implemented in Payara Server and Payara Micro:

Configurable Executor Services

The managed and scheduled managed executor pool can be configured via the set-fault-tolerance-configuration asadmin command.

The executor service is used to execute methods annotated with @Asynchronous, whereas the scheduled executor service is used by the CircuitBreaker and Timeout interceptors for their timeout operations (scheduling the circuit breaker to be set to half open, and just timing out respectively).

Alternative @Asynchronous Annotations

The Payara specific configuration property MP_Fault_Tolerance_Alternative_Asynchronous_Annotations can be used to specify a comma separated list of fully qualified class names of those annotations that should have the same effect as FT’s @Asynchronous. These annotations do not have to be interceptor bindings.

For example:

MP_Fault_Tolerance_Alternative_Asynchronous_Annotations=javax.ejb.Asynchronous

Annotation Priority

Prior to release 5.192 the order that the fault tolerance annotations are invoked is as follows:

  • Fallback (SPECIAL CASE - you’ll ONLY get here if there is an exception kicking off the asynchronous thread)

  • Asynchronous

  • Fallback

  • Retry

  • Bulkhead

  • CircuitBreaker

  • Timeout

Since the asynchronous annotation is invoked first, if it’s combined with any other fault tolerance annotations they will be processed on the asynchronous thread.

Since release 5.192 interactions between annotations are handled as described by the 2.0 specification effectively nesting computation in the following way (skipping handling for annotations not present):

  1. Asynchronous

  2. Fallback

  3. Retry

  4. Circuit Breaker

  5. Timeout

  6. Bulkhead

  7. (calling annotated method; might be wrapped by other interceptors)

As specified the interceptor priority can be changed using the property mp.fault.tolerance.interceptor.priority affecting all annotations including alternative ones.

Fault Tolerance Configuration

Fault Tolerance can be configured by using Admin Console or Asadmin commands.

Using the Admin Console

To configure the Fault Tolerance in the Admin Console, go to Configuration → [instance-configuration (like server-config)] → MicroProfile → Fault Tolerance:

Set Fault Tolerance Configuration

Using Asadmin Commands

set-fault-tolerance-configuration

Usage
asadmin>set-fault-tolerance-configuration
  [--managedexecutorservicename=managedexecutorservicename]
  [--managedscheduledexecutorservicename=managedscheduledexecutorservicename]
  [--target=server-config]
Aim

Provides a way to set the configuration of the fault tolerance service of the targeted config.

Command Options
The options asyncmaxpoolsize and delaymaxpoolsize have been removed as they are no longer applicable due to the configurable managed executor pools being used.
Option Description Default Mandatory

managedexecutorservicename

The Logical JNDI name of the Managed Executor Service to look up. Changes to this configuration require a restart of the server to take effect.

java:comp/DefaultManagedExecutorService

no

managedscheduledexecutorservicename

The Logical JNDI name of the Managed Scheduled Executor Service to look up. Changes to this configuration require a restart of the server to take effect.

java:comp/DefaultManagedScheduledExecutorService

no

target

The target configuration object to apply the change to

server

no
Example
asadmin> set-fault-tolerance-configuration --managedexecutorservicename=java:comp/DefaultManagedExecutorService --target instance1

get-fault-tolerance-configuration

Usage

asadmin> get-fault-tolerance-configuration [--target=server-config]

Aim

Returns the current configuration options for the Fault Tolerance service on the targeted config.

Command Options
Option Description Default Mandatory

target

The config to get the fault tolerance configuration for.

server-config

no
Example
asadmin> get-fault-tolerance-configuration --target=instance1

Upgrading from MicroProfile 3.x to 4.x

MicroProfile 4.0 brings with it a number of changes to MicroProfile Fault Tolerance. There are two incompatible changes with the previous version of MicroProfile Fault Tolerance.

Metric Names and Scopes Changed

The metrics added automatically by MicroProfile Fault Tolerance have been updated to take advantage of support for metric tags which were added to MicroProfile Metrics in version 2.0. As a result, some information which was previously contained in the metric name is now instead included in tags.

In addition, metrics have moved from the application scope to the base scope for consistency with other MicroProfile specifications. Note that this means:

  • Metrics are now exported under /metrics and /metrics/base, instead of /metrics and /metrics/application as in previous versions.

  • In the JSON format, when metrics are retrieved from /metrics they appear in the base object rather than the application object.

  • In the OpenMetrics format, the names are prefixed with base_ instead of application_.

There unfortunately is not a workaround for this change, users must simply ensure that any dashboard or application making use of these metrics must be updated to look at the new locations.

Example

Old format:

application:ft.<name>.timeout.callsTimedOut.total

New format:

base:ft.timeout.calls.total{method="<name>", timedOut="true"}

Lifecycle of CircuitBreakers and Bulkheads Specified

In previous versions of MicroProfile Fault Tolerance, the lifecycle of CircuitBreaker and Bulkhead annotations was not specified. These fault tolerance strategies hold state between invocations, so their lifecycle is important for correct functioning.

CircuitBreaker and Bulkhead are now treated as singletons identified by the annotated bean class and the guarded method. This means that if a RequestScoped bean has a method annotated with CircuitBreaker and/or Bulkhead, all invocations of that method will share the same CircuitBreaker and/or Bulkhead state, even though each request will have a different instance of the annotated bean.

There is unfortunately no workaround for this change in behaviour.