Notification Service

Payara Server comes with a general Notification service that can be used to log events which come from other services, such as the JMX Monitoring Service, the HealthCheck service or the Request Tracing service.

Notifiers

The Notification service provides the ability to disseminate notification events through various channels or notifiers, which relay them to different destinations.

Filtering

All notifiers make use of a filtering system to accept or ignore notification events. Events are emitted with a severity level of INFO, WARNING, or SEVERE. Notifiers may be individually configured to filter out events which are below a certain severity level.

A notifier with a filter level of WARNING (the default) will disregard events at the INFO level, and accept event at the WARNING and SEVERE levels.

The following is a list of the default set of notifiers available in Payara Server.

Log Notifier

The log notifier is the only notifier that is enabled by default when the notification service is activated.

The default output for the log notifier is the configured instance’s log file, which is either the server.log file or cluster.log file, depending on the instance configuration. The log notifier only handles the output of notifications from the notification service sent to the log and is not responsible for any other log output.

Configuration

Because the log notifier is the default notifier for the notification service it can be configured on the same page as the notification service, as shown in the image below:

Log Notifier Configuration

For any new domain, the notification service will be disabled by default. It can be enabled through the admin console as shown above, or via the Asadmin CLI.

To configure the log notifier through the Asadmin CLI, use the set-log-notifier-configuration subcommand:

asadmin> set-log-notifier-configuration --enabled=true --dynamic=true

To get the current configuration of the log notifier, use the get-log-notifier-configuration subcommand:

asadmin> get-log-notifier-configuration

Enabled  Filter   Use Separate Log File
true     WARNING  false
Command get-log-notifier-configuration executed successfully.

JMS Notifier

The JMS notifier makes use of a JMS queue to send notification events from services such as the Request Tracing and HealthCheck services. These messages can then be consumed by a Message Driven Bean or any other JMS compatible client implementation.

Requirements

To use JMS as a notification target, you will need the following a JMS-compatible message broker.

The Payara Server Web Profile distribution does not come with JMS capabilities, so this notifier isn’t supported.
The easiest way to configure the JMS notifier is to use the embedded OpenMQ broker, since Payara Server already comes with it by default.

JMS Notifier Configuration

The notifier configuration requires following a few steps:

  1. Create a new JMS Queue to receive the notification messages from the notifier:

    Create JMS Queue

    To make this change via the Asadmin CLI, use the following subcommand which mirrors the above screenshot:

    asadmin create-jms-resource --enabled=true --property=Name=notifierQueue --restype=javax.jms.queue jms/notifierQueue

  2. Set the JMS Notifier settings:

    Configure JMS Notifier in Admin Console

    The above example uses the embedded OpenMQ broker’s default configuration.

    To make this change via the Asadmin CLI, use the following command which mirrors the above screenshot:

    asadmin set-jms-notifier-configuration --dynamic=true --enabled=true --contextFactoryClass=com.sun.enterprise.naming.SerialInitContextFactory --target=server-config --queueName=notifierQueue --url=localhost:7676 --connectionFactoryName=jms/_defaultConnectionFactory

Verifying the Configuration

For verification purposes, it may be useful to enable a service to push notifications through the JMS notifier to demonstrate that it is working. To do this, you will also need to setup a Message-Driven Bean (MDB) to consume the notifications and demonstrate that they are being stored in the queue.

  1. First, enable a service to push data through the notifier. For example, the HealthCheck service’s CPU metric can be configured to push CPU metrics to a notifier every 5 seconds:

    asadmin> healthcheck-configure --enabled=true --dynamic=true --enableNotifiers=jms-notifier
asadmin> healthcheck-configure-service --serviceName=healthcheck-cpu --enabled=true --dynamic=true --time=5 --unit=SECONDS

    This also configures the HealthCheck service to send notifications to the JMS notifier, which will cause the messages to be sent right away.

  2. To consume the messages using a JMS 2.0 compliant MDB, the following class will work for a pre-configured queue named notifierQueue:

    @MessageDriven(activationConfig = {
    @ActivationConfigProperty(propertyName = "destinationLookup", propertyValue = "jms/notifierQueue"),
    @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
    @ActivationConfigProperty(propertyName = "destination", propertyValue = "notifierQueue"),
})
public class NotificationConsumer implements MessageListener {

    @Override
    public void onMessage(Message message) {
        try {
            System.out.println("Message received: " + message.getBody(String.class));
        }
        catch (JMSException ex) {}
    }
}

  3. View the result of the MessageDrivenBean’s onMessage() command. In this example, the CPU metric of the health check service was configured to notify every 5 seconds, so the result of simply printing to System.out is following log entries:

    [2017-02-24T14:25:02.019+0000] [INFO] [] [fish.payara.nucleus.healthcheck.HealthCheckService] [tid: _ThreadID=151 _ThreadName=admin-thread-pool::admin-listener(9)] [timeMillis: 1487946302019] [levelValue: 800] [[Scheduling Health Check for task: CPUC]]

[2017-02-24T14:25:02.019+0000] [INFO] [] [fish.payara.nucleus.healthcheck.HealthCheckService] [tid: _ThreadID=151 _ThreadName=admin-thread-pool::admin-listener(9)] [timeMillis: 1487946302019] [levelValue: 800] [[Payara Health Check Service Started.]]

[2017-02-24T14:25:02.376+0000] [INFO] [] [] [tid: _ThreadID=48 _ThreadName=p: thread-pool-1; w: 3] [timeMillis: 1487946302376] [levelValue: 800] [[Message received: Health Check notification with severity level: INFO. (host:mike-payara, server:server, domain:domain1,instance:server) CPUC:Health Check Result:[[status=GOOD, message='CPU%: 1.45, Time CPU used: 3 seconds 797 milliseconds'']']]]

[2017-02-24T14:25:02.380+0000] [INFO] [] [] [tid: _ThreadID=50 _ThreadName=p: thread-pool-1; w: 5] [timeMillis: 1487946302380] [levelValue: 800] [[Message received: Health Check notification with severity level: SEVERE. (host:mike-payara, server:server, domain:domain1,instance:server) CPUC:Health Check Result:[[status=CRITICAL, message='CPU%: 109.71, Time CPU used: 7 milliseconds'']']]]

Asadmin Commands

Set the JMS notifier configuration

To set the JMS notifier configuration, the following asadmin command will reflect the configuration done in the previous section:

asadmin> set-jms-notifier-configuration --dynamic=true --enabled=true
  --contextFactoryClass=com.sun.enterprise.naming.SerialInitContextFactory
  --connectionFactoryName=jms/__defaultConnectionFactory
  --queueName=notifierQueue
  --url=localhost:7676
  --username=****
  --password=****
  --target=server-config
Get the JMS notifier configuration

To get the JMS notifier configuration using the Asadmin CLI, run the following command:

asadmin> get-jms-notifier-configuration

Enabled  Filter   Context Factory Class  Connection Factory Name  Queue Name  URL  Username  Password
true     WARNING  com.sun.enterprise.naming.SerialInitContextFactory  jms/__defaultConnectionFactory  notifierQueue  localhost:7676  myusername  mypassword
Command get-jms-notifier-configuration executed successfully.

It will return the details of the current JMS notifier configuration, like in the following example:

Event Bus Notifier

The Event Bus Notifier provides a way to send notifications from the notification service into the internal Payara Server event bus, which is based on Hazelcast distributed topics.

Notifications sent to the internal event bus using this notifier are intended to be received by internal server components and not by user applications. Currently, no built-in internal components consume event bus notifications yet and there are no available APIs that allow applications to consume these events.
The Data Grid (and Hazelcast by extension) must be enabled for the Event Bus to be available.

Event Bus Notifier Configuration

This notifier provides only one configuration option, the Topic Name, which is mandatory.

In the Admin Console, simply set the Topic Name to a desired name, which will be used to setup the Hazelcast topic that will receive the notifications.

Event Bus Configuration

Make sure that the "Enabled" box is ticked so that the notifier will be used. If you would like the changes to take effect without needing a restart, tick the "Dynamic" box as well.

To make these changes via the Asadmin CLI, use the following subcommand:

asadmin> set-eventbus-notifier-configuration --topicName=my-topic --dynamic=true --enabled=true

To check the current applied configuration from asadmin, run the command:

asadmin> get-eventbus-notifier-configuration

Enabled  Filter   Topic Name
false    WARNING  payara.notification.event

Command get-eventbus-notifier-configuration executed successfully.

CDI Event Bus Notifier

The CDI Event Bus notifier provides a way to send notifications from the notification service into the internal Payara Server’s CDI Event bus. These notifications consist of CDI asynchronous events that can be consumed by user applications with valid CDI beans set up as listeners.

CDI Event Bus Notifier Configuration

You can configure the CDI Event Bus notifier from the Admin Console like this:

CDI Event Bus Configuration

The following options are available:

Enabled

Enables/disables the notifier.

Dynamic

Applies changes to the notifier without a server restart.

Loop Back

Whether events should also be observed on the same instance that triggered the instance or not.

To make these changes via the Asadmin CLI, use the following subcommand:

asadmin> set-cdieventbus-notifier-configuration --loopBack=true --dynamic=true --enabled=true --hazelcastEnabled=true

To check the current applied configuration from asadmin, run the command:

asadmin> get-cdieventbus-notifier-configuration

Enabled  Filter   Loopback
false    WARNING  false
Command get-cdieventbus-notifier-configuration executed successfully.

Observing Notification Events

Any application deployed to any instance in the Data Grid can observe notification events triggered by the CDI Event bus notifier.

Event messages are instances of the EventbusMessage class, which provides structured data about the specific event type, such as HealthCheckNotificationData or RequestTracingNotificationData. It also provides the same information in a String form in the title and message fields.

Notification events can be observed as a standard @Inbound CDI event of type EventbusMessage or its super-types:

public void observe(@Observes @Inbound EventbusMessage event) {
    String shortInfo = event.getSubject()
    String detailedMessage = event.getMessage();

    String domainName = event.getDomain();
    String sourceInstanceName = event.getInstance();

    if (event.getData() instanceof HealthCheckNotificationData) {
        Optional<HealthCheckResultEntry> mostCritical = event.getData()
        .as(HealthCheckNotificationData.class).getEntries()
        .stream().sorted().findFirst();
    }
}
For an application to use the Inbound annotation and EventbusMessage class, it will have to define the Payara Public API as a dependency.

Notification Service Configuration Commands

The following Asadmin CLI subcommands can be used to configure the notification service in general.

notification-configure

Usage

asadmin> notification-configure --enabled=true --dynamic=true

Aim

Enables or disables the notification service.

Command Options

Option Type Description Default Mandatory

--enabled=true

Boolean

Enables or disables the service

False

No

--dynamic=true

Boolean

When set to true, applies the changes without a restart. Otherwise a restart is required.

False

No
The argument --dynamic=true is necessary to turn on the service for a running server, otherwise the change would only be applied after a server restart.

Example

asadmin> notification-configure --enabled=true --dynamic=true

list-notifiers

Usage

asadmin> list-notifiers

Aim

Lists all available notifiers. These can then be configured individually or referenced by other service commands, like for example the Request Tracing Configuration commands.

Command Options

There are no available options for this command.

Example

Here’s an example on the output generated by the command:

asadmin>  list-notifiers
Available Notifier Services:
        cdieventbus-notifier
        eventbus-notifier
        log-notifier
        jms-notifier

Command list-notifiers executed successfully.

get-notification-configuration

Usage

asadmin> get-notification-configuration

Aim

This command can be used to view the current configuration of the notification service.

Command Options

There are no available options for this command.

Example

Running the command will give output similar to the following:

asadmin>
Enabled  Notifier Enabled  Notifier Name
false    false             service-log

Command get-notification-configuration executed successfully.

set-notification-configuration

Usage

asadmin> set-notification-configuration --enabled=true --dynamic=true --notifierEnabled=true --notifierDynamic=true --useseparatelogfile=true

Aim

This command can be used to set the configuration of the Notification Service and the Log Notifier at the same time.

Command Options

Option Type Description Default Mandatory

--enabled=true

Boolean

Enables or disables the service

false

No

--dynamic=true

Boolean

When set to true, applies the changes without a restart. Otherwise a restart is required.

false

No

--notifierEnabled

Boolean

Enables or disables the log notifier

false

Yes

--notifierDynamic

Boolean

When set to true, applies the changes to the log notifier without a restart. Otherwise a restart is required.

false

No

--useseparatelogfile

Boolean

When set to true, prints notifications to the configured log file

false

No

Example

asadmin> set-notification-configuration
    --enabled=true
    --dynamic=true
    --notifierEnabled=true
    --notifierDynamic=true
    --useseparatelogfile=false

Removing Notifiers

All Payara notifiers are included in Payara Enterprise by default, however they are fully pluggable and can be removed from your Payara installation. Each notifier consists of 2 parts, a notifier-core and a notifier-console-plugin. The notifier core is responsible for the notifier functionality and the asadmin commands, the notifier-console-plugin is not essential, but allows for the notifier to be configured through the admin console.

To remove a notifier from Payara Enterprise edition, first find the relevant modules to the notifier you want to remove in ${PAYARA_HOME}/glassfish/modules. To fully remove the notifier, you should delete both the notifier-core and the notifier-console-plugin files, then restart the domain.

If you change your mind in the future, you can just drop both the notifier-core and optionally the notifier-console-plugin files back into the ${PAYARA_HOME}/glassfish/modules directory and restart the domain.

See Also