Performance Counters—LR Alarming and Response Manager
- Service name. LogRhythm Alarming and Response Manager (scarm)
- Runs on. Platform Manager
- Performance Object. LogRhythm ARM
The SMTP Notification counters apply only to notifications sent by the ARM. If the AIE Drill Down Cache is enabled, SMTP notifications are sent by the Notification Service. Notification Service metrics are located in C:\Program Files\LogRhythm\LogRhythm Notification Service\logs. The Notification Service settings can be modified in the Configuration Manager.
| Performance Counter | Description |
|---|---|
| # Alarms Insert DB Retries | The total number of alarm bulk insert DB retries. |
# Alarms Processed | The total number of unique alarms generated and processed by the ARM service since it was last started. |
| #ePO Notifications Processed | The total number of ePO notifications processed. |
# Events Processed | The total number of events processed by the ARM service since it was last started. |
# Failed SMTP | The total number of failed SMTP notifications (errors when sending an alarm to a LogRhythm user) processed by the ARM service since it was last started. |
# Failed SNMP | The total number of failed SNMP (trap) notifications (errors when sending a trap notification to an SNMP receiver) processed by the ARM service since it was last started. |
| # Notifications Logs Processed | The total number of notification logs processed. |
| # Notifications Processed | The total number of notifications processed. |
| # SmartResponse™ Processed | The total number of SmartResponse™ actions processed. |
# SMTP Notifications Sent | The total number of SMTP (email) notifications sent by the ARM service since it was last started. |
# SNMP | The total number of SNMP (trap) notifications sent by the ARM service since it was last started. |
| # Text Notifications Processed | The total number of text notifications processed. |
| % Realtime | How current the ARM is in processing the event stream. 100% means the events are being processed in real time. |
| Alarm Rule Hit % | The percentage of events matching one or more Alarm Rules. |
| Queue Count AIE Cache Drilldowns | The number of AIE Cache Drilldown items waiting to be sent. |
Queue Count Alarms | The number of alarms, new and existing, queued for processing and awaiting insertion into the EMDB. |
| Queue Count ePO Notifications | The number of ePO notifications, new and existing, queued for processing. |
| Queue Count Notification Logs | The number of notification logs, new and existing, queued for processing. |
| Queue Count Notifications | The number of notifications, new and existing, queued for processing. |
| Queue Count SmartResponse™ | The number of SmartResponse™ actions, new and existing, queued for processing. |
Queue Count SMTP | The number of SMTP (email) notifications, individual and batch, queued for notification. |
Queue Count SNMP | The number of SNMP trap notifications queued for notification. |
| Queue Count Text Notifications | The number of text notifications, new and existing, queued for processing. |
Rate Alarms Processed | The number of alarms, new and existing, processed per second. |
| Rate ePO Notifications Processed | The number of ePO notifications processed per second. |
Rate Events Processed | The number of events processed per second. |
| Rate Notification Logs Processed | The number of notification logs processed per second. |
| Rate Notifications Processed | The number of notifications processed per second. |
| Rate SmartResponse™ Processed | The number of SmartResponse™ actions processed per second. |
Rate SMTP Notifications Sent | The number of SMTP (email) notifications, individual and batch, sent per second. |
Rate SNMP Notifications Sent | The number of SNMP trap notifications sent per second. |
| Rate Text Notifications Processed | The number of text notifications processed per second. |
The SMTP Notification counters apply only to notifications sent by the ARM. If the AIE Drill Down Cache is enabled, SMTP notifications are sent by the Notification Service. Notification Service metrics are located in C:\Program Files\LogRhythm Notification Service\logs.
To investigate performance of the LogRhythm Alarming and Response Manager service, add the following performance counters to a perfmon console:
- # Failed SMTP Notifications. Happens on occasion but during normal operation there should be little or no SMTP Notification failures. If it shows an excessive number or is continuously increasing, there may be a problem with the LogRhythm ARM service sending email to the configured SMTP server. Look for LogRhythm diagnostic errors or warnings in the LogRhythm dashboard pertaining to the LogRhythm ARM service. Also, ensure that the scarm.ini has the appropriate SMTP configuration and that the configured SMTP server is reachable on the network and will accept email from the scarm service.
- # Failed SNMP Notifications. Typically only results from internal application errors. If it shows an excessive number or is continuously increasing, there may be a problem with the LogRhythmARM service. Look for LogRhythm diagnostic errors or warnings in the LogRhythm dashboard pertaining to the scarm service and additionally examine the local scarm.log file for errors.
- % Full Alarm Queue. Should always be at or near zero. If it continuously rises or hits 100% and stays, there may be a problem with the LogRhythm ARM service inserting alarm records into the EMDB. Check the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm ARM service and the EMDB. Also check the local scarm.log file for any error messages related to the EMDB connection.
- % Full SMTP Queue. Should always be at or near zero. If it continuously rises or hits 100% and stays, there may be a problem with the LogRhythm ARM service sending SMTP (email) notifications to the configured SMTP server(s). Also check the # Failed SMTP Notifications performance counter. If you find errors sending SMTP notifications, examine the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm ARM service. Also, ensure that the scarm.ini has the appropriate SMTP configuration and that the configured SMTP server is reachable on the network and will accept email from the LogRhythm ARM service.
- % Full SNMP Queue. Should always be at or near zero. If it continuously rises or hits 100% and stays, there may be a problem with the LogRhythm ARM service sending SNMP (trap) notifications to the configured SNMP manager(s). Also check the # Failed SNMP Notifications performance counter. If you find errors sending SNMP notifications, check the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm ARM service. Also, ensure the SNMP manager you are attempting to notify is correctly configured via the LogRhythm dashboard.
- % Realtime. Should always be at or near 100% meaning that the LogRhythm ARM service is processing events as quickly as they are written to the EMDB. If it drops below 100%, you may notice that alarms are delayed which could result from increased event volume or could indicate that there is a slowdown in the EMDB. Check the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm ARM service. Also, examine the application event logs and local scarm.log from the Platform Manager to see if there are any specific error or warning messages related to database connectivity or excessive timeouts.
- Queue Count Alarms. This counter should always be at or near zero. If this counter never recovers to zero or continuously increases then there may be a problem inserting new alarms into the Platform Manager database (EMDB). Examine the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm Alarming and Response Manager service and the Platform Manager database (EMDB). In addition, examine the local scarm.log file for any error messages related to the Platform Manager database (EMDB) connection.
- Queue Count SMTP. This counter should always be at or near zero. If it never recovers to zero or continuously increases, there may be a problem sending SMTP (email) notifications to the configured SMTP server. Also check the # Failed SMTP Notifications performance counter. If you find errors sending SMTP notifications, check the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm ARM service. Also, ensure the scarm.ini has the appropriate SMTP configuration and that the configured SMTP server is reachable on the network and will accept email from the LogRhythm Alarming and Response Manager service.
- Queue Count SNMP. Should always be at or near zero. If it never recovers to zero or continuously increases then there may be a problem sending SNMP (trap) notifications from the LogRhythm ARM service. Also examine the # Failed SNMP Notifications performance counter. If you find errors sending SNMP notifications, check the LogRhythm dashboard for any error or warning events pertaining to the LogRhythm ARM service. Also, ensure the SNMP manager you are attempting to notify is correctly configured via the LogRhythm dashboard.
- Rate Alarms Processed. Depends on the event volume and the number of alarms that are enabled. Generally it will only show activity when events processed by the LogRhythm ARM service generate and alarm for insertion into the EMDB. If it remains at zero and you expect alarms to be generated, check the other LogRhythm ARM performance counters to identify potential errors.
- Rate Events Processed. Depends on the event volume. Generally it should show activity shortly after events are inserted into the EMDB. If this counter remains at zero while events are being inserted into the EMDB examine the other LogRhythm ARM performance counters to identify potential errors.
- Rate SMTP Notifications Sent. Depends on the event volume as well as the number of alarms configured for SMTP notification. Generally it should show activity shortly after an alarm configured for SMTP notification is generated. If this counter remains at zero while alarms configured for SMTP notification are being generated examine the other LogRhythm ARM performance counters to identify potential errors.
- Rate SNMP Notifications Sent. Depends on the event volume as well as the number of alarms configured for SNMP notification. Generally it should show activity shortly after an alarm configured for SNMP notification is generated. If it remains at zero while alarms configured for SNMP notification are being generated, check the other LogRhythm ARM performance counters to identify potential errors.