HA Logical Configuration
LogRhythm Professional Services will:
- Install SIOS Protection Suite
- Modify LogRhythm for operations in a HA environment
- Create communications paths between the two nodes in the cluster
- Create dependencies between the resources in the LogRhythm Hierarchy.
- Create and extend Resource Hierarchies needed to run LogRhythm
- Test switchover – planned outages
- Test failover – unplanned outages
- Test HA Rules and Alarms
Backup Management
The LogRhythm HA solution delivers services, databases, and resources that are highly resilient and capable of operating transparently on two physically separate servers. However, you should still perform regular backups of the LogRhythm Archives and the LogRhythm EMDB at a minimum to protect against data corruption.
Service Management
After LogRhythm HA is installed and configured, the services on the system are managed differently than a stand-alone installation. Services and resources that are protected must be managed through the SIOS LifeKeeper GUI rather than the LogRhythm Console or the Windows Services MMC/Computer Management MMC.
To stop a protected service:
- Select the service in the LifeKeeper GUI.
- Right-click and select Out of Service.
- Respond to the confirmation prompt.
To start a protected service:
- Select the service in the LifeKeeper GUI.
- Right-click and select In Service.
- Respond to the confirmation prompt.
To bring a Resource Hierarchy In Service on the other node in the cluster:
- Select the Resource Hierarchy on the Standby node.
- Right-click and select In Service.
- Respond to the confirmation prompt.
All protected services are set to Manual Startup type. This is necessary to facilitate intelligent services starts and stops controlled by LifeKeeper HA software. Changing the startup type can result in unwanted results and should only be done under the guidance of LogRhythm Support.
Use of LifeKeeper to manage services is required. Using any tool other than the SIOS GUI to manage services is not recommended and will cause unpredictable results.
The services that are protected by the SIOS are listed in the following table. Not all services are present on all appliances.
| Windows Service Display Name | Windows Executable Name | SIOS Protection Suite Resource Tag Name |
|---|---|---|
| LogRhythm AI Engine | LRAIEEngine.exe | LRAIEEngine_ResTag |
| LogRhythm AI Engine Communication Manager | LRAIEComMgr.exe | LRAIEComMgr_ResTag |
| LogRhythm Alarming and Response Manager | scarm.exe | scarm_ResTag |
| LogRhythm DX - AllConf | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Cluster Templating Service (consul-template) | nssm.exe | LR_DX_ResTag |
| LogRhythm DX - Configuration Server | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Data Indexer Maintenance (GoMaintain) | nssm.exe | LR_DX_ResTag |
| LogRhythm DX - Elasticsearch Service | nssm.exe | LR_DX_ResTag |
LogRhythm DX - EM to DX Synchronization Service (Carpenter) | nssm.exe | LR_DX_ResTag |
| LogRhythm DX - Grafana Service | nssm.exe | LR_DX_ResTag |
LogRhythm DX - HTTP/REST interface to DX (Transporter) | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Index Query Service (Columbo) | nssm.exe | LR_DX_ResTag |
| LogRhythm DX - InfluxDB Service | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Metrics Collection Service (Vitals) | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Service Monitoring (HeartThrob) | nssm.exe | LR_DX_ResTag |
| LogRhythm DX - SQL Writer Service (Bulldozer) | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Threat Analytics Data Receiver (WatchTower) | nssm.exe | LR_DX_ResTag |
LogRhythm DX - Threat Analytics Data Splitter (Spawn) | nssm.exe | LR_DX_ResTag |
| LogRhythm Job Manager | nssm.exe | LR_DX_ResTag |
| LogRhythm Job Manager | lrjobmgr.exe | lrjobmgr_ResTag |
| LogRhythm Mediator Server Service | scmedsvr.exe | scmedsvr_ResTag |
| LogRhythm System Monitor Service | scsm.exe | scsm_ResTag |
| LogRhythmAPIGateway | LogRhythmAPI Gateway.exe | LogRhythmApIGateway_ResTag |
| LogRhythmServiceRegistry | LogRhythmService Registry.exe | LogRhythmServiceRegistry_ResTag |
Volume Management
Shared Volumes are required.
With LogRhythm High Availability installed and configured, the replicated volumes on the system are managed differently from a stand-alone installation. The target volume on the Standby system is not accessible to the operating system. For instance, if you select a protected volume in Windows Explorer, you get an Access is denied error. This expected behavior ensures that only the replication engine writes to the target volume. Protected and replicated volumes are in the following table. Not all drives are present on all appliances.
| Drive Letter | Contains |
|---|---|
| D: | Data |
| L: | SQL Logs |
| T: | Temp |
SQL Management
Shared SQL is required.
SQL Management is also managed differently from a stand-alone installation. As mentioned earlier, you manage the service start and stop functions through the LifeKeeper GUI. When you connect to the SQL database, you use the shared name rather than the node name thus relieving you of having to know on which node the system is active.
Use of SQL Management tools to control SQL and SQL-dependent services will cause unpredictable results.
The SQL processes that are protected are in the following table.
| SQL Process | Description |
|---|---|
| SQL Server (MSSQLSERVER) | Provides storage, processing and controlled access of data and rapid transaction processing. |
SQL Server Agent (MSSQLSERVER) | Executes jobs, monitors SQL Server, and fires alerts, and allows automation of some administrative tasks. |
Distributed Transaction Coordinator | Coordinates transactions that span multiple resource managers, such as databases, message queues |
The SQL databases that are protected are in the following table.
| SQL Database | Database Path | Log File Path |
|---|---|---|
| Data | D:\Data | L:\System |
| Master | D:\System\ | L:\System\ |
| Model | D:\System\ | L:\System\ |
| Msdb | D:\System\ | L:\System\ |
| LogRhythm_Alarms | D:\LogRhythm\ | L:\LogRhythm\ |
| LogRhythm_CMDB | D:\LogRhythm\ | L:\LogRhythm\ |
| LogRhythm_Events | D:\LogRhythm\ | L:\LogRhythm\ |
| LogRhythm_LogMart | D:\LogRhythm\ | L:\LogRhythm\ |
| LogRhythmEMDB | D:\LogRhythm\ | L:\LogRhythm\ |
SQL Privileged Account Management
LifeKeeper uses the sa account during setup. If your environment requires that you change the sa account, LifeKeeper must be also be updated with the new credentials.
For information on the steps required to change the sa password in LogRhythm, LogRhythm HA Administration Guide.
Once complete, perform the following procedures to update LifeKeeper:
- From the LifeKeeper GUI, select SQL_ResTag Resource Hierarchy.
- Select Properties.
- Select Admin Actions.
- Select Manage User.
- Click Next.
- Select Change Password or Change User and Password, depending on your requirements.
- Enter the new Password and/or User Name.
- Click Done.
IP Address Management
A Shared IP Address is required.
The IP Address Resource Hierarchy requires little additional management.
The shared IP Address is added as a secondary IP Address on the system with the active Resource Hierarchy. The IP address that is specified in the IP Address Resource Hierarchy is the one that all the other Resource Hierarchies on the system will use. For example, if you enable the syslog function in the LogRhythm System Monitor, you must use the shared IP Address, rather than the node IP address.
For more information, see Configure System Monitor for HA Deployments.
Windows (LANMAN) Name Management
A Shared Windows Name is required.
The Shared Windows Name is similar to IP address management. However, the Shared Windows Name defined in LifeKeeper is the one that is used throughout the LogRhythm configuration performed during the Professional Services installation. Like the shared IP Address, this name is transferred between cluster nodes to provide seamless operation, regardless of which node is in an Active state.
All services, including SQL and LogRhythm, reference the Windows Name.
DNS Name Management
Shared DNS names are optional.
The Shared DNS Name performs the same function as the Windows Name, but operates for NS name resolution rather than NetBIOS name resolution.
Remote Host Event Collection for Cluster Nodes
Remote Host Event Collection for cluster nodes is required.
In the LogRhythm active/passive cluster, the System Monitor is active on only one system. However, it is responsible for collecting Windows Event Logs from both nodes.
To accommodate this, the System Monitor is configured to collect Windows Event Logs from each node in the cluster across the network as a Remote Host instead of attempting local collection.
This provides flexibility to the HA System Monitor to collect Windows Event Logs from the persistent IP Addresses on each node reliably and consistently.
Shutdown Management
The LogRhythm HA implementation is configured so that a planned shutdown of the OS does not initiate a failover to the Standby node. However, if the Active node experiences an unplanned shutdown, the Standby node detects that the Active system is no longer available and brings the protected Resource Hierarchy to an Active state.
Recovery from Outages
When a node in the LogRhythm HA solution experiences a fault, you should reboot the system that experienced the failure to ensure that it has successfully joined the cluster. Failure to do so can result in unpredictable results and may prevent the HA cluster from successfully switching or failing over to that faulted node.
When you view the Standby system, all Resource Hierarchies should be shown in a Standby state, or Mirroring state for volumes. If any Resource Hierarchies display a Failed status, reboot the system to clear the error. If rebooting does not clear the error, contact LogRhythm Support.