DR Upgrade Procedure Checklist

This document provides a checklist to reference when upgrading your LogRhythm Disaster Recovery (DR) deployment.

This checklist is not intended to replace the LogRhythm DR Upgrade Guide, which contains important information about the upgrade process. It is strongly recommended that you perform your upgrade by following the detailed procedures in that guide, using this checklist to track your progress.

This checklist assumes that your deployment is not a standard deployment or in HA. For those deployments, use the standard upgrade or HA checklist.

Prepare for the Upgrade

• Get the latest version of the LogRhythm DR Upgrade Guide from docs.logrhythm.com (or download the PDF).
• Review the DR requirements.
• Review the upgrade requirements and considerations.
  
  • Scheduling the upgrade and planning downtime
  • FIPS Mode support
  • Core services and Client Console compatibility
  • SQL Server security hardening impacts
  • System Monitor Agent considerations
  • OS requirements for the Windows components
  • Microsoft .Net Framework 4.7.2 requirements
  • Web Console prerequisites
• Ensure that you have the required Administrator credentials.
  • Local Administrator privileges for the LogRhythm servers
  • SQL Server password for the LogRhythmAdmin account
  • SQL Server sa password for the LogRhythm Databases

  • LogRhythm Service accounts referenced within the Configuration Manager

    

    A list of these passwords can be found in the LogRhythm Default Passwords section of docs.logrhythm.com.
• Download software to upgrade a LogRhythm deployment.
  • Disaster Recovery Upgrade Tool
  • LogRhythm Database Upgrade Tool
  • LogRhythm Install Wizard
  • Linux Data Indexer Installer
    • Required only if you have a Linux Data Indexer
  • (Optional) System Monitor Packages
  • (Optional) Threat Intelligence Service
  • (Optional) TrueIdentity Sync Client
  • (Optional) SOAP API
• Within the DR Control application, verify that all required databases are in Synchronized or Synchronizing status.
• Record service credentials.
  • SQL Server
  • SQL Server Agent
  • LogRhythm Service Registry

• Request a LogRhythm license file at least one business day prior to upgrade.

  

  If you are upgrading to a new LogRhythm major version, a license file is required.
• Modify web.config for the LR API.
  • Required only if you are using the LR API
• Note Web Console environmental variables.
  • Required only if you are overriding the Configuration Manager settings on one or more Web Console servers
    
• Note Platform Manager IP, LogRhythm Web UI password, and login warning banner.
• Synchronize stored Knowledge Base
  • Required only if you have downloaded a Knowledge Base but have not yet synchronized it
• Configure the System Monitor service to Startup Type = Automatic.
• Verify deployment status in the LogRhythm Infrastructure Installer.
• Shut down antivirus and endpoint protection software.

• Exit all LogRhythm Client Consoles.

Upgrade the LogRhythm Deployment

• Stop the LogRhythm core services on Windows appliances.
  • Platform Manager Servers
  • • Alarming and Response Manager
    • Job Manager
    • AI Engine Cache Drilldown
  • Data Processor Servers
    
    • Mediator Server Service
  • AI Engine Servers
    
    • AI Engine
    • AI Engine Communication Manager
  • Web Console Servers
    
    • Web Services Host API
    • Web Indexer
    • Web Console UI
    • Web Console API
    • Case API
  • (Optional) Kibana
  

  • Exit all LogRhythm Client Consoles.
  • System Monitor Agents can remain running throughout the upgrade.
  • Any services not mentioned in the above list are not required to be stopped for the upgrade. The Database Upgrade Tool might stop a few additional services on the PM/XM, but this is expected behavior.

• Run the LogRhythm Database Upgrade Tool on the primary PM/XM.

• Run the DR Upgrade script on both servers.
• Upgrade the LogRhythm appliances.
  • Run the LogRhythm Install Wizard on the primary PM/XM.
  • Configure the remaining hosts by running the LogRhythm Infrastructure Installer package on the required appliances.
    • Platform Managers
      • Run the LogRhythm Infrastructure Installer package on the secondary PM/XM via the command line, specifying the dr-secondary flag: lrii_windows.exe /dr-secondary
    • Data Processors
    • AI Engine Servers
    • Web Console Servers

    • Data Indexers

      

      If you have Linux Data Indexers (DXs), run the LogRhythm Infrastructure Installer package when upgrading the DX. For more information see Upgrade the LogRhythm Data Indexer.
      • Update the hosts file to contain the hot and warm additions with the host entries.
      • Run the PreInstall.sh script from the DX node you want to run the upgrade from.
      • When prompted to enter the path location for the hosts file, enter the full path location—for example, home/logrhythm/Soft/hosts.

      • Run the installer with the hosts file argument.

        • An --es-cluster-name is required only for new install, not for an upgrade

    • (Optional) Configure additional servers (SOAP API Server).
    • Verify cluster status.
  • Run the LogRhythm Install Wizard on all remaining Windows appliances.
    
    • Secondary PM/XM
      • When the LogRhythm Infrastructure Installer opens, click Exit.
    • Data Processors
    • AI Engine Servers
    • Web Console Servers
• Perform post-upgrade procedures.
  • Restart upgraded systems.
  • Import the LogRhythm license file.
  • Start the LogRhythm components.

Verify the Upgrade

• Confirm that all LogRhythm services have started successfully.
  • Verify that only the services set to Startup Type = Automatic are started on the secondary PM/XM.

• Confirm that All Services Up appears in the Configuration Manager.

  

  This could take up to 5 minutes after the upgrade.
• Within the DR Control application, verify that all required databases are in Synchronized or Synchronizing status.
• Open a web browser on the primary PM/XM, enter <localhost>:3000, click Pipeline, and then click Mediator.
  
  • Confirm that all Data Processors are processing logs.
  • Change the dashboard time filter to Last 5 minutes.
• Enter <localhost>:3000, click Data Indexer, click Maintenance, and then confirm that the DX cluster is green or yellow.

• 

  • If you have a large deployment, the cluster may remain yellow for a significant amount of time while the indices come online.
  • Depending on the size of the DX cluster and how much data it contains, it could take several minutes for the cluster to turn from red to yellow or green after the upgrade. After it turns yellow, then indexing and searching capabilities should be reinstated.

• Enter <localhost>:3000, click Data Indexer, click Logs Indexing, and then confirm that logs are being indexing into the DX cluster.
• Enter <localhost>:3000, click AIE, click AIE Metrics, and then confirm that all LMs (DPs) are connected to the AIE servers required, and that the AIE servers are receiving and processing data.
• In the Web Console:
  
  • Verify that you can see data on your key dashboards.
  • Conduct a search in the Web Console with the following parameters:
    • Timeframe: Last 30 minutes
    • Filters: remove all filters
    • Repository: Logs

This check tests 90% of the SIEM's core functionality. If you get results, then the deployment's processing, indexing, and searching functionality are working. If you do not get results, then you may need to wait for the DX cluster to turn yellow before trying this search again.

• If you have any test AIE correlation rules set to trigger an alarm, generate one of these and verify that it appears in the Alarms tab of the Web Console.

  

  If you have already had AIE correlation rules trigger an alarm post-upgrade, then there is no need to complete this step

• (Optional) Complete a failover to confirm that the failover works on your new version.

  

  This does not need to be conducted at the same time as the upgrade.