High Availability Patching Checklist

This checklist can be used to record your progress during the High Availability patching process.

Pre-Patching Health Check

• [ ] Conduct comprehensive system health check to ensure all components are online

• [ ] Verify database integrity via Client Console for both EMDB and LogMart

• [ ] Confirm all components are responding in Deployment Monitor within expected timeframes

• [ ] Perform functional validation tests (AIE Alarms, Drilldowns, Search capabilities)

• [ ] Review SQL maintenance job status in SQL Server Management Studio

Prerequisites

• [ ] Verify current backups exist for all system databases (Master, MSDB, Model)

• [ ] Disable all AV/EDR services before beginning patching

• [ ] Remove node protection by declaring "out of service" using LifeKeeper (LK)

• [ ] Unlock all mirrors via DataKeeper (DK)

Patching Process Selection

• [ ] Determine patching approach needed:

  • [ ] Lightweight Patching (Non-Mirrored Drives/OS only) - Allows for failover

  • [ ] Heavy Patching (Mirrored Drives/SQL) - Requires full downtime

Phase 1: HA2 Passive Node Patching

• [ ] Start SQL Server instance on HA2 (ensure both SQL Server Agent and SQL Server services running)

• [ ] Apply all available patches to HA2 node

• [ ] Reboot HA2 if required by update process

• [ ] Verify all volumes remain unlocked after patching/reboot

• [ ] If volumes are locked, unlock them through the GUI or manually using command line:

  CODE

  cd %extmirrbase%
  EMCMD . UNLOCKVOLUME D
  EMCMD . UNLOCKVOLUME L
  EMCMD . UNLOCKVOLUME S

• [ ] Confirm all patches installed successfully

• [ ] Restart SQL Server instance on HA2, verifying both SQL Server Agent and services are operational

• [ ] Review SQL Server error log for successful startup

Phase 2A: Lightweight Patching - HA1 Production Node

• [ ] Review Event Logs on HA2 for warnings/errors to confirm normal operation

• [ ] Use LifeKeeper GUI to bring Resource Hierarchy In Service on HA2

• [ ] Review Event Logs and LogRhythm logs to confirm HA2 operating normally

• [ ] Apply OS patches to HA1

• [ ] Use LifeKeeper GUI to bring Resource Hierarchy In Service on HA1

• [ ] Review Event Logs and LogRhythm logs to confirm HA1 operating normally

Phase 2B: Heavy Patching - HA1 Production Node

• [ ] Confirm LifeKeeper is out of service and DataKeeper mirrors remain unlocked

• [ ] Take system offline by stopping all services connected to database:

  CODE

  gsv -displayname 'LogRhythm\*' | stop-service

• [ ] Verify in SQL Job Activity Monitor that all services have stopped

• [ ] Apply patches to HA1 node

• [ ] After all reboots complete:

  • [ ] Restart LifeKeeper and DataKeeper services

  • [ ] Confirm all services on HA1 remain stopped

  • [ ] Execute: gsv -displayname 'LogRhythm*' | stop-service

• [ ] Validate SQL Server starts successfully

• [ ] Restart all services including distributed components:

  CODE

  gsv -displayname 'LogRhythm\*' | start-service

• [ ] Review SQL Server error log for absence of component errors

• [ ] Lock mirrors in DataKeeper and verify synchronization has begun

Phase 3: Post-Patching Validation

• [ ] Execute full health check to confirm all components online

• [ ] Log in to Client Console to verify integrity of EMDB and LogMart databases

• [ ] Access Deployment Monitor to confirm all core components heartbeating correctly

• [ ] Perform additional functionality testing (AIE Alarms, Drilldowns, Searches)

• [ ] Once DataKeeper mirror synchronization completes, restore LifeKeeper to "In Service"

• [ ] Optional: Conduct High Availability failover test to validate functionality on both nodes

SIOS Updates (When Required)

Backup Server (HA2) Update

• [ ] Exit LifeKeeper GUI and DataKeeper GUI on HA2

• [ ] Run command to stop LifeKeeper services: C:\\LK\\bin\\lkstop

• [ ] Run setup program to upgrade LifeKeeper

• [ ] Install new LifeKeeper license if necessary

• [ ] Run setup program to upgrade LifeKeeper for SQL

• [ ] Run setup program to upgrade SIOS DataKeeper

• [ ] Verify upgraded versions in Add/Remove Programs

• [ ] Reboot HA2

Primary Server (HA1) Update

• [ ] Exit LifeKeeper GUI and DataKeeper GUI on HA1

• [ ] Run command to stop LifeKeeper services: C:\\LK\\bin\\lkstop

• [ ] Run setup program to upgrade LifeKeeper

• [ ] Install new LifeKeeper license if necessary

• [ ] Run setup program to upgrade LifeKeeper for SQL

• [ ] Run setup program to upgrade SIOS DataKeeper

• [ ] Verify upgraded versions in Add/Remove Programs

• [ ] Reboot HA1

• [ ] Start LifeKeeper GUI and login to HA1

System Monitor Agent Updates

• [ ] Remove Top Level HA ResTag to prevent failover

• [ ] Access Secondary Node (HA2) and update/patch as required

• [ ] Restore Top Level HA ResTag

• [ ] Failover from HA1 to HA2

• [ ] Ensure Secondary Node (HA2) has come back online and is heartbeating properly

• [ ] Remove Top Level HA ResTag to prevent failover

• [ ] Access Primary Node (HA1) and apply identical updates/patches

• [ ] Restore Top Level HA ResTag

• [ ] Failover from HA2 to HA1

• [ ] Ensure Primary Node (HA1) has come back online and is heartbeating properly

• [ ] Perform test failover to verify patching/upgrading is completed

Final Verification

• [ ] Document all changes applied during patching process

• [ ] Ensure all services are running correctly

• [ ] Verify all system functionality is restored

• [ ] Check Deployment Monitor for any warning indicators

• [ ] Close change management documentation