Breadcrumbs

High Availability Patching Checklist

This checklist can be used to record your progress during the High Availability patching process.

Pre-Patching Health Check

  • [ ] Conduct comprehensive system health check to ensure all components are online

  • [ ] Verify database integrity via Client Console for both EMDB and LogMart

  • [ ] Confirm all components are responding in Deployment Monitor within expected timeframes

  • [ ] Perform functional validation tests (AIE Alarms, Drilldowns, Search capabilities)

  • [ ] Review SQL maintenance job status in SQL Server Management Studio

Prerequisites

  • [ ] Verify current backups exist for all system databases (Master, MSDB, Model)

  • [ ] Disable all AV/EDR services before beginning patching

  • [ ] Remove node protection by declaring "out of service" using LifeKeeper (LK)

  • [ ] Unlock all mirrors via DataKeeper (DK)

Patching Process Selection

  • [ ] Determine patching approach needed:

    • [ ] Lightweight Patching (Non-Mirrored Drives/OS only) - Allows for failover

    • [ ] Heavy Patching (Mirrored Drives/SQL) - Requires full downtime

Phase 1: HA2 Passive Node Patching

  • [ ] Start SQL Server instance on HA2 (ensure both SQL Server Agent and SQL Server services running)

  • [ ] Apply all available patches to HA2 node

  • [ ] Reboot HA2 if required by update process

  • [ ] Verify all volumes remain unlocked after patching/reboot

  • [ ] If volumes are locked, unlock them through the GUI or manually using command line:

    cd %extmirrbase%
EMCMD . UNLOCKVOLUME D
EMCMD . UNLOCKVOLUME L
EMCMD . UNLOCKVOLUME S

  • [ ] Confirm all patches installed successfully

  • [ ] Restart SQL Server instance on HA2, verifying both SQL Server Agent and services are operational

  • [ ] Review SQL Server error log for successful startup

Phase 2A: Lightweight Patching - HA1 Production Node

  • [ ] Review Event Logs on HA2 for warnings/errors to confirm normal operation

  • [ ] Use LifeKeeper GUI to bring Resource Hierarchy In Service on HA2

  • [ ] Review Event Logs and LogRhythm logs to confirm HA2 operating normally

  • [ ] Apply OS patches to HA1

  • [ ] Use LifeKeeper GUI to bring Resource Hierarchy In Service on HA1

  • [ ] Review Event Logs and LogRhythm logs to confirm HA1 operating normally

Phase 2B: Heavy Patching - HA1 Production Node

  • [ ] Confirm LifeKeeper is out of service and DataKeeper mirrors remain unlocked

  • [ ] Take system offline by stopping all services connected to database:

    gsv -displayname 'LogRhythm\*' | stop-service

  • [ ] Verify in SQL Job Activity Monitor that all services have stopped

  • [ ] Apply patches to HA1 node

  • [ ] After all reboots complete:

    • [ ] Restart LifeKeeper and DataKeeper services

    • [ ] Confirm all services on HA1 remain stopped

    • [ ] Execute: gsv -displayname 'LogRhythm*' | stop-service

  • [ ] Validate SQL Server starts successfully

  • [ ] Restart all services including distributed components:

    gsv -displayname 'LogRhythm\*' | start-service

  • [ ] Review SQL Server error log for absence of component errors

  • [ ] Lock mirrors in DataKeeper and verify synchronization has begun

Phase 3: Post-Patching Validation

  • [ ] Execute full health check to confirm all components online

  • [ ] Log in to Client Console to verify integrity of EMDB and LogMart databases

  • [ ] Access Deployment Monitor to confirm all core components heartbeating correctly

  • [ ] Perform additional functionality testing (AIE Alarms, Drilldowns, Searches)

  • [ ] Once DataKeeper mirror synchronization completes, restore LifeKeeper to "In Service"

  • [ ] Optional: Conduct High Availability failover test to validate functionality on both nodes

SIOS Updates (When Required)

Backup Server (HA2) Update

  • [ ] Exit LifeKeeper GUI and DataKeeper GUI on HA2

  • [ ] Run command to stop LifeKeeper services: C:\\LK\\bin\\lkstop

  • [ ] Run setup program to upgrade LifeKeeper

  • [ ] Install new LifeKeeper license if necessary

  • [ ] Run setup program to upgrade LifeKeeper for SQL

  • [ ] Run setup program to upgrade SIOS DataKeeper

  • [ ] Verify upgraded versions in Add/Remove Programs

  • [ ] Reboot HA2

Primary Server (HA1) Update

  • [ ] Exit LifeKeeper GUI and DataKeeper GUI on HA1

  • [ ] Run command to stop LifeKeeper services: C:\\LK\\bin\\lkstop

  • [ ] Run setup program to upgrade LifeKeeper

  • [ ] Install new LifeKeeper license if necessary

  • [ ] Run setup program to upgrade LifeKeeper for SQL

  • [ ] Run setup program to upgrade SIOS DataKeeper

  • [ ] Verify upgraded versions in Add/Remove Programs

  • [ ] Reboot HA1

  • [ ] Start LifeKeeper GUI and login to HA1

System Monitor Agent Updates

  • [ ] Remove Top Level HA ResTag to prevent failover

  • [ ] Access Secondary Node (HA2) and update/patch as required

  • [ ] Restore Top Level HA ResTag

  • [ ] Failover from HA1 to HA2

  • [ ] Ensure Secondary Node (HA2) has come back online and is heartbeating properly

  • [ ] Remove Top Level HA ResTag to prevent failover

  • [ ] Access Primary Node (HA1) and apply identical updates/patches

  • [ ] Restore Top Level HA ResTag

  • [ ] Failover from HA2 to HA1

  • [ ] Ensure Primary Node (HA1) has come back online and is heartbeating properly

  • [ ] Perform test failover to verify patching/upgrading is completed

Final Verification

  • [ ] Document all changes applied during patching process

  • [ ] Ensure all services are running correctly

  • [ ] Verify all system functionality is restored

  • [ ] Check Deployment Monitor for any warning indicators

  • [ ] Close change management documentation