The DX Upgrade Checker is used to determine if an existing DX cluster (Windows or Linux) can be upgraded to the latest version. This tool is integrated with the DX Installer and runs with the installer. It can also be run as an independent tool prior to or after an upgrade to clean up incompatible indexes.
Starting with LogRhythm 7.18, indexes created in Elasticsearch 5 (LR versions prior to 7.8) are no longer compatible. To verify if your DX cluster has any incompatible indexes, you can run the DX Upgrade Checker prior to performing your LogRhythm software upgrade to 7.18+.
Starting with LogRhythm 7.26, indexes created in Elasticsearch 6 (LR versions prior to 7.18) are no longer compatible. To verify if your DX cluster has any incompatible indexes, you can run the DX Upgrade Checker prior to performing your LogRhythm software upgrade to 7.26+.
When upgrading a DX cluster crossing these version boundaries (7.18 or 7.26), it is strongly recommended that you run the upgrade checker or manually check if your cluster has incompatible indexes. Without these checks, cleanup after upgrading is more difficult and will result in a significant outage. Do not run upgrades crossing these version boundaries with --skip-upgrade-checker flags.
Windows DX Cluster (XM)
The tool can be run from command line from any location on disk, or you can run the script below to manually check if you have any incompatible indexes. Download the zip and extract the executable and save to a local disk location. Open Powershell or a CMD prompt to execute commands.
Windows Script
# Query settings from local Elasticsearch
$uri = "http://localhost:9200/_all/_settings/index.version.created?expand_wildcards=all"
try {
$settings = Invoke-RestMethod -Uri $uri -Method Get
} catch {
Write-Error "Failed to connect to Elasticsearch: $_"
return
}
# Iterate through each index and evaluate compatibility
foreach ($index in $settings.PSObject.Properties) {
$indexName = $index.Name
$version = $index.Value.settings.index.version.created
# Determine compatibility based on leading digit of the index version
$status = "UNKNOWN"
if ($version) {
if ($version.StartsWith("6")) {
$status = "INCOMPATIBLE"
} elseif ($version.StartsWith("7")) {
$status = "COMPATIBLE"
}
}
# Print the output in the same format
Write-Host "$indexName $version $status"
Linux DX Cluster
The Upgrade Checker Tool can be installed, or you can run the script below to check for incompatible indexes prior to upgrading.
Linux Script
curl -s "localhost:9200/_all/_settings/index.version.created?expand_wildcards=all" | \
jq -r 'to_entries[] | "\(.key) \(.value.settings.index.version.created)"' | \
while read -r index version; do
case "$version" in
6*) status="INCOMPATIBLE" ;;
7*) status="COMPATIBLE" ;;
*) status="UNKNOWN" ;;
esac
echo "$index $version $status"
done
Linux Upgrade Checker Installation
The tool can be installed and run from any node in the DX cluster for pre-upgrade checks. If you need to run post-upgrade clean-up in a multi-node cluster, you will need to install the DX Upgrade Checker on every node. Download the rpm and install:
sudo yum localinstall upgradechecker-14.21.0.noarch.rpm
Commands
Help
.\UpgradeChecker.exe --help
Pre-Upgrade
To perform a Pre-Upgrade, check if your indexes are compatible to be upgraded to LogRhythm 7.26+. If your cluster is “upgrade ready,” at the bottom of the output you should see a result stating that you are okay to proceed:
.\UpgradeChecker.exe live-cleanup
2024-08-27 19:53:27 [INFO] - Did not find any es2, es5 or es6 indices
2024-08-27 19:53:27 [INFO] - You are okay to proceed with the upgrade
If your cluster contains ES2, ES5, or ES6 indexes, which are incompatible with LogRhythm 7.26+, you will see an output which indicates you cannot upgrade. To prep your cluster for the upgrade, you can run live-cleanup with the -d flag, which will remove any incompatible indexes.
.\UpgradeChecker.exe live-cleanup -d
Post-Upgrade
If you have upgraded your DX Cluster to LogRhythm 7.26+ and found that OpenSearch will not start due to the presence of incompatible indexes, you will see an output in the OpenSearch log similar to the one below, and the service will be continually restarting.
The index [[logs-2019-07-11/Dr_e-1QwRoq4pNwpLi3Xng]] was created with version [5.5.0] but the minimum compatible version is [7.0.0]. It should be re-indexed in OpenSearch 1.x before upgrading to OpenSearch 2.x
To recover your OpenSearch cluster, you must stop the service and remove the incompatible indexes from the data path, then restart the service.
Run the DX Upgrade Checker offline-cleanup destructive -d argument with the OpenSearch data path specified with the -i argument. The OpenSearch Data Path directory will vary depending on your specific deployment.
Windows Offline Cleanup
Run the following command:
C:\Program Files\LogRhythm\Data Indexer\tools\UpgradeChecker.exe offline-cleanup -d -i D:\LRIndexer
Linux Offline Cleanup
On Linux DXs, you must stop all Elasticsearch processes in the cluster prior to running the offline-cleanup. You must run the offline-cleanup on all nodes in the cluster prior to restarting Elasticsearch by following the steps below:
-
Stop Elasticsearch on all nodes in the cluster.
-
Run offline-cleanup using the code below on each node in the cluster.
/usr/local/logrhythm/tools/UpgradeChecker offline-cleanup -d -i /usr/local/logrhythm/db/elasticsearch/data
-
Restart Elasticsearch on all nodes in the cluster.