Upgrade the LogRhythm Data Indexer
Upgrade a Single-Node Cluster
The script used to install or upgrade a Data Indexer cluster requires the use of a hosts file to assign a hostname to each Data Indexer node. If you used a hosts file when you first installed the Data Indexer or during a previous upgrade, point to the same file during the upgrade. If you have not previously used a hosts file, you need to create the file for the upgrade.
Before starting the Data Indexer installation or upgrade, ensure that firewalld is running on all cluster nodes. To do this, log on to each node and run: sudo systemctl start firewalld
- Log on to your Indexer appliance or server as logrhythm.
Change to the /home/logrhythm/Soft directory where you copied the updated installation or upgrade script.
If you need to create a hosts file, use
vi
to create a file in /home/logrhythm/Soft called hosts.If you are creating a new file, ensure that you specify the current Data Indexer hostname.The hosts file must follow a defined pattern of {IPv4 address}, {hostname}, {boxtype}(optional) on each line. You must separate the address and hostname with a space. The file might look like the following:
10.1.23.91 LRLinux1 hot
If you do not specify a boxtype here, it will assume it is a hot node. This means the warm node configuration may be lost if you do not update the hosts file prior to running the upgrade.Do not use fully qualified domain names for Indexer hosts. For example, use only LRLinux1 instead of LRLinux1.myorg.com.
The following command sequence illustrates how to create and modify a file with vi:
- To create the hosts file and open for editing, type vi hosts.
- To enter INSERT mode, type i.
- Enter the IPv4 address, hostname to use for the Indexer, and box type, separated by a space.
- Press Esc.
- To exit and save your hosts file type:
:wq
To install DX and make the machine accessible without a password, download the DataIndexerLinux.zip file from the Documentation & Downloads section of the LogRhythm Community, extract the PreInstall.sh file to /home/logrhythm and execute the script.
This cannot be run assudo
or the DX Installer will fail.CODEsh ./PreInstall.sh
Run the installer with the hosts file argument:
CODEsudo sh LRDataIndexer-<version>.x86_64.run --hosts <absolute path to .hosts file> --plan /home/logrhythm/Soft/plan.yml
Press Tab after starting to type out the installer name, and the filename autocompletes for you.
If prompted for the SSH password, enter the password for the logrhythm user.
The script installs or upgrades the Data Indexer.This process may take up to 10 minutes.When the installation or upgrade is complete, a confirmation message appears.
Check the status of services by typing sudo systemctl at the prompt, and then look for failed services.
If the installation or upgrade fails with the error — failed to connect to the firewalld daemon — ensure that firewalld is running on all cluster nodes and start this procedure again. To do this, log in to each node and run the following command:sudo systemctl start firewalld
Once the cluster restarts, there will be a short period of downtime as the DX update finalizes.
Upgrade a Multi-Node Cluster
The script used to install or upgrade a Data Indexer cluster requires the use of a hosts file to assign a hostname to each Data Indexer node. If you used a hosts file when you first installed the Data Indexer or during a previous upgrade, point to the same file during the upgrade. If you have not previously used a hosts file, you need to create the file for the upgrade.
Before starting the Data Indexer installation or upgrade, ensure that firewalld is running on all cluster nodes. To do this, log in to each node and run the following command: sudo systemctl start firewalld
- Log on to your Indexer appliance or server as logrhythm.
Change to the /home/logrhythm/Soft directory where you copied the script.
You should have a file named hosts in the /home/logrhythm/Soft directory that was used during the original installation. The hosts file must follow a defined pattern of {IPv4 address}, {hostname}, {boxtype}(optional) on each line. You must separate the address and hostname with a space.
The contents of the file might look like the following:10.1.23.65 LRLinux1 hot
10.1.23.67 LRLinux2 warm
10.1.23.91 LRLinux3 warmThe box type parameter is optional in the hosts file, if you do not specify a boxtype here, it will assume it is a hot node. This means the warm node configuration may be lost if you do not update the hosts file prior to running the upgrade.If you need to create a hosts file, use
vi
to create a file in /home/logrhythm/Soft called hosts.Do not use fully qualified domain names for Indexer hosts. For example, use only LRLinux1 instead of LRLinux1.myorg.com.
The following command sequence illustrates how to create and modify a file with
vi
:- To create the hosts file and open for editing, type
vi hosts
. - To enter INSERT mode, type i.
- Enter the IPv4 address, the hostname to use for the Indexer, and the box type, separated by spaces.
- Press Esc.
- To exit and save your hosts file type
:wq
.
- To create the hosts file and open for editing, type
To install DX and make the machine accessible without a password, download the DataIndexerLinux.zip file from the Documentation & Downloads section of the LogRhythm Community, extract the the PreInstall.sh file to /home/logrhythm and execute the script.
This cannot be run assudo
or the DX Installer will fail.CODEsh ./PreInstall.sh
If there are any changes in the plan file (for example, altering hosts or version upgrades), you must copy the new plan.yml file (and remove the old plan.yml file) at /home/logrhythm/Soft.Run the installer using the original or updated hosts file:
CODEsudo sh LRDataIndexer-<version>.x86_64.run --hosts <absolute path to .hosts file> --plan /home/logrhythm/plan.yml
Press Tab after starting to type out the installer name, and the filename autocompletes for you.
If prompted for the SSH password, enter the password for the logrhythm user.
The script installs or upgrades the Data Indexer on each of the DX machines.This process may take up to 30 minutes.When the installation or upgrade is complete, a confirmation message appears.
Check the status of services by typing
sudo systemctl
at the prompt, looking for “failed” services.If the installation or upgrade fails with the error — failed to connect to the firewalld daemon — ensure that firewalld is running on all cluster nodes and start the installation again. To do this, log in to each node and run the following command:sudo systemctl start firewalld
Once the cluster restarts, there will be a short period of downtime as the DX update finalizes.
(New Install or Upgrade from LR pre-7.8 only) Upgrade Using the Data Indexer Node Installer
The LogRhythm Data Indexer (LRDX) Node Installer is available to users that have purchased a DX7500 or DX7600. The installer leverages the resources on large boxes to improve the indexing and TTL performance by adding a second Elasticsearch instance to each machine.
The LRDX Node Installer installs and adds the second instance of Elasticsearch on each Hot DX host in the cluster.
The LRDX Node Installer is needed to hit the specified performance numbers for the DX 7500/7600. Failing to run the DX Node Installer when using DX 7500/7600 hosts will result in 50% TTL reduction along with reduced indexing and searching performance.
Prerequisites
A CPU core of at least 50 and 124 GB of RAM are required for the LRDX Node Installer to run.
Upgrade an Existing DX 7500/7600
To upgrade a system running multiple nodes:
Upgrade the LogRhythm deployment using the standard upgrade documentation for the version of software you are deploying. For more information, see Upgrade a LogRhythm Deployment.
- Connect to the Data Indexer system as a LogRhythm user.
If you have not already, download the LRDataIndexer-.<version>.x86_64.run package installer to the Logrhythm user’s home directory on one of your Data Indexer appliances (for example, /home/logrhythm/Soft).
Run the LRDataIndexer with the host file created in the initial install:
CODEsudo sh LRDataIndexer-<version>.x86_64.run --hosts /home/logrhythm/Soft/hosts --plan /home/logrhythm/plan.yml
The hosts file must follow a defined pattern of {IPv4 address}, {hostname}, {boxtype}(mandatory) on each line. The file might look like the following:
10.1.23.91 LRLinux1 hot
The box type parameter is mandatory in the hosts file, if not designated the installer will fail with a —missing parameter— error.Run the LRDX Node Installer with the host file:
CODEsudo sh <installer> --hosts /home/logrhythm/Soft/hosts --add
After LR 7.6.0, it was not required to run node installer to upgrade, running the latest DX was sufficient to upgrade.When prompted for the SSH password, enter the password for the LogRhythm user.
Validate the Linux Indexer Upgrade
To validate a successful upgrade of the Linux Indexer, check the following logs in /var/log/persistent:
- ansible.log echoes console output from the upgrade, and should end with details about the number of components that upgraded successfully, as well as any issues (unreachable or failed)
- logrhythm-node-install.sh.log lists all components that were installed or updated, along with current versions
- logrhythm-cluster-install.sh.log should end with a message stating that the Indexer was successfully installed
Additionally, you can issue the following command and verify the installed version of various LogRhythm services, tools, and libraries, as well as third party tools:
sudo yum list installed | grep -i logrhythm
- Verify that the following LogRhythm services are at the same version as the main installer version:
- Bulldozer
- Carpenter
- Columbo
- GoMaintain
- Transporter
- Watchtower
- Verify that the following tools/libraries have been updated to the version matching the installer name:
- Cluster Health
- Conductor
- Persistent
- Silence
- Unique ID
- Upgrade Checker
- Verify the following versions of these services and third party tools:
- elasticsearch 6.8.3