Install the Open Collector

If you are upgrading an existing Open Collector, skip this procedure and go to Upgrade the Open Collector.

If you want to configure Open Collector using SIEM web console, skip this procedure and go to Configure Open Collector Connection to the SIEM.

If you would like to copy/paste these commands, using PUTTY or other SSH tools will give you better results than using the VM console.

1. Install WGET:

   CODE

   sudo yum install -y wget

2. Download the Open Collector Control Script:

   CODE

   wget https://raw.githubusercontent.com/logrhythm/versions/master/lrctl

3. Change permissions:

   CODE

   chmod +x lrctl

4. Initialize the Open Collector and start all components. 

   1. The following command installs Docker - Community Edition:

      CODE

      ./lrctl init

      

      If you plan to use a proxy server, additional configuration is necessary. For more information, see Configure Open Collector Via Proxy for steps to take before continuing with this installation.

      

      If installing on RHEL 8, modification of the firewalld.conf file is required before starting the Metrics service or the Open Collector. Please see the RHEL 8 installation page for details.

   2. Start the Metrics service:

      CODE

      ./lrctl metrics start

      

      The system will prompt you to configure centralized metrics, as a configuration must exist in order to start the Metrics service. Answer the question at the prompt in order to completely start the Metrics service.

   3. Start the Open Collector:

      CODE

      ./lrctl open-collector start

Configure the Open Collector

Type the following configuration information:

• System Monitor Host Name/IP: The hostname or IP address of your syslog-enabled LogRhythm SysMon agent.  SysMon agent must be network accessible from the Open Collector.
• System Monitor syslog Port: The port # on which your LogRhythm SysMon agent is configured to receive syslog messages. By default, this is port 514.
• Syslog Server Timezone: The the timezone your LogRhythm SysMon agent is in.  Important for ensuring timestamps on syslog messages are accurate.

• Syslog Transport Mechanism: The protocol with which you would like to send syslog messages to the LogRhythm SysMon agent.

  • • TCP - Transmission Control Transport. Recommended transport option when encryption is not required.
    • UDP - User Datagram Transport. Not recommended unless log size is small AND reliability is not a concern.
    • TLS - Transport Layer Security v1.2 over a TCP connection. Requires certificate.

Open Collector supports syslog collection on Linux SysMon agents from System Monitor version 7.9.0.8004 onwards. The Linux SysMon agent cannot be installed on the same machine as the Open Collector.

TLS is unsupported by Linux at this time.

For commands to inspect or edit configuration, see the configuration information in Open Collector Installation Tips.

(Optional) Configure Open Collector Advanced Properties

When configuring the Open Collector, you are prompted about whether you want to configure advanced properties. Advanced properties include two fields:

• Concurrency. The number of components to run in parallel. In the context of a multi-core server, cores are utilized more thoroughly with increased concurrency. This setting uses more system processing power and increases the performance of the Open Collector.
• Capacity. The size of the queue in an Open Collector component. Messages are passed from component to component after processing for each component is complete. Because certain components require less processing than others, there are queues in place for messages waiting to be processed.

You can also reach this prompt by running the following command:

CODE

./lrctl oc config edit

Recommended Advanced Properties Configuration Settings

The following configuration settings may improve performance of the Open Collector. For more on the individual components, see Understand the Open Collector Components.

Setting concurrency or capacity to unnecessarily high levels can cause system slowdown, and in some cases may crash the application. For more information on configuring these settings, see How Should I Configure Concurrency and Capacity in Advanced Properties?

Understand the Open Collector Components

• Lumberjack. The listener for ingesting raw beat logs. This component forwards messages to the demux.
• Demux (demultiplexer). Routes an input log to its corresponding pipeline. For example, an Azure log is routed to the Eventhub pipeline, a G Suite log to the G Suite pipeline, and so on.
• Pipelines. JQ pipelines where most of the transform logic lives. Raw messages are transformed to map to LogRhythm schema.
• Outputs. Formats the message to Linux syslog and sends it to the agent.

Validate the Installation

To validate that you have installed Open Collector correctly and initialized the Beats:

1. Validate that services are running with the following three commands:

   CODE

   ./lrctl open-collector status
   ./lrctl metrics status
   ./lrctl <beat name> status

2. View the metrics in Grafana.

   1. Open a supported browser (Chrome or Firefox), and type http://<opencollectorip>:3000.

      

      CentOS and Open Collector are command-line only, so you will need to access Open Collector Metrics from a browser on another machine. Ensure that port 3000 inbound is open on the Open Collector.

   2. In Grafana, go to Open Collector, and then Open Collector Overview.

      The default Open Collector Overview dashboard has three columns. Each column includes a Messages Per Second and a Counters (total) graph. The Pipelines and Output columns also have Errors graphs.

      • Left column. Input - a Beat is successfully sending logs to the Open Collector.
      • Middle column. Pipelines - the logs are matching our MDI.
      • Right column. Output - the logs are successfully sent to the System Monitor Agent.
   3. If data is flowing through the Open Collector, the graphs will be populated with data regarding total counts and MPS for various parts of the pipeline. Each graph has an information icon in the top-left corner. Point to this icon for a description of what each graph displays.