Configure Open Collector Connection to the SIEM (Legacy-Syslog)

Customers requiring custom beats, OCAdmin, or who are on LogRhythm SIEM versions prior to 7.14 should use the Syslog-based connectivity between the Open Collector and SIEM described in this guide. This is planned to be retired in the future as functionality for Open Collector administration is moved to the Web Console. However, it is still applicable for certain use-cases. 

Start and Configure the Open Collector

Legacy customers not requiring custom OC beats should proceed to Configure Open Collector Connection to the SIEM (WebUI).

1. Start the Open Collector:

   CODE

   ./lrctl open-collector start

2. Type the following configuration information:

• System Monitor Host Name/IP: The hostname or IP address of your syslog-enabled LogRhythm SysMon agent. The SysMon agent must be network accessible from the Open Collector.
• System Monitor syslog Port: The port # on which your LogRhythm SysMon agent is configured to receive syslog messages. By default, this is port 514.
• Syslog Server Timezone: The the timezone in which your LogRhythm SysMon agent is located.  Important for ensuring timestamps on syslog messages are accurate.

• Syslog Transport Mechanism: The protocol with which you would like to send syslog messages to the LogRhythm SysMon agent.

  • TCP - Transmission Control Transport. Recommended transport option when encryption is not required.
  • UDP - User Datagram Transport. Not recommended unless log size is small AND reliability is not a concern.
  • TLS - Transport Layer Security v1.2 over a TCP connection. Requires certificate.

Open Collector supports syslog collection on Linux SysMon agents from System Monitor version 7.9.0.8004 onwards. The Linux SysMon agent cannot be installed on the same machine as the Open Collector.

TLS is unsupported by Linux at this time.

For commands to inspect or edit configuration, see the configuration information in Open Collector Installation Tips.

(Optional) Configure Open Collector Advanced Properties

When configuring the Open Collector, you are prompted about whether you want to configure advanced properties. Advanced properties include two fields:

• Concurrency. The number of components to run in parallel. In the context of a multi-core server, cores are utilized more thoroughly with increased concurrency. This setting uses more system processing power and increases the performance of the Open Collector.
• Capacity. The size of the queue in an Open Collector component. Messages are passed from component to component after processing for each component is complete. Because certain components require less processing than others, there are queues in place for messages waiting to be processed.

You can also reach this prompt by running the following command:

CODE

./lrctl oc config edit

Recommended Advanced Properties Configuration Settings

The following configuration settings may improve performance of the Open Collector. For more on the individual components, see the Understand the Open Collector Components section of the Install the Open Collector topic.

Setting concurrency or capacity to unnecessarily high levels can cause system slowdown, and in some cases may crash the application. For more information on configuring these settings, see How Should I Configure Concurrency and Capacity in Advanced Properties?

Understand the Open Collector Components

• Lumberjack. The listener for ingesting raw beat logs. This component forwards messages to the demux.
• Demux (demultiplexer). Routes an input log to its corresponding pipeline. For example, an Azure log is routed to the Eventhub pipeline, a GSuite log to the GSuite pipeline, and so on.
• Pipelines. JQ pipelines where most of the transform logic lives. Raw messages are transformed to map to LogRhythm schema.
• Outputs. Formats the message to Linux syslog and sends it to the agent.

Validate the Installation

To validate that you have installed Open Collector correctly and initialized the Beats:

1. Validate that services are running with the following three commands:

   CODE

   ./lrctl open-collector status
   ./lrctl metrics status
   ./lrctl <beat name> status

2. View the metrics in Grafana.

   1. Open a supported browser (Chrome or Firefox), and type http://<opencollectorip>:3000.

      

      CentOS and Open Collector are command-line only, so you will need to access Open Collector Metrics from a browser on another machine. Ensure that port 3000 inbound is open on the Open Collector.

   2. In Grafana, go to Open Collector, and then Open Collector Overview.

      The default Open Collector Overview dashboard has three columns. Each column includes a Messages Per Second and a Counters (total) graph. The Pipelines and Output columns also have Errors graphs.

      • Left column. Input - a Beat is successfully sending logs to the Open Collector.
      • Middle column. Pipelines - the logs are matching our MDI.
      • Right column. Output - the logs are successfully sent to the System Monitor Agent.
   3. If data is flowing through the Open Collector, the graphs will be populated with data regarding total counts and MPS for various parts of the pipeline. Each graph has an information icon in the top-left corner. Point to this icon for a description of what each graph displays.