Configure the PubSub Beat Log Source in the SIEM

This section provides instructions for configuring the PubSub log source in the LogRhythm SIEM using the log source virtualization template.

The Open Collector sends the output of every Beat to the Agent in a single syslog stream. The parent log source is a generic type: "Syslog - Open Collector." A log source virtualization template included with the LogRhythm Knowledge Base (KB) creates child log sources for each beat.

Prerequisites

• LogRhythm Client Console 

• LogRhythm Administrator Account

• Open the following port:

  Direction	Port	Protocol	Source
  Outbound	443	HTTPS	PubSub Beat

Step 1: Syslog Relay Configuration

This step explains how to configure the Syslog Relay. The Open Collector needs Syslog Relay for the following reasons:

• By default, the agent timestamps syslog messages as they come in. The timestamp in the SIEM should reflect when the log was generated, not when the agent received this log.
• An additional Syslog Relay Regular Expression is required to correctly extract the timestamp.

Beats configured using the JSON parsing method should use the regex relay outlined in the Configure Beats for JSON Parsing topic and skip this step.

To configure Syslog Relay:

1. Click the System Monitors tab.
2. Double-click the agent to which you will send the Open Collector syslog.
3. Click the Syslog and Flow Settings tab.
4. Select the Enable Syslog Server check box, if not already selected.
5. Type the Open Collector IP Address in the Syslog Relay Hosts field on the left.

6. Type the following regular expression as the first line in Syslog Relay Regular Expressions:

   ^<(?<priority>\d{1,3})>\s*(?<message>(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})T(?<hour>\d{2}):(?<minute>\d{2}):(?<seconds>\d{2})(\.(?<ms>\d+))?Z?[-+]?[0-9:]{0,}\s.*)

7. Click OK.
   Here is an example of a configured Syslog Relay, where the Open Collector IP address is 10.3.0.1.
   

   

Step 2: Accept the Pending Log Source

After Open Collector logs are sent to the Windows System Monitor Agent, you need to accept the pending log source.

1. Click the Log Sources tab.
2. In the New Log Sources grid, select the Action check boxes for the following:

   • Log Source Type. Syslog - Open Collector

     

     Do not select the PubSub-specific log source types yet. You will do that in a later step.

   • Log Processing Policy. LogRhythm Default

3. Right-click the selection, click Actions, and then click Accept.

4. Select one of the following:

   • Click Customize and change the following as needed:
     

     • Collection System Monitor Entity

     • Log Message Processing Settings

     • Log Data Management and Processing Settings

     • Silent Log Message Source Settings

   • Click Default to select customized defaults that were previously selected. 

   • Select a default batch amount between 100 and 5000.

5. Click OK.
6. To see the newly accepted Log Source in the grid, click Refresh.

Step 3: Apply the Log Source Virtualization Template for PubSub Log Messages

Use the log source virtualization template included in the KB to create a log source specifically for PubSub logs.

1. Double-click to open the newly accepted Open Collector Log Source.
   The Log Message Source Properties window appears.

2. Click the Log Source Virtualization tab.

3. Select the Enable Virtualization check box.

4. Click Create Virtual Log Sources.
   The Create Virtual Log Sources dialog box appears.

5. In the Log Source Virtualization Template menu, select Open Collector - GCP Pub Sub.

6. Click Save.
   The confirmation prompt appears.
7. Click OK.
   New Log Sources appear in the grid as children of your parent log source.
   

   

   
   

   

Step 4: Apply the Log Source Virtualization Template for PubSub Beat Heartbeat Messages

Use the log source virtualization template included in the KB to create a log source specifically for PubSub heartbeat logs.

This step is not required for beats configured using the JSON Parsing method that have had Long-Running LRCTL configured so that their heartbeat status can be monitored in the Beats Grid in the Web Console UI.

1. Double-click the newly accepted Open Collector Log Source.
   The Log Message Source Properties window appears.
2. Click the Log Source Virtualization tab.
3. Select the Enable Virtualization check box.
4. Click Create Virtual Log Sources.
   The Create Virtual Log Sources dialog box appears.
5. In the Log Source Virtualization Template menu, select Open Collector - PubSubBeat Heartbeat.
6. Click Save.
   The confirmation prompt appears.
7. Click OK.
   New Log Sources appear in the grid as children of your parent log source.
   

   

   
   

   

Step 5: (Optional) Enable Silent Log Source Detection

Silent Log Source Detection indicates when one of your log sources has stopped reporting logs.

1. Double-click a child log source—for example, Syslog - Open Collector - PubSub.
   The Virtual Log Message Source Properties window appears.
2. Click the Additional Settings tab.
3. Select the Enable Silent Log Source Detection check box.
4. Configure warning and error intervals. LogRhythm recommends warning after 1 hour and error after 2 hours.
   

   

5. Click OK.
6. Click the Alarm Rules tab.
7. Search for LogRhythm Silent Log Source Error and ensure that the value in the Status column is Enabled.