Configure the Azure Event Hub Beat Log Source in the SIEM

This section provides instructions for configuring the Azure Event Hubs log source in the LogRhythm SIEM using the log source virtualization template.

The Open Collector sends the output of every Beat to the Agent in a single syslog stream. The parent log source is a generic type: "Syslog - Open Collector." A log source virtualization template included with the LogRhythm Knowledge Base (KB) creates child log sources for each beat.

Prerequisites

• LogRhythm Client Console 
• LogRhythm Administrator Account

• Open the following ports:

  Direction	Port	Protocol	Source	Destination	Purpose
  Outbound	443	HTTPS	Event Hub Beat	*.windows.net	Azure collection from Event Hub
  Outbound	5671	AMQPS	Event Hub Beat	*.windows.net	Azure collection from Event Hub

Step 1: Configure Syslog Relay

This step explains how to configure the Syslog Relay. The Open Collector needs Syslog Relay for the following reasons:

• By default, the agent timestamps syslog messages as they come in. The timestamp in the SIEM should reflect when the log was generated, not when the agent received this log.
• An additional Syslog Relay Regular Expression is required to correctly extract the timestamp.

Beats configured using the JSON parsing method should use the regex relay outlined in the Configure Beats for JSON Parsing topic and skip this step.

To configure Syslog Relay:

1. Click the System Monitors tab.
2. Double-click the agent you will send the Open Collector syslog to.
3. Click the Syslog and Flow Settings tab.
4. Select the Enable Syslog Server check box, if it is not already selected.
5. Type the Open Collector IP Address in the Syslog Relay Hosts field on the left.

6. As the first line in the Syslog Relay Regular Expressions field, type the following regular expression:

   ^<(?<priority>\d{1,3})>\s*(?<message>(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})T(?<hour>\d{2}):(?<minute>\d{2}):(?<seconds>\d{2})(\.(?<ms>\d+))?Z?[-+]?[0-9:]{0,}\s.*)

7. Click OK.
   Here is an example of a configured Syslog Relay, where the Open Collector IP address is 10.3.0.1.
   

   

Step 2: Accept the Pending Log Source

After Open Collector logs are sent to the Windows System Monitor Agent, you need to accept the pending log source.

1. Click the Log Sources tab.
2. In the New Log Sources grid, select the Action check boxes of the following:

   • Log Source Type. Syslog - Open Collector

     

     Do not select the Azure specific log source types yet. You will do that in a later step.

   • Log Processing Policy. LogRhythm Default

3. Right-click the selection, click Actions, and then click Accept.

4. Select one of the following:

   • Click Customize and change the following as needed:
     

     • Collection System Monitor Entity

     • Log Message Processing Settings

     • Log Data Management and Processing Settings

     • Silent Log Message Source Settings

   • Click Default to select customized defaults that were previously selected. 

   • Select a default batch amount between 100 and 5000.

5. Click OK.
6. To see the newly accepted Log Source in the grid, click Refresh.

Step 3: Apply the Log Source Virtualization Template for Azure Event Hub Log Messages

Use the log source virtualization template included in the KB to create a log source specifically for Azure logs.

1. Double-click the open the newly accepted Open Collector Log Source.
   The Log Message Source Properties window appears.

2. Click the Log Source Virtualization tab.

3. Select the Enable Virtualization check box.

4. Click Create Virtual Log Sources.
   The Create Virtual Log Sources dialog box appears.

5. In the Log Source Virtualization Template drop-down menu, select Open Collector - Azure Event Hub.

6. Click Save.
   The confirmation prompt appears.
7. Click OK.
   New Log Sources appear in the grid as children of your parent log source.
   

   

   
   

   

Step 4: Apply the Log Source Virtualization Template for Azure Event Hub Beat Heartbeat Messages

Use the log source virtualization template included in the KB to create a log source specifically for Azure Event Hub heartbeat logs.

This step is not required for beats configured using the JSON Parsing method that have had Long-Running LRCTL configured so that their heartbeat status can be monitored in the Beats Grid in the Web Console UI.

1. Double-click the newly accepted Open Collector Log Source.
   The Log Message Source Properties window appears.
2. Click the Log Source Virtualization tab.
3. Select the Enable Virtualization check box.
4. Click Create Virtual Log Sources.
   The Create Virtual Log Sources dialog box appears.
5. In the Log Source Virtualization Template menu, select Open Collector - EventHubBeat Heartbeat.
6. Click Save.
   The confirmation prompt appears.
7. Click OK.
   New Log Sources appear in the grid as children of your parent log source.
   

   

   
   

   

Step 5: (Optional) Enable Silent Log Source Detection

Silent Log Source Detection tells you when one of your log sources has stopped reporting logs.

1. Double-click a child log source—for example, Syslog - Open Collector - Azure Event Hub.
   The Virtual Log Message Source Properties window appears.
2. Click the Additional Settings tab.
3. Select the Enable Silent Log Source Detection check box.
4. Configure warning and error intervals. LogRhythm recommends warning after 1 hour and error after 2 hours.
   

   

5. Click OK.
6. Click the Alarm Rules tab.
7. Search for LogRhythm Silent Log Source Error and ensure that the value in the Status column is Enabled.