Breadcrumbs

Initialize the PubSub Beat

Prerequisites

  • The Open Collector is installed. If you have not already installed it, follow the instructions in the Open Collector Installation and User Guide, and then return to this topic.

  • To collect log from any GCP service, the services must be enabled on the GCP portal, and you should have one topic and subscription.

  • The following port is open:

    Direction

    Port

    Protocol

    Source

    Outbound

    443

    HTTPS

    pubsubbeat

Initialize the Beat via the Web Console (Recommended)

  1. Ensure that the Open Collector Connection to the SIEM (WebUI) setup has been completed.

  2. Ensure that the System Monitor Agent to which you intend to send these logs has been Configured for JSON Parsing.

Use either the Enable JSON Parsing on System Monitor Agents or the Enable JSON Parsing for an Existing System Monitor Agent sections at the above link to configure the System Monitor Agent for JSON Parsing.

  1. Follow the steps outlined in Add a Beat in the Web Console to create the Beat via the Web UI.

Initialize the Beat via Command Line (Legacy)

  1. In the Open Collector, run the following command:

    ./lrctl pubsubbeat start

  2. Use the Up and Down Arrow keys to select New pubsubbeat instance from the list, and then press Enter.

  3. Enter the unique identifier for this pubsubbeat instance, and then press Enter.

  4. Enter the GCP Project ID, and then press Enter.

  5. Enter the GCP Topic name, only the portion that appears after …./topics/, and then press Enter.
    For example, if your GCP console listed the Topic name as projects/datacollector-0000/topics/sample-topic, you would enter sample-topic.

  6. Enter the GCP Subscription name, only the portion that appears after /subscriptions/, and then press Enter
    For example, if your GCP console listed the Subscription name as projects/datacollector-0000/topics/sample-subscription, you would enter sample-subscription.

  7. Enter the hostname or IP address of the System Monitor Agent that has been Configured for JSON Parsing, and then press Enter.

Use either the Enable JSON Parsing on System Monitor Agents or the Enable JSON Parsing for an Existing System Monitor Agent sections at the above link to configure the System Monitor Agent for JSON Parsing.

  1. Enter the port on which the System Monitor Agent is configured to listen for JSON data (the default is 5044), and then press Enter.
    The pubsubbeat service started message appears.

  2. Check the status of the service to confirm that it’s running:

    ./lrctl pubsubbeat status

  3. (Optional) Edit the pubsubbeat configuration to update the values set above if needed. Ensure that you have all the needed information for each step available as you will need to re-enter it:

    ./lrctl pubsubbeat config edit

Default Config Values for Pub Sub Beat:

S. No.

Field Name

Default Value

1.

project

User Provided

2.

HeartbeatInterval

5m0s 

3.

HeartbeatDisabled

false

4.

CredentialsFile

User Provided

5.

Topic

User Provided

6.

Subscription.name

User Provided

7.

json.enabled

true

8.

json.add_error_key

true

9.

subscription.Create

true