Initialize the AWS S3 Beat

This section provides instructions for initializing the AWS S3 beat after configuration.

Prerequisites

  • Open Collector is installed. If you have not already installed it, follow the instructions in the Open Collector Installation and User Guide, and then return to this topic.

  • AWS queues are set up.

  • The event on the bucket is configured so that the actions performed on the bucket are sent to the queue from which the s3beat service is listening.

  • The queues and bucket are on the same region, and the proper permission has been granted to the queue.

  • The following port is open:

    Direction

    Port

    Protocol

    Source

    Outbound

    443

    HTTPS

    AWSS3 Beat

Initialize the Beat via Web Console (Recommended)

  1. Ensure that the Open Collector Connection to the SIEM (WebUI) setup has been completed.

  2. Ensure that the System Monitor Agent to which you intend to send these logs has been Configured for JSON Parsing.

Use either the Enable JSON Parsing on System Monitor Agents or the Enable JSON Parsing for an Existing System Monitor Agent sections at the above link to configure the System Monitor Agent for JSON Parsing.

  1. Follow the steps outlined in Add a Beat in the Web Console to create the Beat via the Web UI.

Initialize the Beat via Command Line (Legacy)

  1. Start the beat:

    ./lrctl s3beat start
    
  2. Select a new s3beat instance and enter a unique identifier for this instance.

  3. Tell the service whether you have deployed the s3beat service in AWS.

    • If you enter N, it is assumed that deployment is done on-premise (not in AWS), and the user must enter security credentials.

      1. Enter the access key of your s3beat AWS application, which you should have saved from the Configure AWS S3 section.

        The secret access key and access keys are saved in encrypted format.

      2. Enter the s3beat secret access key of your s3beat AWS application, which you should have saved from the Configure AWS S3 section.

    • If you enter Y, it is assumed that the deployment is done in AWS, so security credentials are not required.

  4. Enter your s3beat sqs queue and region in the format queuename:region. You can configure multiple queuename and region combinations.

  5. To save the configuration, type c.

  6. If you want to enable cross-account access of some objects, type y. Otherwise, press Enter and go to the next numbered step.

    1. Enter your s3beat ARN for the assume role cross account access in the mentioned format. You can enter multiple ARNs. When you have finished entering your ARNs, type c.

  7. If you want to provide multiline pattern regex, type y. Otherwise, press Enter.

  8. Enter the hostname or IP address of the System Monitor Agent that has been Configured for JSON Parsing, and then press Enter.

Use either the Enable JSON Parsing on System Monitor Agents or the Enable JSON Parsing for an Existing System Monitor Agent sections at the above link to configure the System Monitor Agent for JSON Parsing.

  1. Enter the port on which the System Monitor Agent is configured to listen for JSON data (the default is 5044), and then press Enter.
    The s3beat service started message appears.

  2. The configuration has been saved and the service has started successfully.

  3. Check the status of the service:

    ./lrctl s3beat status 
    

Default Config Values for S3Beat:

S. No.

Field Name

Default Value

1.

AWSAccessKeyID

User Provided

2.

HeartbeatInterval

5m0s 

3.

HeartbeatDisabled

false

4.

AWSSecretAccessKey

User Provided

5.

traits.inclusion

Blank

6.

QueueList

User Provided

7.

traits.exclusion

Blank

8.

MaxKeys

0

9.

AWSFlag

false

10.

multiline.pattern

User Provided

11.

multiline.negate

false

12.

multiline.match

after

13.

assumeRoleArn


This flag is the Assume role ARN from AWS for cross account access in the format:

arn:aws:iam::{Account-A-ID}:role/{Assume_role_name}

14.

assumeRoleFlag

false

This flag enables user to access cross account logs retrieval using Sts:AssumeRole for s3beat deployment.

15.

stsCredsExpirationTime

1hr

This flag sets the maximum session duration of IAM role assigned to ARN used for Assume role.

For commands to inspect or edit a configuration, see the configuration information in Open Collector Installation Tips.