(LRCloud Only) Initialize the AWS S3 Beat Using Cloud to Cloud Collection

This document explains how to initialize the AWS S3 Beat using cloud-to-cloud collection. This feature is available only to LRCloud customers.

Prerequisites

Before initializing the AWS S3 tracking beat using collection from Amazon, do the following:

• Make sure that the customer is an LRCloud customer and has their environment hosted.
• Check if Open Collector has been installed in the customer's LRCloud environment on a separate instance. If not, an Open Collector instance must be requested via a support case.
• Ensure that the Open Collector log source has been accepted and configured for log source virtualization.
• Check if you have the required keys: AWS Access Key, AWS Secret Access Key, and the SQS queue names.
• Multiline pattern regex is not required to parse out logs.

Apply the Log Source Virtualization Template  

1. Log in to the Client Console in Cameyo.
2. Click Deployment Manager from the toolbar.
3. Click the Log Sources tab.
4. Double-click the required Open Collector Log Source (such as, {instance}-opencollector.c.e3-hub-753dd405.internal Open Collector).
   The Log Message Source Properties window appears.
5. Click the Log Source Virtualization tab. 
6. If not checked, select the Enable Virtualization check box. 
7. Click Create Virtual Log Sources.
   The Create Virtual Log Sources dialog box appears. 
8. In the Virtual Log Sources menu, check the Action check box corresponding to the following items:
   • "Syslog - Open Collector - AWS S3"
     
   • "Syslog - Open Collector - AWS Guard Duty"
   • "Syslog - Open Collector -AWS Config Events"
   • "Syslog - Open Collector -AWS CloudWatch"
   • "Syslog - Open Collector -AWS CloudTrail"
   • "Syslog - Open Collector -AWS S3 CloudTrail"
9. Click  Save.
   The Virtual Log Source(s) created prompt appears.
10. Click Ok.
11. Click Apply. 
12. Click Ok.
    The new Log Sources will appear in the grid as children of your parent log source.
    
13. Click the System Monitors tab. 
14. Select the Action check box corresponding to the (customerid)-dpawc agent. 
15. Right-click the selection, click Actions and then click Service Restart.

Initialize the Beat

1. Log in to the Web Console as a Restricted Administrator User.
2. On the top navigation bar, click the Administration icon and select Cloud Log Collection.
3. At the top of the Cloud Log Collection page, click New Log Source.
   The New cloud log collection dialog box appears.
4. Select the AWS S3 - Open Collector tile.
   The Add Aws S3 Log Source window appears.
   

   

   
   
   

5. Enter the following details:

   Setting	Description
   Name	Enter the name for this log source.
   Description (Optional)	Enter a description for this log source.
   Access Key ID	The AWS access key (for example, ABCD12ABCDEF12ABCDEF).
   Secret Access Key	Enter the AWS secret access key (for example, aBDcef12AbDEF12/aBDcef12AbcDEF123ab/A1B2).
   SQS Queues	The SQS queue and region in the format: queuename:region For example, queue:us-east-2 If you want to monitor multiple queue name and region combinations, then it is suggested that you create multiple log sources.

6. Click Save.

7. Log in to the Client Console in Cameyo.
8. Click Deployment Manager from the toolbar.
9. Click the System Monitors tab. 
10. Select the Action check box corresponding to the dpwac agent.
11. Right-click the selection, click Actions and then click Service Restart.

A new log source is created with the provided information based on the virtualized log source that was already created. Collection should start automatically in few minutes.

The Open Collector hosts the log sources. It is recommended to create a new host entity and move the log source to the new host which is done in the log source properties screen and not from the log source grid.

For security, the values entered are encrypted using LRCrypt.

Default Config Values for AWS S3 Beat

Setting	Field Name	Default Value
1.	AWSAccessKeyID	User-provided
2.	HeartbeatInterval	5m0s
3.	HeartbeatDisabled	false
4.	AWSSecretAccessKey	User-provided
5.	traits.inclusion	Blank
6.	QueueList	User-provided
7.	traits.exclusion	Blank
8.	MaxKeys	0
9.	AWSFlag	false
10.	multiline.pattern	This feature is not currently in use for Cloud-to-Cloud collection.
11.	multiline.negate	false
12.	multiline.match	after
13.	assumeRoleArn	This flag is the Assume role ARN from AWS for cross account access in the format: CODE arn:aws:iam::{Account-A-ID}:role/{Assume_role_name}
14.	assumeRoleFlag	false This flag enables a user to access cross-account logs retrieval using Sts:AssumeRole for AWS S3 Beat deployment.
15.	stsCredsExpirationTime	1hr This flag sets the maximum session duration of the IAM role assigned to ARN used for the Assume role.