AWS S3 Beat

Version 6.2.3

Amazon Web Services (AWS) is a comprehensive cloud platform that offers more than 165 fully featured services, including:

Enterprise file storage
File sharing
Environment segregation
Identity and access management
Virtual machine hosting

Logs provided by AWS include:

CloudTrail
Core AWS information
An audit trail for the AWS environment
User and group management
Virtual machine (VM) creation
CloudWatch
Alarms
Events
Logs
Metrics
Cloud Config
AWS resource inventory
Configuration history
Configuration change notifications

FAQ

Can I use a single instance of the S3 beat to collect from multiple AWS S3 regions?

Yes. When configuring the S3 beat, you may add multiple SQS:Region combinations. These must be accessible with the same set of secret/access keys.

Can I use multiple sets of secret/access keys with a single instance of the S3 beat?

No, an instance of the S3 beat has a 1:1 relationship with a set of secret/access keys. If you require a second set of secret/access keys for another SQS queue, you must create a second instance of the S3 beat.

Why does S3 beat fail with "NoCredentialProviders" error when using IMDSv2 on EC2?

When running the S3 beat in a Docker container on an EC2 instance with IMDSv2 enforced (HttpTokens: required), you must configure the EC2 instance metadata hop limit to allow container access.

Issue: IMDSv2 has a default hop limit of 1, which only allows the EC2 host to access the Instance Metadata Service. Docker containers require an additional network hop and will be blocked with the default setting.

Solution: Increase the hop limit to 2 or higher on your EC2 instance:

BASH

aws ec2 modify-instance-metadata-options \
    --instance-id <your-instance-id> \
    --http-put-response-hop-limit 2 \
    --http-endpoint enabled

This configuration allows the S3 beat container to access EC2 instance credentials while maintaining IMDSv2 security benefits.

Symptoms of incorrect configuration: - Error: NoCredentialProviders: no valid providers in chain - S3 beat fails to start when awsFlag is set to true - Works with IMDSv1 but fails with IMDSv2 enforced

For more information, see the AWS documentation on configuring IMDS.

The information provided about working with AWS may change as Amazon updates its products. For additional questions on working with AWS, see the official AWS documentation.