Troubleshoot the GMT Beat

Common Error Message

You may receive an error like this one:

2020-03-20T12:36:09.762Z ERROR beater/gmailmessagetracking.go:235 Not found: Dataset gmtlogsautomation:gmal_logs_7_days was not found in location US

If you see this error, cross-verify the dataset you are using. This is likely due to a misspelled dataset name.

No Error Logs and No Data

Run the following command sequence to run the beat in debug mode:

1. ./lrctl gmtb config export --outfile gmtbconfig.yml
2. vim gmtbconfig.yml
3. Change "logging.level: info" it to logging.level: debugand save it.
4. cat gmtbconfig.yml | ./lrctl gmtb config import
5. rm gmtbconfig.yml
6. ./lrctl gmtb restart
7. ./lrctl gmtb logs view

Logs will start coming in. Check for the following error:

Error: beater/gmailmessagetracking.go:238 Not found: Project gmtlogsautomation-test

If you see this error, verify that the dataset you are using is in the project ID you have used for GMT Beat configuration.

I Do Not See Data Older Than Seven Days

By default, only data from the past seven days is visible from the beat. If you want to see older data, the number of days can be increased up to 180 days by running the following command sequence:

1. ./lrctl gmtb config export --outfile gmtbconfig.yml
2. vim gmtbconfig.yml
3. Change "numbackdaysData: 7" to numbackdaysData: 180and save it.
4. cat gmtbconfig.yml | ./lrctl gmtb config import
5. rm gmtbconfig.yml
6. ./lrctl gmtb restart
7. ./lrctl gmtb logs

If you still do not see logs, contact LogRhythm Customer Support.

High BigQuery Costs or _PARTITIONTIME Errors

Symptoms: Unexpectedly high Google BigQuery costs, or errors mentioning _PARTITIONTIME in the logs.

Cause: Starting with version 6.0.6, the GMT Beat uses BigQuery partition filtering to reduce scan costs by ~95%. This optimization requires tables to be partitioned by date, which is the default behavior when using Google Workspace's automated log export to BigQuery.

Solution:

• High Costs: Ensure you are running the latest version of the GMT Beat (6.0.6 or later) which includes partition filtering optimizations. Older versions scan entire tables, resulting in significantly higher BigQuery costs.
• _PARTITIONTIME Errors: This indicates your BigQuery tables are not partitioned. Verify that:
  • Tables were created using Google Workspace's official "Google Workspace data export to BigQuery" service
  • The gapps-reports@system.gserviceaccount.com service account has Editor role
  • Tables follow the activity_* or gmail_log_* naming convention

If tables were manually created or migrated from an older setup, you may need to recreate them using Google's automated export service. 

Note: Tables created through Google Workspace's automated export are partitioned by default and require no additional configuration.

Other Problems

Ensure that the following table and schema information has not changed.

• The daily_ table is present by default. Do not delete this table.
• daily_20200319 is a general pattern for every table according to the date/time (for example, daily_yyyymmdd).
• The schema of the table is highly specific. Do not change it. For more information about the table schema, see https://support.google.com/a/answer/7230050.