Working with the Generic Beat

Start and Monitor the Beat

When the beat configuration is saved, the beat is also started. To monitor the beat, run the following command:

./lrctl genericbeat logs

Modify the Generic Beat Configuration File

To change the credentials for the configuration file:

  1. Run the following command:

    ./lrctl genericbeat config edit
    


  2. Modify the configuration as desired.

  3. Once required fields have been added, restart the beat with the following command:

    ./lrctl genericbeat restart
    


Upgrade the Beat

To upgrade the Generic Beat to the latest version, run the following command:

./lrctl genericbeat restart

This will automatically apply migration to the Generic Beat.

Default Configuration for the Generic Beat

The following table displays the default configuration values for the fields in the Generic Beat.

S.NO

Field Name

Default Value

1

heartbeatinterval

60s

2

heartbeatdisabled

false

3

http_timeout

120s

The amount of time, in seconds, before an HTTP connection timeout. This value should not be less than or equal to zero.


4

logsource_name

USER_DEFINED

The log source name with which the Generic Beat instance should be configured. For example, proofpoint.


5

url

USER_DEFINED

The endpoint URL from which logs are fetched.


6

auth_type

basic

Authentication type supported by the API. Below are the macros for the authentication types:

Basic authentication: basic

Header based authentication: tokenauth

OAuth: oauth

No authentication: noauth


7

request_method

GET and POST

8

basic_auth->username

USER_DEFINED

9

basic_auth->password

USER_DEFINED

10

token_auth->auth_header

USER_DEFINED

11

token_auth->auth_token

USER_DEFINED

12

oauth20_auth->method

POST

13

oauth20_auth->content_type

application/x-www-form-urlencoded

14

oauth20_auth->body

USER_DEFINED

15

oauth20_auth->body_param

USER_DEFINED

16

oauth20_auth->auth_header_field

USER_DEFINED

17

oauth20_auth->auth_append_field

USER_DEFINED

18

oauth20_auth->headers

USER_DEFINED

19

oauth20_auth->params

USER_DEFINED

20

oauth20_auth->access_token_format

jsonkey

21

oauth20_auth->isRefershTokenSupported

false

22

oauth20_auth->access_token_field

access_token

23

oauth20_auth->url

USER_DEFINED

24

pagination_type

limitoffset

25

pagination->limit_offset->limit_field

USER_DEFINED

26

pagination->limit_offset->limit_value

1000

27

pagination->limit_offset->offset_field

USER_DEFINED

28

pagination->page_based->pagesize_field

USER_DEFINED

29

pagination->page_based->pagesize_value

1000

30

pagination->page_based->pagenumber_field

USER_DEFINED

31

pagination->cursor_based->cursor_type

url

32

pagination->cursor_based->cursor_location

header

33

pagination->cursor_based->cursor_query_param

USER_DEFINED

34

pagination->cursor_based->cursor_header_type

link


35

pagination->cursor_based->cursor_header_field

USER_DEFINED

The cursor header field is the header field name in which the auth token appears in the response.


36

sorting_enabled

false

Change to "true" if your API supports sorting.


37

sorting->sorting_field

USER_DEFINED

Sorting field name in which the API expects the sorting value.


38

sorting->sorting_value

USER_DEFINED

Sorting value string that the API supports. For example, asc.


39

filter_type

startend

Filter types are the different date range-supported filters that the API supports. Below are the macros for the filter types:

Between start and end date: startend

After any specific date: afterstart

Within an interval: interval

No filter: nofilter


40

filter->delay_time

0s

If the API supports any delay in real time data, then add that delay here in seconds. For example, 2s.


41

filter->start_and_end_filter->start_field

USER_DEFINED

Field name in which the API expects the start time.


42

filter->start_and_end_filter->start_value

USER_DEFINED

The start date value from which logs should be fetched.


43

filter->start_and_end_filter->end_value

USER_DEFINED

The end date value through which logs should be fetched.


44

filter->start_and_end_filter->end_field

USER_DEFINED

Field name in which the API expects the end time.


45

filter->after_start_filter->start_field

USER_DEFINED

Field name in which the API expects the start time.


46

filter->after_start_filter->start_value

USER_DEFINED

The start date value from which logs should be fetched.


47

filter->after_start_filter->response_date_field

USER_DEFINED

The date field to parse from the response body to fetch the next start date for fetching the next set of records.


48

filter->interval_filter->interval_field

USER_DEFINED

The field name in which the API expects the date fields. For example, interval.


49

filter->interval_filter->interval_value

USER_DEFINED

The interval value consists of the start time and end time string, separated by a delimiter (split char). Start time and end time are used to send the first request on the server. For example, 2021-10-08T00:00:00.000Z/2021-10-08T01:00:00.000Z.


50

filter->interval_filter->split_char

USER_DEFINED

The delimiter (split character) between the start and end date in the interval. For example, in the interval string 2021-10-08T00:00:00.000Z/2021-10-08T01:00:00.000Z, the split char is /.


51

filter->time_format

Monday, 02-Jan-06 15:04:05 MST

The time format that the API supports.


52

headers

empty structure

Response headers to send in the request.


53

params

empty structure

Request query parameters to send in the request.


54

response_field_flag

true

Set to true to parse any specific field from response body.


55

response_field

USER_DEFINED

The field to extract the relevant data from the response body. If a field is not present in the response, then entire response will be forwarded to Open Collector.


56

period

30s

The interval at which back-to-back requests will be sent to pull the logs from server.