Skip to main content
Skip table of contents

Initialize the Kafka Beat

This section provides instructions for initializing the Kafka Beat.


  • Open Collector is installed. If you have not already installed it, follow the instructions in the Open Collector Installation and User Guide, and then return to this topic.
  • The Kafka server is running. If you have not yet setup the Kafka server, setup the Kafka server first, and then return here.
  • Obtain a Kafka broker IP (IP on which the Kafka server is running).
  • Obtain a Kafka broker port (Port on which the Kafka server is running).
  • The Kafka topic (Topic you want to collect data from).
  • CA signed certificate, if your Kafka server is running in SSL mode with a CA signed certificate and requires the beat to perform host verification.

  • The following port is open:






    To configure the Kafka Beat for SentinelOne, see:

    SentinelOne via Kafka Beat Setup

    Configure the SentinelOne Log Source in SIEM

Initialize the Beat 

  1. Confirm Open Collector is running:

    ./lrctl status

    You should see the open_collector and metrics versions:

    If Open Collector is not running correctly, see Troubleshoot the Open Collector in the Open Collector Installation and User Guide.
  2. Start the Beat:

    ./lrctl kafkabeat start
  3. Select New kafkabeat instance from the list and hit Enter.
  4. Enter the unique beat identifier for this Kafka Beat instance and hit Enter.

  5. Enter the Kafka broker as: KAFKA_SERVER_HOST:KAFKA_SERVER_PORT, replacing KAFKA_SERVER_HOST with the IP of the Kafka server, and KAKFA_SERVER_PORT with the server port Kafka is listening to.

    Alternatively, you can use a fully-qualified domain name (FQDN) as the KAFKA_SERVER_HOST as shown below: 

  6. Enter the Kafka topic name from which you want to collect data. It should be the same topic on which your producer is sending data.
  7. Enter the consumer ID for this beat instance. You can always change the consumer ID by using config edit to fetch messages from the beginning.
  8. Select one of the supported authentication mechanisms from the list that your Kafka server supports.
  9. Enter Y to skip host verification on the Kafka Beat. This is recommended for self-signed certificates.

    If you enter N, then you have to upload the certificate file in the following steps. Otherwise, skip to step 10.

    1. Paste your server SSL certificate. (All content, including '-----------BEGIN CERTIFICATE------------------' to '-----------------------END CERTIFICATE------------------'). 
    2. Paste your client SSL certificate. (All content, including '-----BEGIN CERTIFICATE-----' and '-----END CERTIFICATE-----').

      If your certificate files have more than one BEGIN CERTIFICATE line, paste the section that contains a friendly name localhost or your actual hostname. This is usually the section in the middle.

    3. Paste your private key file content. (All content, including '-----BEGIN PRIVATE KEY-----' and '-----END PRIVATE KEY-----').

  10. Complete the following steps only if the SASL_SSL authentication mechanism was selected in step 8. Otherwise, skip to step 11.
    1. Select the SASL_SSL mechanism you want to enable in the Kafka Beat.

    2. Enter the user name to use for SASL authentication.

    3. Enter the password for SASL authentication.

  11. Enter Y if you want to enable SentinelOne parsing support within the Kafka Beat.

     The Kafka Beat starts.
  12. Check the status of the service:

    ./lrctl kafkabeat status

Default Config Values for Kafka Beat


Field Name

Default Values


User Provided

Combination of the broker's IP or FQDN and broker's port in the format IP:Port.


User Provided


Valid characters for the topic name are "[a-z, A-Z, 0-9, and \\._\\-]". Using any other characters will result in a error.



Enables or disables SSL support on the Kafka Beat.




Enables or disables host verification in SSL Mode (recommended for self-signed certificates).



User Provided

This must be unique for each beat instance. The user can change the consumer ID whenever the logs needed to be fetched from the beginning.

Valid characters for the consumer ID are "[a-z, A-Z, 0-9, and \\._\\-]". Using any other characters will result in a error.




The maximum amount of time, in milliseconds, the broker waits for consumer acknowledgment before declaring it to be disconnected.




The maximum amount of time, in milliseconds, the consumer expects a message to take to process for the user. 




Enables or disables SASL support in the Kafka Beat.




The SASL mechanism to be used by the Kafka Beat. This should always be same as the Kafka server SASL mechanism.

The following are the SASL mechanisms currently supported by the Kafka Beat.

PLAIN: For no encryption

SCRAM_256: for SHA256 encryption

SCRAM_512: for SHA512 encryption


User Provided

Username for SASL authentication on the Kafka server.


User Provided

Password for SASL authentication on the Kafka server.



Enables or disables SentinelOne message parsing, allowing SentinelOne to send compressed protocol buffer (protobuf) messages.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.