Skip to main content
Skip table of contents

Scaling Open Collector and the AZURE Eventhubbeat

The information on this page is prepared using the Machine and Log Specifications for the Azure EventHub Beat.

Scaling Options

If your instance is not performing with the desired message per second (MPS) count, then Open Collector can be scaled vertically and horizontally using the following options.




Throughput Unit




Beats Instance



For these configurations, the beat is streaming logs to a single instance of Open Collector running on the same machine with default advanced settings.

Scaling Horizontally

The beat can be scaled horizontally by configuring more than one EventHub in the EventHub beat instance by providing different storage container, EventHub connection string and unique container name. For more information on scaling the Azure EventHub Beat horizontally, see Scaling Options for Open Collector and the Azure EventHub Beat.

If CPU usage is going above 70% even when using the given scaling and configuration options, then no further horizontal scaling will be possible on a single instance. At this point, it is recommended to launch a new instance of Open Collector and Beats to achieve the desired goal.



Grafana Dashboard Stats:

Note: In case of EventHub beat CPU utilization is not constant, it is higher when it is processing logs with max MPS, CPU and memory utilization manages to come down and gave low output MPS.


Scaling Vertically

The options for vertically scaling the Azure EventHub Beat vary depending on the monitoring system used within the beat.

Scaling Open Collector and the Azure EventHub Beat

Open Collector can be scaled vertically with the given specifications and system configuration options to align with the MPS capabilities. If the required MPS is 2000, then the specification presented below is recommended. Beyond this point, the beat can be scaled further by adding more machines in order to achieve a higher MPS performance with same specification

If at any point your CPU consumption is more than 70%, it is recommended to add a new machine to share the load.

Log File Size

Sample Log File

61 KB


3 KB


2 KB


OC1 (8CPU, 16GB RAM)
2000 MPS
EventHub Throughput Unit Value
EventHub Throughput unit: 3
Scale OC Horizontally
    capacity: 20000
    concurrency: 15
    capacity: 1
    concurrency: 5
    capacity: 20000
    concurrency: 10
    capacity: 20000
    concurrency: 10
    capacity: 1000
    concurrency: 1
CPU/RAM Consumption
RAM: ~3.75GB
CPU: ~90%
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.