Log Source Virtualization

Log Source Virtualization is supported on Windows agents and on Linux agents from System Monitor 7.5.1.8001 and later.

Log Source Virtualization makes it possible to consume all the available intelligence within individual log source files that contain multiple records from different sources. When virtualization is enabled on a log source, it is referred to as a “parent” log source, and the different records inside it are referred to as either “virtual” or “child” log sources (when referencing log sources, the terms “virtual” and “child” are often used interchangeably).

Virtual log sources are treated in the same way as other log sources. They are processed in accordance with their assigned MPE policies and they appear in the same lists as the other log sources within the deployment. In contrast to Syslog Virtualization, which applies only to syslog relay logs received by the System Monitor syslog server, Log Source Virtualization can be applied to syslog relay sources, Windows Event Logs, flat files, and any other log source within your deployment that contains multiple records.

To begin Log Source Virtualization, you need to apply properly configured Log Source Virtualization templates to the parent log sources at the agent level. Virtualization templates contain one or more virtual log sources that include identifier regular expressions (also called “regexes”) to run against and parse data in the parent sources. When one of the records matches a particular regex, a child log is created and assigned to the virtual log source associated with that regex. Records that do not match any of the regexes are assigned to the Catch-All log source.

Virtual log sources cannot be edited to the same extent as their non-virtual counterparts because certain properties (including their lifecycle) remain tied to their parent sources. For example, if a parent log source is retired or has its virtualization disabled, all of its child log sources are retired or disabled as well. You can, however, edit the name, regex, MPE policy, and log source type properties for virtual log sources.

You can add, modify, and delete both virtual log sources and virtualization templates from the Log Source Virtualization Template Manager. As you create or modify templates and virtual log sources, you can check their regex parsing and distribution accuracy by pasting sample logs into the testing tool. Keep in mind that changes you make to the properties of either virtual log sources or virtualization templates only affect the future child log sources created by them (existing child sources are unaffected).

Load Balanced Virtual Log Sources

Load balanced virtual log sources let you specify the log sources that are being sent to a load balancer and the System Monitors to which the load balancer is sending log messages. These options enable deployments with larger volumes to utilize load balanced log sources without data loss.

In LogRhythm, you can designate one or more System Monitors and one or more Log Sources as "load balancing." After doing this, all load balancing System Monitors must be made aware of all load balanced Log Sources, regardless of where the Log Sources originated. As more System Monitors or Log Sources are created, the updated configuration is pushed out to all load balancing System Monitors.

System Monitors are designated under the Syslog and Flow Settings tab of the System Monitor Agent Properties dialog box.

Log Sources are designated under the Additional Settings tab of the Log Message Source Properties dialog box or the Log Source Settings dialog box (when adding in batch).