Modify Data Processor Advanced Properties

Make changes to the Data Processor Advanced Properties with extreme care! LogRhythm recommends that the Data Processor Advanced Properties only be modified with the assistance of LogRhythm Support, or by advanced users who have attended LogRhythm training.

To modify the advanced properties of a data processor:

On the main toolbar, click Deployment Manager.
Click the Data Processors tab.
Double-click the Data Processor you want to configure.
Click the Advanced button at the lower-left corner.
The Advanced Properties window appears.

Find the component you want to configure and adjust the settings in the Value column according to the information in the following table.

Property	Range	Default	Description
AIE Provider: Provider
ClientAddress			IPv4/IPv6 address the AI Engine Data Provider will use to connect to the AI Communication Manager for data/management communications. Only use a static IP address. Do not use DHCP. Default is blank (recommended.)
LocalLogLifeTime	1-30	7	Time to keep AI Engine Data Provider application logs (in days).
LogLevel		Warning	Sets the AI Engine Data Provider logging level. The log is written to the lraiedp.log file.
AIE Provider: Sending
CompressionStrength	0-10	5	The compression level to use when sending logs to the AI Engine Communication Manager. Values from 1-10 indicate the compression strength where 0 is off/no compression. Stronger compression requires more CPU to compress and uncompress the data.
FlushBatch	1000-10000	1000	The (maximum) number of logs that should be batched and sent to the AI Engine Communication Manager during each socket send.
MaxDataQueueSize	10-2048	256	The maximum size of the AI Engine Data Provider's in-memory data queue (in MB). When the queue size exceeds this amount, incoming logs will be buffered into spool files until the queue size is reduced.
MaxSpoolStorage	0-1024	20	The maximum amount of storage available to hold AI Engine Data Provider spooled data filed (in GB). When the amount of space the spool files occupy exceeds this amount, the oldest spool files will be deleted.
SendAfterXLogsQueued	1-100000	100	The threshold number of logs in the queue required to send logs to the AI Engine Communication Manager, independent of time.
SendAfterXSecondsElapsed	1-3600	10	The threshold number of records required to send logs to the AI Engine Communication Manager, independent of number of logs.
AIE Provider: TCP/IP
SocketConnectionTimeout	1-300	120	AI Engine Data Provider socket connection timeout (in seconds).
SocketDontLinger		Enabled	AI Engine Data Provider don't linger socket option. Close the socket gracefully without lingering.
SocketNoDelay		Enabled	AI Engine Data Provider no delay socket option. Disable the Nagle algorithm for send coalescing.
SocketReceiveBuffer	16384-65535	65535	AI Engine Data Provider socket receiver buffer (in bytes).
SocketReceiveTimeout	1-300	60	AI Engine Data Provider socket receive timeout (in seconds).
SocketReuseAddress		Enabled	AI Engine Data Provider reuse address socket option. Allow the socket to be bound to an address that is already in use.
SocketSendBuffer	16384-65535	65535	AI Engine Data Provider socket send buffer (in bytes).
SocketSendTimeout	1-300	60	AI Engine Data Provider socket send timeout (in seconds).
AIE Provider: TLS Security
AIEComMgrTLSCertOCSPURL			OCSP URL for AIE ComMgr TLS certificate revocation checking.
AIEDPTLSCertLocation		LocalMachine	AI Engine Data Provider TLS certificate location. Values: LocalMachine or CurrentUser
AIEDPTCertStore		MY	AI Engine Data Provider TLS certificate store. Values: MY or Root
AIEDPTLSCertSubject			AI Engine Data Provider TLS certificate subject. Example values: CN=10.1.0.79 or CN=lr-0872ed-msa or CN=lr-0872ed-msa.exampledomain.com.
EnforceAIEComMgrTLSCertRevocation		Disabled	Enforce AI Engine Communication Manager TLS certificate revocation check.
EnforceAIEComMgrTLSCertTrust		Disabled	Enforce AI Engine Communication Manager TLS certificate trusted authority check.
UseAIEDPTLSCert		Disabled	Enable AI Engine Data Provider client TLS certificate.
Mediator: DataIndexerProvider
DataLingerTimeoutMiliSec	1-10000	1000	The data socket linger timeout (in milliseconds).
DataQueueHighWaterMark	10000-100000	100000	The high water mark for the Data Indexer data queue.
DataSendTimeoutMilliSec	0-10000	250	The data socket send timeout (in milliseconds).
ParseQueueThreadCount	1-25	3	The maximum number of unparsed messages that will be serialized concurrently.
StatsLingerTimeoutMilliSec	1-10000	1000	The data socket linger timeout (in milliseconds).
StatsQueueHighWaterMark	10000-100000	10000	The high water mark for the Data Indexer stats queue.
StatsSendTimeoutMilliSec	0-10000	0	The stats socket send timeout (in milliseconds).
ThreadJoinTimeoutMilliSec	1000-10000	5000	The maximum amount of time to wait for the Data Indexer provider threads to exit on shutdown.
UnparsedItemsQueueSize	10000-10000000	250000	The number of unparsed reliable messages allowed to accumulate in memory before the Mediator goes into a Suspend state.
Mediator: General
ActiveArchivePath		C:\LogRhythmArchives \Active	Archiving directory path (full path to the directory in which archive files are written). If the requested directory does not exist, it is created.
ActiveArchiveProtection		File size and last modification date tracking	Active archive protection mode File size and last modification date tracking No Protections Full SHA1 hashing of archive files
ArchiveAge	1-7	7	Maximum days an archive can live in active directory (in days).
ArchiveBatch	1000-10000000	102400	The number of logs that are allowed to build up in the archive queue before being processed by the archiver.
ArchiveByEntity		Disabled	Stores inactive archives according to entity structure.
ArchiveCompression		Enabled	Determines if inactive archive files are gzip compressed
ArchiveSize	1024-131072	10240	Maximum size for archive before moving to inactive directory (in KB).
ArchiveWriteThreadCount	1-20	3	The maximum number of archives that will be serialized and written to disk concurrently.
AutomaticLogSource ConfigurationNetflow		Disabled	Automatic Log Source Configuration (Netflow/J-Flow Sources). When enabled, the Data Processor automatically registers new message sources for NetFlow/J-Flow sending devices which can be automatically identified.
AutomaticLogSource ConfigurationsFlow		Disabled	Automatic Log Source Configuration (sFlow Sources). When enabled, the Data Processor automatically registers new message sources for sFlow sending devices which can be automatically identified.
AutomaticLogSource ConfigurationSNMPTimeout	1-120	10	Automatic Log Source Configuration (SNMP Discovery). Defines the timeout value (in seconds) for SNMP communications used in SNMP Device Identification.
AutomaticLogSource ConfigurationSNMPTrap		Disabled	Automatic Log Source Configuration (SNMP Trap Sources). When enabled, the Data Processor automatically registers new message sources for SNMP trap sending devices which can be automatically identified.
AutomaticLogSource ConfigurationSyslog		Disabled	Automatic Log Source Configuration (Syslog Sources). When enabled the Data Processor automatically registers new message sources for syslog sending devices which can be automatically identified.
ClientSocket ReceiveTimeout	1000-7200000	60000	Client socket receive timeout for Agent socket connections (in ms).
ClientSocketSendTimeout	1000-7200000	60000	Client socket send timeout for Agent socket connections (in ms).
ComponentVersion			The version of this LogRhythm component
ConnectionTimeout	3-7200	120	Connection timeout for Agent socket connections (in seconds).
InactiveArchivePath		C:\LogRhythmArchives \Inactive	Directory (full path) where the inactive archive files are written. If the requested directory does not exist, it is created.
InactiveArchiveProtection		Full SHA1 hashing of archive files	Inactive archive protection mode File size and last modification date tracking No Protections Full SHA1 hashing of archive files
InactiveSubdirectory FileCount	100-10000	10000	Inactive archive subdirectory maximum file count.
LocalLogLifetime	1-30	7	The number of days to keep Mediator and MPE log files.
LogLevel		VERBOSE	Sets the Data Processor logging level (log written to scmedsvr.log)
MaxAgentUpdates	1-10000	10	The maximum number of concurrent Agent updates that can be delivered.
MaxConnections	0-10000	100	Maximum number of Agent connections to allow.
MaxLogArchivingRate	0-10000	0	Maximum rate at which logs can be archived.
MaxLogProcessingRate	0-100000	0	Maximum rate at which logs can be processed.
MaxLogReceiveRate	0-100000	0	Maximum rate at which logs can be received.
MaxServiceMemory	512-65536	1024	Maximum memory allowed for the Data Processor process (in MB).
MaxUnprocessed DiskQueueSpace	0-1000	100	The maximum amount of space (in GB) to be used by the Unprocessed Log Disk Queue. A value of 0 indicates no maximum. A warning event is written when 80% of the specified space is used. If the maximum is reached, the mediator goes into suspend mode.
MinAgentSocketSecurity		TLS 1.0	Sets the minimum encryption standard to be used for Agent connections. If set to TLS 1.0, the Mediator generates a 1024-bit key. If set to TLS 1.2, the Mediator generates a 2048-bit key.
MinUnprocessed DiskQueueSpace	1-1000	1	The minimum amount of space (in GB) that must be available on the volume that the Unprocessed Log Disk Queue spool files are being written to. If the minimum is reached, the mediator goes into suspend mode.
ProcessPriority		Normal	Process priority for the Data Processor process.
QueueSize	10000-500000	20000	The maximum size of the archive queue and the unprocessed log queue.
SecondaryServerIP			An external facing IP address that an Agent can use to connect to the Mediator. This IP address will be used by Agents when they can't connect using the Primary Server IP address (ServerIP). The Secondary Server IP/Port must be forwarded to the Primary Server IP/Port by a firewall or router. This parameter must be a static IP v4/v6 address with a maximum length of 45 or a DNS name with a maximum length of 255. DNS names are only supported for version 6.x System Monitors and later. You must configure your firewall or router to forward this IP/Port to the Primary Server. This is important for deployments that use NAT.
SecondaryServerSSLPort	1-65535	443	The external facing IP port to use with the Secondary Server IP address. The Secondary Server IP/Port must be forwarded to the Primary Server IP/Port by a firewall or router.
ServerDNS			DNS address that agents will use to connect to this Data Processor. If this parameter is not specified, Agents uses the ServerIP address to connect to this Data Processor. If you created custom certificates for the Mediator, this must match the DNS name specified in the custom certificate.
ServerIP			IPv4 address that the Data Processor listens on for Agent communications. This parameter must be a static IPv4 address with a maximum length of 16
ServerIPv6			IPv6 address that the Data Processor listens on for Agent communications. This parameter must be a static IPv6 address with a maximum length of 45.
ServerSSLPort	1-65535	443	Port that the Data Processor listens on for Agent communications.
TertiaryServerIP			An external facing IP address or DNS name that an Agent outside the network can use to connect to the Mediator. This IP address will be used by Agents when they can't connect using the Primary or Secondary Server IP addresses (ServerIP/SecondaryServerIP). The Tertiary Server IP/Port must be forwarded to the Primary Server IP/Port by a firewall or router. This parameter must be a static IP v4/v6 address with a maximum length of 45 or a DNS name with a maximum length of 255. DNS names are only supported for version 6.x System Monitors and later. You must configure your firewall or router to forward this IP/Port to the Tertiary Server. This is important for deployments that use NAT.
TertiaryServerSSLPort	1-65535	443	The external facing IP port to use with the Tertiary Server IP address. The Tertiary Server IP/Port must be forwarded to the Primary Server IP/Port by a firewall or router.
Unprocessed DiskQueueLocation			The directory where Data Processor unprocessed log disk queue spool files are written. The default directory is the mediator state folder. After changing the directory location, any remaining spool files must be manually moved to the new location.
Mediator: InsertManagerEM
AllowAutomaticRateOverride		Enabled	Enable/disable automatic event insert rate override.
BatchInterval	1-300	5	This value determines how often (in seconds) batches are submitted to the Platform Manager database for insertion. Max Insert Batch is determined at startup by the Max Insert Rate and Batch Interval.
DiskQueueLocation			The directory where Platform Manager Insert Manager disk queue spool files are written. The default directory is the mediator state folder. After changing the directory location, any remaining spool files must be manually moved to the new location.
MaxAutomaticInsert RateOverridePercent	10-100	50	The maximum override percentage that is applied to Max Insert Rate throttling level.
MaxEMInsert DiskQueueSpace	0-1000	100	The maximum amount of space (in GB) to be used by the PM Insert Manager Disk Queue. A value of 0 indicates no maximum. A warning event is written when 80% of the specified space is used. If the maximum is reached, the oldest spool files are deleted until the space used by the spool files is less than the specified maximum.
MaxInsertRate	1-100000	3000	This value determines the maximum number of Platform Manager logs that will be inserted per second. The insertion rate will not exceed this value. Note that this value is based on the performance profile of the system.
MinEMInsert DiskQueueSpace	1-1000	1	The minimum of amount of space (in GB) that must be available on the volume that the PM Insert Disk Queue spool files are being written to. If the minimum is reached, the oldest spool files are deleted until the space used by the spool files is above the specified minimum.
SystemMaxInsertBatch	100-100000	50000	This is a fixed constant that determines the maximum number of inserts the system will process in a single batch.
Mediator: LDS
LDSDistributionQueueSize	1000-100000	10000	Specify the size for each log distribution receiver queue. Every receiver has its own queue. If this queue reaches maximum size, logs will be dropped. However setting queue size too high could result in excessive memory utilization.
LDSDistributionThreadCount	1-100	10	Specify the number of threads to use for the log distribution receiver process.
LDSEngineQueueSize	1000-500000	60000	Specify the size of primary log distribution queue. If this queue reaches maximum size, logs will be dropped. However setting queue size too high could result in excessive memory utilization.
LDSEngineThreadCount	1-100	5	Specify the number of threads to use for the primary log distribution process.
Mediator: TLSCertificates
AgentTLSCertOCSPURL			The OCSP URL for Agent certificate revocation checking.
EnforceAgentTLSCert Revocation		Disabled	Enforce Agent Certificate Revocation Check. If this fails, the Mediator will disconnect from the Agent and logs will be written to the scmedsvr.log.
EnforceAgentTLSCertTrust		Disabled	Enforce Agent certificate Trusted Authority Check. If this fails, the Mediator will disconnect from the Agent and logs will be written to the scmedsvr.log.
MediatorTLSCertLocation			The location of the Windows certificate where the Mediator server certificates are installed--can be LocalMachine or CurrentUser.
MediatorTLSCertStore			The Windows certificate store where the Mediator server certificate is installed--can be MY or ROOT.
MediatorTLSCertSubject			The Subject of the server certificate that the Mediator should use (e.g., CN=190.1.2.123 or CN=lr-0870eds-msa or CN=lr-0870eds-msa.secious.com).
RequireAgentTLSCert		Disabled	Require agents to present a client certificate when connecting.
UseMediatorTLSCert		Disabled	If checked, the Mediator will use the specified server certificate when connecting with Agents; otherwise, the Mediator will use a self-generated/signed certificate (default).
Mediator: Unidirectional Agent
Enabled		Disabled	Check to enable unidirectional Agent communications with the Data Processor.
Mediator Port	1-65535	40000	Specifies the Data Processor port to use when running in Unidirectional Agent mode.
MPE: Engine
CacheSize_Dimension	1000-1000000	10000	Specify the size for the unique metadata value cache. There are nine metadata caches containing unique metadata values for processed log messages. The larger the queue size the more unique values will be stored in memory resulting in more efficient log processing. However setting queue size too high could result in excessive memory utilization.
CacheSize_Msg	100000-5000000	200000	Specify the size for the unique log message cache. The larger the queue size, the more unique log messages will be stored in memory resulting in more efficient online log storage. However setting queue size too high could result in excessive memory utilization.
DataAndIndexCompression		None	Specifies the level of compression to apply to data and indices. This is obsolete in the current version and changing its value has no impact on indexing and compression.
DenormalizeLogMetadata		True	Setting this property to false reduces data transmission volume at the cost of normalizing enumerable values. Metadata fields such as Log Source Type and Common Event will be presented as ID numbers instead of readable text. Disabling this feature decreases the usability of downstream features such as Log Distribution Service and full text search.
DNSCachedRecordTTL	5-1440	15	The time to live for cached DNS Name to Known Host to IP host resolution records (in minutes).
DNSCacheMaintCycle	1-60	5	The frequency to launch DNS cache maintenance (in minutes).
DNSCacheRecord ExternalIPToNameTTL	5-1440	15	The time to live for cached DNS external IP to Name host resolution records (in minutes).
DNSCacheRecord InternalIPToNameTTL	5-1440	5	The time to live for cached DNS private/internal IP to Name host resolution records (in minutes).
DNSIPToName		Off	IP to Name DNS resolution mode. Values: Off, Resolve All, Resolve Internal Resolve IP addresses to their associated DNS names.
DNSLogLevel		Error	The logging level for the DNS resolution engine.
DNSNameToIP		Disabled	Resolve DNS names to their associated IP addresses.
DNSResolveMsgSourceHostIP		Enabled	Resolve host IP addresses when logs match a rule where the source or destination is assigned to the message source host.
GeoIPResolutionMode		None	The level of detail to resolve for Geographic IP lookup. Options = None, Country, Region, and City. If this is left set to None, GeoIP location will not be resolved for logs or Network Visualization.
LogProcessingThreads	1-50	10	The number of log processing threads.
PerfOptimizedLogIndexing		Disabled	Enables or disables performance-optimized indexing of logs.
RulePerfLogSampleSize	1-1000	10	The minimum number of logs that must be processed before a rule will be disabled due to not meeting the minimum logs per second requirement.
RulePerfMinLogsPerSecond	1-1000	50	The minimum allowed average logs per second a rule must meet.
MPE: General
IdentityInference		Enabled	Enables or disables Identity Inference for the MPE (as long as Globally disabled).
LogLevel		WARNING	Sets the MPE logging level (log written to scmpe.log). Options: Off, Error, Warning, Info, Verbose, Debug
MaintenanceInterval	1-120	60	How often to perform internal process maintenance (in seconds).
RulePerformanceStatsMode		Off	Rule performance statistics mode: Off - no not write report (lps_detail.log) or data file (lps_stats.dat) locally or submit to LogRhythm (default). Local - write the report (lps_detail.log) and data file (lps_stats.dat) locally. Local and Send - write the Report (lps_detail.log) and data file (lps_stats.dat) locally and submit to LogRhythm.
RulePerformanceStats SubmitInterval	1-24	24	How often to submit rule performance information to LogRhythm (in hours). The latest lps_detail.log and lps_stats.dat files will be submitted each interval.
MPE: LogMart
LogMartCommitInterval	1-120	60	How often (in seconds) the LogMart is updated with new data.
LogMartCommitTimeout	1-120	40	How long (in seconds) a single commit operation can take before timing out.
MPE: StatKeeper
StatkeeperCommitInterval HeartbeatInfo	1-300	10	How often (in seconds) heartbeat information is committed to the database.
StatKeeperCommitIntervalLogInfo	1-300	60	How often (in seconds) log collection statistics are committed to the database.
StatKeeperCommitTimeout	30-120	30	How long (in seconds) a single commit operation can take before timing out
StatKeeperEnabled		Enabled	Specify if StatKeeper should be enabled.