diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/Makefile.am | 4 | ||||
| -rw-r--r-- | doc/dev_oplugins.html | 5 | ||||
| -rw-r--r-- | doc/imuxsock.html | 2 | ||||
| -rw-r--r-- | doc/manual.html | 4 | ||||
| -rw-r--r-- | doc/omfile.html | 167 | ||||
| -rw-r--r-- | doc/omfwd.html | 118 | ||||
| -rw-r--r-- | doc/ompipe.html | 62 | ||||
| -rw-r--r-- | doc/omsnmp.html | 148 | ||||
| -rw-r--r-- | doc/omusrmsg.html | 64 | ||||
| -rw-r--r-- | doc/omuxsock.html | 2 | ||||
| -rw-r--r-- | doc/property_replacer.html | 256 | ||||
| -rw-r--r-- | doc/rainerscript.html | 13 | ||||
| -rw-r--r-- | doc/rsyslog_conf_filter.html | 178 | ||||
| -rw-r--r-- | doc/rsyslog_conf_global.html | 48 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 4 | ||||
| -rw-r--r-- | doc/rsyslog_conf_templates.html | 258 | ||||
| -rw-r--r-- | doc/v4compatibility.html | 2 | ||||
| -rw-r--r-- | doc/v6compatibility.html | 19 | ||||
| -rw-r--r-- | doc/v7compatibility.html | 33 |
19 files changed, 1204 insertions, 183 deletions
diff --git a/doc/Makefile.am b/doc/Makefile.am index 04fda6b3..cc1a3209 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -31,6 +31,10 @@ html_files = \ version_naming.html \ contributors.html \ dev_queue.html \ + ompipe.html \ + omfwd.html \ + omfile.html \ + omusrmsg.html \ omstdout.html \ omudpspoof.html \ omruleset.html \ diff --git a/doc/dev_oplugins.html b/doc/dev_oplugins.html index 63c186a3..b33b67f9 100644 --- a/doc/dev_oplugins.html +++ b/doc/dev_oplugins.html @@ -143,6 +143,11 @@ omstdout, you can see how a plugin may deal with the situation. array-passing capability not blindly be used.</b> In such cases, we can not guard the plugin from segfaulting and if the plugin (as currently always) is run within rsyslog's process space, that results in a segfault for rsyslog. So do not do this. +<p>Another possible mode is OMSR_TPL_AS_JSON, where instead of the template +a json-c memory object tree is passed to the module. The module can extract data +via json-c API calls. It MUST NOT modify the provided structure. This mode is +primarily aimed at plugins that need to process tree-like data, as found +for example in MongoDB or ElasticSearch. <h3>Batching of Messages</h3> <p>Starting with rsyslog 4.3.x, batching of output messages is supported. Previously, only a single-message interface was supported. diff --git a/doc/imuxsock.html b/doc/imuxsock.html index 34a696d9..19f9cf51 100644 --- a/doc/imuxsock.html +++ b/doc/imuxsock.html @@ -104,6 +104,8 @@ to the system log socket.</li> be obtained from the log socket itself. If so, the TAG part of the message is rewritten. It is recommended to turn this option on, but the default is "off" to keep compatible with earlier versions of rsyslog. This option was introduced in 5.7.0.</li> +<li><b>$SystemLogParseTrusted</b> [on/<b>off</b>] - specifies if Trusted Properties shall be parsed +and saved into libee event structure. This option needs $SystemLogSocketAnnotate to be on.</li> <li><b>$SystemLogRateLimitInterval</b> [number] - specifies the rate-limiting interval in seconds. Default value is 5 seconds. Set it to 0 to turn rate limiting off. </li> diff --git a/doc/manual.html b/doc/manual.html index 1114b9e2..82b0dc2d 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 6.4.2 (stable branch) of rsyslog.</b> +<p><b>This documentation is for version 7.1.4 (devel branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might @@ -35,6 +35,8 @@ if you upgrade from v4, read the <a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and if you upgrade from v5, read the <a href="v6compatibility.html">rsyslog v6 compatibility notes</a>. +if you upgrade from v6, read the +<a href="v7compatibility.html">rsyslog v7 compatibility notes</a>. <p>Rsyslog will work even if you do not read the doc, but doing so will definitely improve your experience.</p> <p><b>Follow the links below for the</b></p> diff --git a/doc/omfile.html b/doc/omfile.html new file mode 100644 index 00000000..23ecc034 --- /dev/null +++ b/doc/omfile.html @@ -0,0 +1,167 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>File Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>File Output Module</h1> +<p><b>Module Name: omfile</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The omfile plug-in provides the core functionality of writing messages to files residing inside the local file system (which may actually be remote if methods like NFS are used). Both files named with static names as well files with names based on message content are supported by this module. It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>DynaFileCacheSize </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>ZipLevel </strong>0..9 [default 0]<br> + if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> + + <li><strong>FlushInterval </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>ASyncWriting </strong>on/off [default off]<br> + if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br> + + <li><strong>FlushOnTXEnd </strong>on/off [default on]<br> + Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br> + + <li><strong>IOBufferSize </strong><size_nbr>, default 4k<br> + size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br> + + <li><strong>DirOwner </strong><br> + Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>DirGroup </strong><br> + Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>FileOwner </strong><br> + Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>FileGroup </strong><br> + Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>DirCreateMode </strong>[defaul 0700]<br> + This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br> + + <li><strong>FileCreateMode </strong>[default 0644]<br> + The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br> + + <li><strong>FailOnCHOwnFailure </strong>on/off [default on]<br> + This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br> + + <li><strong>CreateDirs </strong>on/off [default on]<br> + create directories on an as-needed basis<br></li><br> + + <li><strong>Sync </strong>on/off [default off]<br> + enables file syncing capability of omfile.<br></li><br> + + <li><strong>File </strong><br> + If the file already exists, new data is appended to it. Existing data is not truncated. If the file does not already exist, it is created. Files are kept open as long as rsyslogd is active. This conflicts with external log file rotation. In order to close a file after rotation, send rsyslogd a HUP signal after the file has been rotated away. <br></li><br> + + <li><strong>DynaFile </strong><br> + For each message, the file name is generated based on the given template. Then, this file is opened. As with the ``file'' property, data is appended if the file already exists. If the file does not exist, a new file is created. A cache of recent files is kept. Note that this cache can consume quite some memory (especially if large buffer sizes are used). Files are kept open as long as they stay inside the cache. Currently, files are only evicted from the cache when there is need to do so (due to insufficient cache size). To force-close (and evict) a dynafile from cache, send a HUP signal to rsyslogd. <br></li><br> + + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li><br> + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command writes all syslog messages into a file.</p> +<textarea rows="5" cols="60">Module (path="builtin:omfile") +*.* action(type="omfile" +DirCreateMode="0700" +FileCreateMode="0644" +File="/var/log/messages") +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> + <li><strong>$DynaFileCacheSize </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>$OMFileZipLevel </strong>0..9 [default 0]<br> + if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> + + <li><strong>$OMFileFlushInterval </strong>(not mandatory, default will be used)<br> + Defines a template to be used for the output. <br></li><br> + + <li><strong>$OMFileASyncWriting </strong>on/off [default off]<br> + if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br> + + <li><strong>$OMFileFlushOnTXEnd </strong>on/off [default on]<br> + Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br> + + <li><strong>$OMFileIOBufferSize </strong><size_nbr>, default 4k<br> + size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br> + + <li><strong>$DirOwner </strong><br> + Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$DirGroup </strong><br> + Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$FileOwner </strong><br> + Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$FileGroup </strong><br> + Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>$DirCreateMode </strong>[defaul 0700]<br> + This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br> + + <li><strong>$FileCreateMode </strong>[default 0644]<br> + The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br> + + <li><strong>$FailOnCHOwnFailure </strong>on/off [default on]<br> + This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br> + + <li><strong>$F$OMFileForceCHOwn </strong><br> + force ownership change for all files<br></li><br> + + <li><strong>$CreateDirs </strong>on/off [default on]<br> + create directories on an as-needed basis<br></li><br> + + <li><strong>$ActionFileEnableSync </strong>on/off [default off]<br> + enables file syncing capability of omfile.<br></li><br> + + <li><strong>$ActionFileDefaultTemplate </strong>[templateName]<br> + sets a new default template for file actions.<br></li><br> + + <li><strong>$ResetConfigVariables </strong><br> + Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br> + +</ul> + +<p><b>Legacy Sample:</b></p> +<p>The following command writes all syslog messages into a file.</p> +<textarea rows="5" cols="60">$ModLoad omfile +$DirCreateMode 0700 +$FileCreateMode 0644 +*.* /var/log/messages +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omfwd.html b/doc/omfwd.html new file mode 100644 index 00000000..5599ae39 --- /dev/null +++ b/doc/omfwd.html @@ -0,0 +1,118 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>Forwarding Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Forwarding Output Module</h1> +<p><b>Module Name: omfwd</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The omfwd plug-in provides the core functionality of traditional message forwarding via UDP and plain TCP. It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>Target </strong>string<br> + Name or IP-Address of the system that shall receive messages. Any resolvable name is fine. <br></li><br> + + <li><strong>Port </strong>[Default 514]<br> + Name or numerical value of port to use when connecting to target. <br></li><br> + + <li><strong>Protocol </strong>udp/tcp [default udp]<br> + Type of protocol to use for forwarding. Note that ``tcp'' means both legacy plain tcp syslog as well as RFC5425-based TCL-encrypted syslog. Which one is selected depends on the protocol drivers set before the action commend. Note that as of 6.3.6, there is no way to specify this within the action itself. <br></li><br> + + <li><strong>TCP_Framing </strong>``traditional'' or ``octet-counted'' [default traditional]<br> + Framing-Mode to be for forwarding. This affects only TCP-based protocols. It is ignored for UDP. In protocol engineering, ``framing'' means how multiple messages over the same connection are separated. Usually, this is transparent to users. Unfortunately, the early syslog protocol evolved, and so there are cases where users need to specify the framing. The traditional framing is nontransparent. With it, messages are end when a LF (aka ``line break'', ``return'') is encountered, and the next message starts immediately after the LF. If multi-line messages are received, these are essentially broken up into multiple message, usually with all but the first message segment being incorrectly formatted. The octet-counting framing solves this issue. With it, each message is prefixed with the actual message length, so that a receivers knows exactly where the message ends. Multi-line messages cause no problem here. This mode is very close to the method described in RFC5425 for TLS-enabled syslog. Unfortunately, only few syslogd implementations support octet-counted framing. As such, the traditional framing is set as default, even though it has defects. If it is known that the receiver supports octet-counted framing, it is suggested to use that framing mode. <br></li><br> + + <li><strong>ZipLevel </strong>0..9 [default 0]<br> + Compression level for messages. Rsyslog implements a proprietary capability to zip transmitted messages. Note that compression happens on a message-per-message basis. As such, there is a performance gain only for larger messages. Before compressing a message, rsyslog checks if there is some gain by compression. If so, the message is sent compressed. If not, it is sent uncompressed. As such, it is totally valid that compressed and uncompressed messages are intermixed within a conversation. <br>The compression level is specified via the usual factor of 0 to 9, with 9 being the strongest compression (taking up most processing time) and 0 being no compression at all (taking up no extra processing time). <br></li><br> + + <li><strong>RebindInterval </strong>integer<br> + Permits to specify an interval at which the current connection is broken and re-established. This setting is primarily an aid to load balancers. After the configured number of messages has been transmitted, the current connection is terminated and a new one started. Note that this setting applies to both TCP and UDP traffic. For UDP, the new ``connection'' uses a different source port (ports are cycled and not reused too frequently). This usually is perceived as a ``new connection'' by load balancers, which in turn forward messages to another physical target system. <br></li><br> + + <li><strong>StreamDriver </strong>string<br> + Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> + + <li><strong>StreamDriverMode </strong>integer [default 0]<br> + mode to use with the stream driver (driver-specific)<br></li><br> + + <li><strong>StreamDriverAuthMode </strong>string<br> + authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific).<br></li><br> + + <li><strong>StreamDriverPermittedPeers </strong>string<br> + accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific)<br></li><br> + + <li><strong>ResendLastMSGOnReconnect </strong>on/off<br> + Permits to resend the last message when a connection is reconnected. This setting affects TCP-based syslog, only. It is most useful for traditional, plain TCP syslog. Using this protocol, it is not always possible to know which messages were successfully transmitted to the receiver when a connection breaks. In many cases, the last message sent is lost. By switching this setting to "yes", rsyslog will always retransmit the last message when a connection is reestablished. This reduces potential message loss, but comes at the price that some messages may be duplicated (what usually is more acceptable). <br></li><br> + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">Module (path="builtin:omfwd") +*.* action(type="omfwd" +Target="192.168.2.11" +Port="10514" +Protocol="tcp" +) +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> + <li><strong>$ActionForwardDefaultTemplateName </strong>string [templatename]<br> + sets a new default template for UDP and plain TCP forwarding action<br></li><br> + + <li><strong>$ActionSendTCPRebindInterval </strong>integer<br> + instructs the TCP send action to close and re-open the connection to the remote host every nbr of messages sent. Zero, the default, means that no such processing is done. This directive is useful for use with load-balancers. Note that there is some performance overhead associated with it, so it is advisable to not too often "rebind" the connection (what "too often" actually means depends on your configuration, a rule of thumb is that it should be not be much more often than once per second).<br></li><br> + + <li><strong>$ActionSendUDPRebindInterval </strong>integer<br> + instructs the UDP send action to rebind the send socket every nbr of messages sent. Zero, the default, means that no rebind is done. This directive is useful for use with load-balancers.<br></li><br> + + <li><strong>$ActionSendStreamDriver </strong><driver basename><br> + just like $DefaultNetstreamDriver, but for the specific action <br></li><br> + + <li><strong>$ActionSendStreamDriverMode </strong><mode> [default 0]<br> + mode to use with the stream driver (driver-specific)<br></li><br> + + <li><strong>$ActionSendStreamDriverAuthMode </strong><mode><br> + authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific))<br></li><br> + + <li><strong>$ActionSendStreamDriverPermittedPeers </strong><ID><br> + accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific) <br></li><br> + + <li><strong>$ActionSendResendLastMsgOnReconnect </strong>on/off [default off]<br> + specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. <br></li><br> + + <li><strong>$ResetConfigVariables </strong><br> + Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br> + +</ul> + +<p><b>Legacy Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">$ModLoad omfwd +*.* @@192.168.2.11:10514 +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/ompipe.html b/doc/ompipe.html new file mode 100644 index 00000000..49915b78 --- /dev/null +++ b/doc/ompipe.html @@ -0,0 +1,62 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>Pipe Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Pipe Output Module</h1> +<p><b>Module Name: omfwd</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The ompipe plug-in provides the core functionality for logging output to named pipes (fifos). It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>Pipe </strong>string<br> + A fifo or named pipe can be used as a destination for log messages.<br></li><br> + + + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">Module (path="builtin:ompipe") +*.* action(type="ompipe" +Pipe="NameofPipe" +) +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<p>rsyslog has support for logging output to named pipes (fifos). A fifo or named pipe can be used as a destination for log messages by prepending a pipe symbol ("|'') to the name of the file. This is handy for debugging. Note that the fifo must be created with the mkfifo(1) command before rsyslogd is started. + +</p> + +<p><b>Legacy Sample:</b></p> +<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> +<textarea rows="5" cols="60">$ModLoad ompipe +*.* |/var/log/pipe +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omsnmp.html b/doc/omsnmp.html index b38a594f..202bb5bb 100644 --- a/doc/omsnmp.html +++ b/doc/omsnmp.html @@ -22,6 +22,153 @@ developer (headers) package installed. </p> <p> </p> <p><b>Configuration Directives</b>:</p> <ul> + <li><strong>transport </strong>(This parameter is optional, the + default value is "udp")<br> + <br> + Defines the transport type you wish to use. Technically we can support all + transport types which are supported by NET-SNMP. <br> + To name a few possible values: <br> + <br> + udp, tcp, udp6, tcp6, icmp, icmp6 ...<br> + <br> + Example: <strong>transport udp<br> + </strong></li> + <li><strong>server</strong><br> + <br> + This can be a hostname or ip address, and is our snmp target host. This + parameter is required, if the snmptarget is not defined, nothing will be + send. <br> + <br> + Example: <strong>server server.domain.xxx</strong><br> + </li> + <li><strong>port </strong>(This parameter is optional, the + default value is "162")<br> + <br> + The port which will be used, common values are port 162 or 161. <br> + <br> + Example: <strong>port 162</strong><br> + </li> + <li><strong>version </strong>(This parameter is optional, the + default value is "1")<br> + <br> + There can only be two choices for this parameter for now. <br> + 0 means SNMPv1 will be used.<br> + 1 means SNMPv2c will be used. <br> + Any other value will default to 1. <br> + <br> + Example: <strong>version 1</strong><br> + </li> + <li><strong>community </strong>(This parameter is optional, the + default value is "public")<br> + <br> + This sets the used SNMP Community.<br> + <br> + Example:<strong> community public<br> + </strong><br> + </li> + <li><strong>trapoid </strong>(This parameter is + optional, the default value is "1.3.6.1.4.1.19406.1.2.1" which means + "ADISCON-MONITORWARE-MIB::syslogtrap")<br> + This configuration parameter is used for <strong>SNMPv2</strong> only.<br> + <br> + This is the OID which defines the trap-type, or notifcation-type rsyslog + uses to send the trap. <br> + In order to decode this OID, you will need to have the + ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. Downloads of these mib files + can be found here: <br> + <a href="http://www.adiscon.org/download/ADISCON-MIB.txt"> + http://www.adiscon.org/download/ADISCON-MIB.txt</a><br> + <a href="http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt"> + http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt</a><br> + <br> + Thanks to the net-snmp + mailinglist for the help and the recommendations ;).<br> + <br> + Example: <strong>trapoid 1.3.6.1.4.1.19406.1.2.1<br> + </strong>If you have this MIBS installed, you can also configured with the + OID Name: <strong>trapoid ADISCON-MONITORWARE-MIB::syslogtrap<br> + </strong> + </li> + <li><strong>messageoid </strong>(This parameter is + optional, the default value is "1.3.6.1.4.1.19406.1.1.2.1" which means + "ADISCON-MONITORWARE-MIB::syslogMsg")<br> + <br> + This OID will be used as a variable, type "OCTET STRING". This variable will + contain up to 255 characters of the original syslog message including syslog header. It is recommend to + use the default OID. <br> + In order to decode this OID, you will need to have the + ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. + To download these custom mibs, see the description of <strong>$actionsnmptrapoid. + </strong><br> + <br> + Example: <strong>messageoid 1.3.6.1.4.1.19406.1.1.2.1<br> + </strong>If you have this MIBS installed, you can also configured with the + OID Name: <strong>messageoid + ADISCON-MONITORWARE-MIB::syslogMsg<br> + </strong><br> + </li> + <li><strong>enterpriseoid </strong>(This parameter is optional, + the default value is "1.3.6.1.4.1.3.1.1" which means "enterprises.cmu.1.1")<br> + <br> + Customize this value if needed. I recommend to use the default value unless + you require to use a different OID. <br> + This configuration parameter is used for <strong>SNMPv1</strong> only. It + has no effect if <strong>SNMPv2</strong> is used. <br> + <br> + Example: <strong>enterpriseoid 1.3.6.1.4.1.3.1.1 <br> + </strong><br> + </li> + <li><strong>specifictype </strong>(This parameter is optional, + the default value is "0")<strong> </strong><br> + <br> + This is the specific trap number. This configuration parameter is used for + <strong>SNMPv1</strong> only. It has no effect if <strong>SNMPv2</strong> is + used. <br> + <br> + Example: <strong>specifictype 0<br> + </strong><br> + </li> + <li><strong>traptype</strong> (This parameter is optional, the + default value is "6" which means SNMP_TRAP_ENTERPRISESPECIFIC) <br> + <br> + There are only 7 Possible trap types defined which can be used here. These + trap types are: <br> + 0 = SNMP_TRAP_COLDSTART<br> + 1 = SNMP_TRAP_WARMSTART<br> + 2 = SNMP_TRAP_LINKDOWN<br> + 3 = SNMP_TRAP_LINKUP<br> + 4 = SNMP_TRAP_AUTHFAIL<br> + 5 = SNMP_TRAP_EGPNEIGHBORLOSS<br> + 6 = SNMP_TRAP_ENTERPRISESPECIFIC<br> + <br> + Any other value will default to 6 automatically. This configuration + parameter is used for <strong>SNMPv1</strong> only. It has no effect if + <strong>SNMPv2</strong> is used. <br> + <br> + Example: <strong>traptype 6</strong><br> + </li> + <li><strong>template </strong>[templateName]<strong> </strong><br> + <br> + sets a new default template for file actions. + </li> +</ul> +<p> </p> +<p><b>Caveats/Known Bugs:</b></p><ul><li>In order to decode the custom OIDs, you + will need to have the adiscon mibs installed. </li></ul> +<p><b>Sample:</b></p> +<p>The following commands send every message as a snmp trap.</p> +<textarea rows="10" cols="60">Module (path="omsnmp") +*.* action( type="omsnmp" +transport="udp" +target="localhost" +targetport="162" +version="1" +community="public") + +</textarea> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> <li><strong>$actionsnmptransport </strong>(This parameter is optional, the default value is "udp")<br> <br> @@ -164,6 +311,7 @@ $actionsnmpcommunity public *.* :omsnmp: </textarea> + <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the diff --git a/doc/omusrmsg.html b/doc/omusrmsg.html new file mode 100644 index 00000000..eccfef2d --- /dev/null +++ b/doc/omusrmsg.html @@ -0,0 +1,64 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>User Message Output Module</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>User Message Output Module</h1> +<p><b>Module Name: omusrmsg</b></p> +<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> +<p><b>Description</b>:</p> +<p>The omusrmsg plug-in provides the core functionality for logging output to a logged on user. It is a built-in module that does not need to be loaded. </p> +<p> </p> + +<p><b>Global Configuration Directives</b>:</p> +<ul> + <li><strong>Template </strong>[templateName]<br> + sets a new default template for file actions.<br></li> + +</ul> +<p> </p> +<p><b>Action specific Configuration Directives</b>:</p> +<ul> + <li><strong>Users </strong>string<br> + Must be a valid user name or root.<br></li><br> + + + +</ul> +<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> +<p><b>Sample:</b></p> +<p>The following command sends all critical syslog messages to a user and to root.</p> +<textarea rows="5" cols="60">Module (path="builtin:omusrmsg") +*.=crit action(type="omusrmsg" +Users="ExampleUser" +Users="root" +) +</textarea> + +<br><br> + +<p><b>Legacy Configuration Directives</b>:</p> +<p> + No specific configuration directives available. See configuration sample below for details on using the plugin. +</p> + +<p><b>Legacy Sample:</b></p> +<p>The following command sends all critical syslog messages to a user and to root.</p> +<textarea rows="5" cols="60">$ModLoad omusrmsg +*.=crit :omusrmsg:exampleuser +& root +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omuxsock.html b/doc/omuxsock.html index 5fa569eb..a1c09228 100644 --- a/doc/omuxsock.html +++ b/doc/omuxsock.html @@ -29,7 +29,7 @@ actions and each of them should use the same template.</li> <p><b>Sample:</b></p> <p>The following sample writes all messages to the "/tmp/socksample" socket. </p> -<textarea rows="4" cols="80">$ModLoad omucsock +<textarea rows="4" cols="80">$ModLoad omuxsock $OMUxSockSocket /tmp/socksample *.* :omuxsock: </textarea> diff --git a/doc/property_replacer.html b/doc/property_replacer.html index 86a07474..dc09d33c 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -236,7 +236,7 @@ or none if no rule with this field did match. <p>Properties starting with a $-sign are so-called system properties. These do NOT stem from the message but are rather internally-generated.</p> -<h2>Character Positions</h2> +<h2>Legacy Character Positions</h2> <p><b><code>FromChar</code></b> and <b><code>toChar</code></b> are used to build substrings. They specify the offset within the string that should be copied. Offset counting starts at 1, so if you need to @@ -304,15 +304,6 @@ fields in the property is requested. The field number must be placed in the "ToChar" parameter. An example where the 3rd field (delimited by TAB) from the msg property is extracted is as follows: "%msg:F:3%". The same example with semicolon as delimiter is "%msg:F,59:3%".</p> -<p>The use of fields does not permit to select substrings, what is rather -unfortunate. To solve this issue, starting with 6.3.9, fromPos and toPos -can be specified for strings as well. However, the syntax is quite ugly, but -it was the only way to integrate this functonality into the already-existing -system. To do so, use ",fromPos" and ",toPos" during field extraction. -Let's assume you want to extract the substring from position 5 to 9 in the previous -example. Then, the syntax is as follows: "%msg:F,59,5:3,9%". As you can see, -"F,59" means field-mode, with semicolon delimiter and ",5" means starting -at position 5. Then "3,9" means field 3 and string extraction to position 9. <p>Please note that the special characters "F" and "R" are case-sensitive. Only upper case works, lower case will return an error. There are no white spaces permitted inside the sequence (that will lead @@ -342,6 +333,243 @@ It is modeled after perl compatible regular expressions. <h2>Property Options</h2> <b><code>property options</code></b> are +case-insensitive. They are available as of version 6.5.0. +Currently, the following options are defined: +<p></p> +<table> +<tbody> +<tr> +<td><b>Name</b></td> +<td>New format. Name of the template / property / constant.</td> +</tr> +<tr> +<td><b>Outname</b></td> +<td>This field permits to specify a field name for structured-data emitting property replacer options. +It is most useful to set, for example, the name for JSON-based fields (like used in ommngodb). For +text-based modules, it is simply ignored. +If not specified, the original property name is used, with the exception of properties starting with +"$!", where that prefix is removed. Note that unnamaned constants are NOT forwarded to output modules +that expect structure (like ommnogodb). To pass constants, an outname must be set. +</tr> +<tr> +<td><b>CaseConversion</b></td> +<td>New format. Additional values below.</td> +</tr> +<tr> +<td>upper</td> +<td>convert property to lowercase only</td> +</tr> +<tr> +<td>lower</td> +<td>convert property text to uppercase only</td> +</tr> +<tr> +<td><b>DateFormat</b></td> +<td>New format, additional parameter is needed. See below.</td> +</tr> +<tr> +<td>mysql</td> +<td>format as mysql date</td> +</tr> +<tr> +<td>pgsql</td> +<td>format as pgsql date</td> +</tr> +<tr> +<td>rfc3164</td> +<td>format as RFC 3164 date</td> +</tr> +<tr> +<tr> +<td valign="top">rfc3164-buggyday</td> +<td>similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands +that a space is written for single-digit days. With this option, a zero is +written instead. This format seems to be used by syslog-ng and the +date-rfc3164-buggyday option can be used in migration scenarios where otherwise +lots of scripts would need to be adjusted. It is recommended <i>not</i> to use this +option when forwarding to remote hosts - they may treat the date as invalid +(especially when parsing strictly according to RFC 3164).</td> +<br><i>This feature was introduced in rsyslog 4.6.2 and v4 versions above and +5.5.3 and all versions above.</i> +</tr> +<tr> +<td>rfc3339</td> +<td>format as RFC 3339 date</td> +</tr> +<tr> +<td>unixtimestamp</td> +<td>format as unix timestamp (seconds since epoch)</td> +</tr> +<tr> +<td>subseconds</td> +<td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td> +</tr> +<tr> +<td><b>ControlCharacters</b></td> +<td>Option values for how to process control characters</td> +</tr> +<tr> +<td valign="top">escape</td> +<td>replace control characters (ASCII value 127 and values +less then 32) with an escape sequence. The sequnce is +"#<charval>" where charval is the 3-digit decimal value +of the control character. For example, a tabulator would be replaced by +"#009".<br> +Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> +is set to off.</td> +</tr> +<tr> +<td valign="top">space</td> +<td>replace control characters by spaces<br> +Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> +is set to off.</td> +</tr> +<tr> +<td valign="top">drop</td> +<td>drop control characters - the resulting string will +neither contain control characters, escape sequences nor any other +replacement character like space.<br> +Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> +is set to off.</td> +</tr> +<tr> +<td><b>SecurePath</b></td> +<td>Option values for securing path templates.</td> +</tr> +<tr> +<td valign="top">drop</td> +<td>Drops slashes inside the field (e.g. "a/b" becomes "ab"). +Useful for secure pathname generation (with dynafiles). +</td> +</tr> +<tr> +<td valign="top">replace</td> +<td>Replace slashes inside the field by an underscore. (e.g. "a/b" becomes "a_b"). +Useful for secure pathname generation (with dynafiles). +</td> +</tr> +<tr> +<td><b>Format</b></td> +<td>Option values for the general output format.</td> +</tr> +<tr> +<td>json</td> +<td>encode the value so that it can be used inside a JSON field. This means +that several characters (according to the JSON spec) are being escaped, for +example US-ASCII LF is replaced by "\n". +The json option cannot be used together with either jsonf or csv options. +</td> +</tr> +<tr> +<td>jsonf</td> +<td><i>(available in 6.3.9+)</i> +This signifies that the property should be expressed as a json <b>f</b>ield. +That means not only the property is written, but rather a complete json field in +the format<br> +"fieldname"="value"</b> +where "filedname" is the assigend field name (or the property name if none was assigned) +and value is the end result of property replacer operation. Note that value supports +all property replacer options, like substrings, case converson and the like. +Values are properly json-escaped. However, field names are (currently) not. It is +expected that proper field names are configured. +The jsonf option cannot be used together with either json or csv options. +</td> +</tr> +<tr> +<td valign="top">csv</td> +<td>formats the resulting field (after all modifications) in CSV format +as specified in <a href="http://www.ietf.org/rfc/rfc4180.txt">RFC 4180</a>. +Rsyslog will always use double quotes. Note that in order to have full CSV-formatted +text, you need to define a proper template. An example is this one: +<br>$template csvline,"%syslogtag:::csv%,%msg:::csv%" +<br>Most importantly, you need to provide the commas between the fields +inside the template. +The csv option cannot be used together with either json or jsonf options. +<br><i>This feature was introduced in rsyslog 4.1.6.</i> +</td> +</tr> +<tr> +<td><b>droplastlf</b></td> +<td>The last LF in the message (if any), is dropped. +Especially useful for PIX.</td> +</tr> +<tr> +<td valign="top"><b>spifno1stsp</b></td> +<td>This option looks scary and should probably not be used by a user. For any field +given, it returns either a single space character or no character at all. Field content +is never returned. A space is returned if (and only if) the first character of the +field's content is NOT a space. This option is kind of a hack to solve a problem rooted +in RFC 3164: 3164 specifies no delimiter between the syslog tag sequence and the actual +message text. Almost all implementation in fact delemit the two by a space. As of +RFC 3164, this space is part of the message text itself. This leads to a problem when +building the message (e.g. when writing to disk or forwarding). Should a delimiting +space be included if the message does not start with one? If not, the tag is immediately +followed by another non-space character, which can lead some log parsers to misinterpret +what is the tag and what the message. The problem finally surfaced when the klog module +was restructured and the tag correctly written. It exists with other message sources, +too. The solution was the introduction of this special property replacer option. Now, +the default template can contain a conditional space, which exists only if the +message does not start with one. While this does not solve all issues, it should +work good enough in the far majority of all cases. If you read this text and have +no idea of what it is talking about - relax: this is a good indication you will never +need this option. Simply forget about it ;) +</td> +</tr> +<tr> +<td></td> +<td></td> +</tr> +<tr> +<td><b>New character position</b></td> +<td>In addition to the above mentioned Character Positions in the legacy format, +positions can be determined by specifying the correct options for the properties. +Again, this is mostly for using the list format.</td> +</tr> +<tr> +<td>position.From</td> +<td>Character position in the property to start from.</td> +</tr> +<tr> +<td>position.To</td> +<td>Character position that determines the end for extraction. If the value is "$" +then the end of the string will be used.</td> +</tr> +<tr> +<td>field.Number</td> +<td>The number of the field, which should be used for the search operation with Regex.</td> +</tr> +<tr> +<td>field.Delimiter</td> +<td>The Character that should delimit a field. Example: ",". Everything in a +property until this character is considered a field.</td> +</tr> +<tr> +<td>regex.Expression</td> +<td>Value to be compared to property.</td> +</tr> +<tr> +<td>regex.Type</td> +<td>Values BRE or ERE</td> +</tr> +<tr> +<td>regex.NoMatchMode</td> +<td>DFLT, BLANK, ZERO, FIELD</td> +</tr> +<tr> +<td>regex.Match</td> +<td>Match to use.</td> +</tr> +<tr> +<td>regex.Submatch</td> +<td>Submatch to use. Values 0-9 whereas 0 = All</td> +</tr> +</tbody> +</table> + + + +<h2>Legacy Property Options</h2> +<b><code>property options</code></b> are case-insensitive. Currently, the following options are defined: <p></p> <table> @@ -368,7 +596,7 @@ The json option cannot be used together with either jsonf or csv options. This signifies that the property should be expressed as a json <b>f</b>ield. That means not only the property is written, but rather a complete json field in the format<br> -"fieldname":"value"</b> +"fieldname"="value"</b> where "filedname" is the assigend field name (or the property name if none was assigned) and value is the end result of property replacer operation. Note that value supports all property replacer options, like substrings, case converson and the like. @@ -486,6 +714,12 @@ Useful for secure pathname generation (with dynafiles). Useful for secure pathname generation (with dynafiles). </td> </tr> +<tr> +<td><b>mandatory-field</b></td> +<td>In templates that are used for building field lists (in particular, ommongodb), include +this field, even if it is empty (or NULL). If not set, the field will be removed from +the output field set if empty. The latter is the default case. +</tr> </tbody> </table> <p>To use multiple options, simply place them one after each other with a comma delmimiting diff --git a/doc/rainerscript.html b/doc/rainerscript.html index 63a79040..84261bdd 100644 --- a/doc/rainerscript.html +++ b/doc/rainerscript.html @@ -58,6 +58,17 @@ recommended. variable, if it exists. Returns an empty string if it does not exist. <li>strlen(str) - returns the length of the provided string <li>tolower(str) - converts the provided string into lowercase +<li>cstr(expr) - converts expr to a string value +<li>cnum(expr) - converts expr to a number (integer) +<li>re_match(expr, re) - returns 1, if expr matches re, 0 otherwise +<li>field(str, delim, matchnbr) - returns a field-based substring. str is the string +to search, delim is the numerical ascii value of the field delimiter (so that +non-printable characters can by specified) and matchnbr is the match to search +for (the first match starts at 1). This works similar as the field based +property-replacer option. +<li>prifilt(constant) - mimics a traditional PRI-based filter (like "*.*" or +"mail.info"). The traditional filter string must be given as a <b>constant string</b>. +Dynamic string evaluation is not permitted (for performance reasons). </ul> <p>The following example can be used to build a dynamic filter based on some environment variable: @@ -69,7 +80,7 @@ if $msg contains getenv('TRIGGERVAR') then /path/to/errfile <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body></html> diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index fbced4a3..3efa3967 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -4,38 +4,95 @@ <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">back</a> <h2>Filter Conditions</h2> -<p>Rsyslog offers four different types "filter conditions":</p> +<p>Rsyslog offers three different types "filter conditions":</p> <ul> -<li>BSD-style blocks</li> +<li><a href="http://www.rainerscript.com/">RainerScript</a>-based filters</li> <li>"traditional" severity and facility based selectors</li> <li>property-based filters</li> -<li>expression-based filters</li> </ul> -<h3>Blocks</h3> -<p>Rsyslogd supports BSD-style blocks inside rsyslog.conf. Each -block of lines is separated from the previous block by a program or -hostname specification. A block will only log messages corresponding to -the most recent program and hostname specifications given. Thus, a -block which selects ‘ppp’ as the program, directly followed by a block -that selects messages from the hostname ‘dialhost’, then the second -block will only log messages from the ppp program on dialhost. -</p> -<p>A program specification is a line beginning with ‘!prog’ and -the following blocks will be associated with calls to syslog from that -specific program. A program specification for ‘foo’ will also match any -message logged by the kernel with the prefix ‘foo: ’. Alternatively, a -program specification ‘-foo’ causes the following blocks to be applied -to messages from any program but the one specified. A hostname -specification of the form ‘+hostname’ and the following blocks will be -applied to messages received from the specified hostname. -Alternatively, a hostname specification ‘-hostname’ causes the -following blocks to be applied to messages from any host but the one -specified. If the hostname is given as ‘@’, the local hostname will be -used. (NOT YET IMPLEMENTED) A program or hostname specification may be -reset by giving the program or hostname as ‘*’.</p> -<p>Please note that the "#!prog", "#+hostname" and "#-hostname" -syntax available in BSD syslogd is not supported by rsyslogd. By -default, no hostname or program is set.</p> +<h3>RainerScript-Based Filters</h3> +RainerScript based filters are the prime means of creating complex rsyslog configuration. +The permit filtering on arbitrary complex expressions, which can include boolean, +arithmetic and string operations. They also support full nesting of filters, just +as you know from other scripting environments. +<br> +Scripts based filters are indicated by the keyword "if", as usual. +They have this format:<br> +<br> +if expr then block else block +<br> +"If" and "then" are fixed keywords that mus be present. "expr" is a +(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for +details. +The keyword "else" and its associated block is optional. Note that a block can contain either +a single action (chain), or an arbitrary complex script enclosed in curly braces, e.g.: +<br> +<pre> +if $programname == 'prog1' then { + action(type="omfile" file="/var/log/prog1.log") + if $msg contains 'test' then + action(type="omfile" file="/var/log/prog1test.log") + else + action(type="omfile" file="/var/log/prog1notest.log") +} +</pre> +<br> +Other types of filtes can also be combined with the pure RainerScript ones. This makes +it particularly easy to migrate from early config files to RainerScript. Also, the traditional +syslog PRI-based filters are a good and easy to use addition. While they are legacy, we still +recommend there use where they are up to the job. We do NOT, however, recommend property-based +filters any longer. As an example, the following is perfectly valid: +<br> +<pre> +if $fromhost == 'host1' then { + mail.* action(type="omfile" file="/var/log/host1/mail.log") + *.err /var/log/host1/errlog # this is also still valid + # + # more "old-style rules" ... + # +} else { + mail.* action(type="omfile" file="/var/log/mail.log") + *.err /var/log/errlog + # + # more "old-style rules" ... + # +} +</pre> +<br> + +Right now, you need to specify numerical values if you would like to +check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. +If you don't like that, you can of course also use the textual property +- just be sure to use the right one. As expression support is enhanced, +this will change. For example, if you would like to filter on message +that have facility local0, start with "DEVNAME" and have either +"error1" or "error0" in their message content, you could use the +following filter:<br> +<br> +<code> +if $syslogfacility-text == 'local0' and $msg +startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains +'error0') then /var/log/somelog<br> +</code> +<br> +Please note that the above <span style="font-weight: bold;">must +all be on one line</span>! And if you would like to store all +messages except those that contain "error1" or "error0", you just need +to add a "not":<br> +<br> +<code> +if $syslogfacility-text == 'local0' and $msg +startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> +($msg contains 'error1' or $msg contains +'error0') then /var/log/somelog<br> +</code> +<br> +If you would like to do case-insensitive comparisons, use +"contains_i" instead of "contains" and "startswith_i" instead of +"startswith".<br> +<br> +Regular expressions are supported via functions (see function list). + <h3>Selectors</h3> <p><b>Selectors are the traditional way of filtering syslog messages.</b> They have been kept in rsyslog with their original @@ -213,71 +270,6 @@ supported (except for "not" as outlined above). Please note that while it is possible to query facility and severity via property-based filters, it is far more advisable to use classic selectors (see above) for those cases.</p> -<h3>Expression-Based Filters</h3> -Expression based filters allow -filtering on arbitrary complex expressions, which can include boolean, -arithmetic and string operations. Expression filters will evolve into a -full configuration scripting language. Unfortunately, their syntax will -slightly change during that process. So if you use them now, you need -to be prepared to change your configuration files some time later. -However, we try to implement the scripting facility as soon as possible -(also in respect to stage work needed). So the window of exposure is -probably not too long.<br> -<br> -Expression based filters are indicated by the keyword "if" in column 1 -of a new line. They have this format:<br> -<br> -if expr then action-part-of-selector-line<br> -<br> -"If" and "then" are fixed keywords that mus be present. "expr" is a -(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for -details. "action-part-of-selector-line" is an action, just as you know -it (e.g. "/var/log/logfile" to write to that file).<br> -<br> -A few quick samples:<br> -<br> -<code> -*.* /var/log/file1 # the traditional way<br> -if $msg contains 'error' then /var/log/errlog # the expression-based way<br> -</code> -<br> -Right now, you need to specify numerical values if you would like to -check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. -If you don't like that, you can of course also use the textual property -- just be sure to use the right one. As expression support is enhanced, -this will change. For example, if you would like to filter on message -that have facility local0, start with "DEVNAME" and have either -"error1" or "error0" in their message content, you could use the -following filter:<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -Please note that the above <span style="font-weight: bold;">must -all be on one line</span>! And if you would like to store all -messages except those that contain "error1" or "error0", you just need -to add a "not":<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> -($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -If you would like to do case-insensitive comparisons, use -"contains_i" instead of "contains" and "startswith_i" instead of -"startswith".<br> -<br> -Note that regular expressions are currently NOT -supported in expression-based filters. These will be added later when -function support is added to the expression engine (the reason is that -regular expressions will be a separate loadable module, which requires -some more prequisites before it can be implemented).<br> - <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> diff --git a/doc/rsyslog_conf_global.html b/doc/rsyslog_conf_global.html index 6c20f4c2..a4d760eb 100644 --- a/doc/rsyslog_conf_global.html +++ b/doc/rsyslog_conf_global.html @@ -52,10 +52,10 @@ This directive will timeout previous messages seen if they are older than 20 minutes. In the example above, the count would now be always 1 and consequently no rule would ever be triggered. -<li>$ActionFileDefaultTemplate [templateName] - sets a new default template for file actions</li> -<li>$ActionFileEnableSync [on/<span style="font-weight: bold;">off</span>] - enables file +<li><a href="omfile.html">$ActionFileDefaultTemplate</a> [templateName] - sets a new default template for file actions</li> +<li><a href="omfile.html">$ActionFileEnableSync</a> [on/<span style="font-weight: bold;">off</span>] - enables file syncing capability of omfile</li> -<li>$ActionForwardDefaultTemplate [templateName] - sets a new +<li><a href="omfwd.html">$ActionForwardDefaultTemplate</a> [templateName] - sets a new default template for UDP and plain TCP forwarding action</li> <li>$ActionGSSForwardDefaultTemplate [templateName] - sets a new default template for GSS-API forwarding action</li> @@ -93,23 +93,23 @@ default 60000 (1 minute)]</li> <li>$ActionQueueWorkerThreadMinumumMessages <number>, default 100</li> <li><a href="rsconf1_actionresumeinterval.html">$ActionResumeInterval</a></li> <li>$ActionResumeRetryCount <number> [default 0, -1 means eternal]</li> -<li>$ActionSendResendLastMsgOnReconnect <[on/<b>off</b>]> specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. -<li>$ActionSendStreamDriver <driver basename> just like $DefaultNetstreamDriver, but for the specific action</li> -<li>$ActionSendStreamDriverMode <mode>, default 0, mode to use with the stream driver (driver-specific)</li> -<li>$ActionSendStreamDriverAuthMode <mode>, authentication mode to use with the stream driver. Note that this directive requires TLS +<li><a href="omfwd.html">$ActionSendResendLastMsgOnReconnect</a> <[on/<b>off</b>]> specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. +<li><a href="omfwd.html">$ActionSendStreamDriver</a> <driver basename> just like $DefaultNetstreamDriver, but for the specific action</li> +<li><a href="omfwd.html">$ActionSendStreamDriverMode</a> <mode>, default 0, mode to use with the stream driver (driver-specific)</li> +<li><a href="omfwd.html">$ActionSendStreamDriverAuthMode</a> <mode>, authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific)</li> -<li>$ActionSendStreamDriverPermittedPeer <ID>, accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS +<li><a href="omfwd.html">$ActionSendStreamDriverPermittedPeer</a> <ID>, accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific) -<span style="font-weight: bold;"> directive may go away</span>!</li> -<li><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send +<li><a href="omfwd.html"><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send action to close and re-open the connection to the remote host every nbr of messages sent. Zero, the default, means that no such processing is done. This directive is useful for use with load-balancers. Note that there is some performance overhead associated with it, so it is advisable to not too often "rebind" the connection (what "too often" actually means depends on your configuration, a rule of thumb is that it should be not be much more often than once per second).</li> -<li><b>$ActionSendUDPRebindInterval</b> nbr</a>- [available since 4.3.2] - instructs the UDP send +<li><a href="omfwd.html"><b>$ActionSendUDPRebindInterval</b> nbr</a>- [available since 4.3.2] - instructs the UDP send action to rebind the send socket every nbr of messages sent. Zero, the default, means that no rebind is done. This directive is useful for use with load-balancers.</li> <li><b>$ActionWriteAllMarkMessages</b> [on/<b>off</b>]- [available since 5.1.5] - normally, mark messages @@ -133,22 +133,22 @@ functionality. the provided <i>name</i> (the default default ruleset is named "RSYSLOG_DefaultRuleset"). It is advised to also read our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li> -<li><b>$CreateDirs</b> [<b>on</b>/off] - create directories on an as-needed basis</li> -<li><a href="rsconf1_dircreatemode.html">$DirCreateMode</a></li> -<li><a href="rsconf1_dirgroup.html">$DirGroup</a></li> -<li><a href="rsconf1_dirowner.html">$DirOwner</a></li> +<li><a href="omfile.html"><b>$CreateDirs</b></a> [<b>on</b>/off] - create directories on an as-needed basis</li> +<li><a href="omfile.html">$DirCreateMode</a></li> +<li><a href="omfile.html">$DirGroup</a></li> +<li><a href="omfile.html">$DirOwner</a></li> <li><a href="rsconf1_dropmsgswithmaliciousdnsptrrecords.html">$DropMsgsWithMaliciousDnsPTRRecords</a></li> <li><a href="rsconf1_droptrailinglfonreception.html">$DropTrailingLFOnReception</a></li> -<li><a href="rsconf1_dynafilecachesize.html">$DynaFileCacheSize</a></li> +<li><a href="omfile.html">$DynaFileCacheSize</a></li> <li><a href="rsconf1_escape8bitcharsonreceive.html">$Escape8BitCharactersOnReceive</a></li> <li><a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a></li> <li><b>$EscapeControlCharactersOnReceive</b> [<b>on</b>|off] - escape USASCII HT character</li> <li>$SpaceLFOnReceive [on/<b>off</b>] - instructs rsyslogd to replace LF with spaces during message reception (sysklogd compatibility aid)</li> <li>$ErrorMessagesToStderr [<b>on</b>|off] - direct rsyslogd error message to stderr (in addition to other targets)</li> -<li><a href="rsconf1_failonchownfailure.html">$FailOnChownFailure</a></li> -<li><a href="rsconf1_filecreatemode.html">$FileCreateMode</a></li> -<li><a href="rsconf1_filegroup.html">$FileGroup</a></li> -<li><a href="rsconf1_fileowner.html">$FileOwner</a></li> +<li><a href="omfile.html">$FailOnChownFailure</a></li> +<li><a href="omfile.html">$FileCreateMode</a></li> +<li><a href="omfile.html">$FileGroup</a></li> +<li><a href="omfile.html">$FileOwner</a></li> <li><a href="rsconf1_generateconfiggraph.html">$GenerateConfigGraph</a></li> <li><a href="rsconf1_gssforwardservicename.html">$GssForwardServiceName</a></li> <li><a href="rsconf1_gsslistenservicename.html">$GssListenServiceName</a></li> @@ -224,7 +224,7 @@ supported in order to be compliant to the upcoming new syslog RFC series. <li><a href="rsconf1_maxopenfiles.html">$MaxOpenFiles</a></li> <li><a href="rsconf1_moddir.html">$ModDir</a></li> <li><a href="rsconf1_modload.html">$ModLoad</a></li> -<li><b>$OMFileAsyncWriting</b> [on/<b>off</b>], if turned on, the files will be written +<li><a href="omfile.html"><b>$OMFileAsyncWriting</b></a> [on/<b>off</b>], if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable $OMFileFlushInterval, $OMFileAsyncWriting must be set to "on". Otherwise, the flush @@ -232,11 +232,11 @@ interval will be ignored. Also note that when $OMFileFlushOnTXEnd is "on" but $OMFileAsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. -<li><b>$OMFileZipLevel</b> 0..9 [default 0] - if greater 0, turns on gzip compression +<li><a href="omfile.html"><b>$OMFileZipLevel</b></a> 0..9 [default 0] - if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.</li> -<li><b>$OMFileIOBufferSize</b> <size_nbr>, default 4k, size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)</li> -<li><b>$OMFileFlushOnTXEnd</b> <[<b>on</b>/off]>, default on. Omfile has the +<li><a href="omfile.html"><b>$OMFileIOBufferSize</b></a> <size_nbr>, default 4k, size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)</li> +<li><a href="omfile.html"><b>$OMFileFlushOnTXEnd</b></a> <[<b>on</b>/off]>, default on. Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where @@ -246,7 +246,7 @@ error recovery thus can handle write errors without data loss. Note that this op severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.</li> -<li><a href="rsconf1_omfileforcechown.html">$omfileForceChown</a> - force ownership change for all files</li> +<li><a href="omfile.html">$omfileForceChown</a> - force ownership change for all files</li> <li><b>$RepeatedMsgContainsOriginalMsg</b> [on/<b>off</b>] - "last message repeated n times" messages, if generated, have a different format that contains the message that is being repeated. Note that only the first "n" characters are included, with n to be at least 80 characters, most diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 650e20ad..cbd60faf 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -51,6 +51,10 @@ to message generators. <p>Output modules process messages. With them, message formats can be transformed and messages be transmitted to various different targets. <ul> +<li><a href="omfile.html">omfile</a> - file output module</li> +<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li> +<li><a href="ompipe.html">ompipe</a> - named pipe output module</li> +<li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li> <li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> <li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li> <li><a href="omrelp.html">omrelp</a> - RELP output module</li> diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html index bd0b3253..4360c691 100644 --- a/doc/rsyslog_conf_templates.html +++ b/doc/rsyslog_conf_templates.html @@ -3,7 +3,7 @@ <body> <p>This is a part of the rsyslog.conf - documentation.</p> <a href="rsyslog_conf.html">back</a> -<h2>Templates</h2> +<h1>Templates</h1> <p>Templates are a key feature of rsyslog. They allow to specify any format a user might want. They are also used for dynamic file name @@ -16,7 +16,161 @@ compatible with the stock syslogd formats are hardcoded into rsyslogd. So if no template is specified, we use one of these hardcoded templates. Search for "template_" in syslogd.c and you will find the hardcoded ones.</p> -<p>Starting with 5.5.6, there are actually two differnt types of template: +<p>Templates are specified by template() statements. They can also be specified +via $Template legacy statements. Note that these are scheduled for removal in +later versions of rsyslog, so it is probably a good idea to avoid them +for new uses. +<h2>The template() statement</h2> +<p>The template() statement is used to define templates. Note that it is a +<b>static</b> statement, that means all templates are defined when rsyslog +reads the config file. As such, templates are not affected by if-statements +or config nesting. +<p>The basic structure of the template statement is as follows: +<br><br> +<code>template(parameters)</code> +<br><br> +In addition to this simpler syntax, list templates (to be described below) +support an extended syntax: +<br><br> +<code>template(parameters) { list-descriptions }</code> +<p>Each template has a parameter <b>name</b>, which specifies the templates +name, and a parameter <b>type</b>, which specifies the template type. The name +parameter must be unique, and behaviour is unpredictable if it is not. The <b>type</b> +parameter specifies different template types. Different types simply enable +different ways to specify the template content. The template type <b>does not</b> +affect what an (output) plugin can do with it. So use the type that best fits your +needs (from a config writing point of view!). The following types are available: +<ul> +<li>list +<li>subtree +<li>string +<li>plugin +</ul> +The various types are described below. + +<h3>list</h3> +<p>In this case, the template is generated by a list of constant and +variable statements. These follow the template spec in curly braces. This type is +also primarily meant for use with structure-aware outputs, like ommongodb. However, +it also works perfectly with text-based outputs. We recommend to use this mode +if more complex property substitutions needs to be done. In that case, the list-based +template syntax is much clearer than the simple string-based one. +<b>subtree</b> must be specified, which tells which subtree to use. For example +template(name="tpl1" type="subtree" subtree="$!") includes all CEE data, while +template(name="tpl2" type="subtree" subtree="$!usr!tpl2") includes only the +subtree starting at $!usr!tpl2. The core idea when using this type of template +is that the actual data is prefabricated via set and unset script statements, +and the resulting strucuture is then used inside the template. This method MUST +be used if a complete subtree needs to be placed <i>directly</i> into the +object's root. With all other template types, only subcontainers can be generated. +Note that subtree type can also be used with text-based outputs, like omfile. HOWEVER, +you do not have any capability to specify constant text, and as such cannot include +line breaks. As a consequence, using this template type for text outputs is usually +only useful for debugging or very special cases (e.g. where the text is interpreted +by a JSON parser later on). + +<h3>subtree</h3> +<p>Available since rsyslog 7.1.4 +<p> +In this case, the template is generated based on a complete +(CEE) subtree. This type of template is most useful for outputs that know how to +process hierarchical structure, like ommongodb. With that type, the parameter + +<h3>string</h3> +<p>This closely resembles the legacy template statement. It +has a mandatory parameter <b>string</b>, which holds the template string to be +applied. A template string is a mix of constant text and replacement variables +(see property replacer). These variables are taken from message or other dynamic +content when the final string to be passed to a plugin is generated. String-based +templates are a great way to specify textual content, especially if no complex +manipulation to properties is necessary. Full details on how to specify template +text can be found below. + +<h3>plugin</h3> +In this case, the template is generated by a plugin (which +is then called +a "strgen" or "string generator"). The format is fix as it is coded. While this +is inflexible, it provides superior performance, and is often used for that +reason (not that "regular" templates are slow - but in very demanding environments +that "last bit" can make a difference). Refer to the plugin's documentation +for further details. For this type, the paramter <b>plugin</b> must be specified and +must contain the name of the plugin as it identifies itself. Note that the +plugin must be loaded prior to being used inside a template. + +<h2>legacy format</h2> +<p>In pre v6-versions of rsyslog, you need to use the <code>$template</code> +statement to configure templates. They provide the equivalent to string- and +plugin-based templates. The legacy syntax continous to work in v7, however +we recommend to avoid legacy format for newly written config files. Legacy and +current config statements can coexist within the same config file. +<p>The general format is +<br><br><code>$template name,param[,options]</code></br></br> +where "name" is the template name and "param" is a single parameter +that specifies template content. The optional "options" part is used to +set template options. +<h3>string</h3> +The parameter is the same string that with the current-style format you +specify in the <b>string</b> parameter, for example: +<br><br><code>$template strtpl,"PRI: %pri%, MSG: %msg%\n"</code> +<p>Note that list templates are not available in legacy format, so you need +to use complex property replacer constructs to do complex things. + +<h3>plugin</h3> +This is equivalent to the "plugin"-type template directive. Here, the +parameter is the plugin name, with an equal sign prepended. An example +is: +<br><br><code>$template plugintpl,=myplugin</code> + +<h2>Reserved Template Names</h2> +<p>Template +names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT +use them if, otherwise you may receive a conflict in the future (and +quite unpredictable behaviour). There is a small set of pre-defined +templates that you can use without the need to define it:</p> +<ul> +<li><span style="font-weight: bold;">RSYSLOG_TraditionalFileFormat</span> +- the "old style" default log file format with low-precision timestamps</li> +<li><span style="font-weight: bold;">RSYSLOG_FileFormat</span> +- a modern-style logfile format similar to TraditionalFileFormat, buth +with high-precision timestamps and timezone information</li> +<li><span style="font-weight: bold;">RSYSLOG_TraditionalForwardFormat</span> +- the traditional forwarding format with low-precision timestamps. Most +useful if you send messages to other syslogd's or rsyslogd +below +version 3.12.5.</li> +<li><span style="font-weight: bold;">RSYSLOG_SysklogdFileFormat</span> +- sysklogd compatible log file format. If used with options: $SpaceLFOnReceive on; +$EscapeControlCharactersOnReceive off; $DropTrailingLFOnReception off, +the log format will conform to sysklogd log format.</li> +<li><span style="font-weight: bold;">RSYSLOG_ForwardFormat</span> +- a new high-precision forwarding format very similar to the +traditional one, but with high-precision timestamps and timezone +information. Recommended to be used when sending messages to rsyslog +3.12.5 or above.</li> +<li><span style="font-weight: bold;">RSYSLOG_SyslogProtocol23Format</span> +- the format specified in IETF's internet-draft +ietf-syslog-protocol-23, which is assumed to be come the new syslog +standard RFC. This format includes several improvements. The rsyslog +message parser understands this format, so you can use it together with +all relatively recent versions of rsyslog. Other syslogd's may get +hopelessly confused if receiving that format, so check before you use +it. Note that the format is unlikely to change when the final RFC comes +out, but this may happen.</li> +<li><span style="font-weight: bold;">RSYSLOG_DebugFormat</span> +- a special format used for troubleshooting property problems. This format +is meant to be written to a log file. Do <b>not</b> use for production or remote +forwarding.</li> +</ul> + +<h2>The following is legacy documentation soon to be integrated.</h2> + +<!--<table> +<tr><td>param name</td><td>meaning</td></tr> +<tr><td>name</td><td>name of the template</td></tr> +</table> +--> + +<p>Starting with 5.5.6, there are actually two different types of template: <ul> <li>string based <li>string-generator module based @@ -33,26 +187,31 @@ string generator module, you need to know how to call it. Each such module has a which you need to know (look it up in the module doc or ask the developer). Let's assume that "mystrgen" is the module name. Then you can define a template for that strgen in the following way: + +<blockquote><code>template(name="MyTemplateName" type="plugin" string="mystrgen")</code></blockquote> +<p>Legacy example:</p> <blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote> (Of course, you must have first loaded the module via $ModLoad). -<p>The important part is the equal sign: it tells the rsyslog config parser that +<p>The important part is the equal sign in the legacy format: it tells the rsyslog config parser that no string follows but a strgen module name. <p>There are no additional parameters but the module name supported. This is because there is no way to customize anything inside such a "template" other than by modifying the code of the string generator. <p>So for most use cases, string-generator module based templates are <b>not</b> -the route to take. Usually, us use <b>string based templates</b> instead. +the route to take. Usually, we use <b>string based templates</b> instead. This is what the rest of the documentation now talks about. <p>A template consists of a template directive, a name, the actual template text and optional options. A sample is:</p> +<blockquote><code>template(name="MyTemplateName" type="string" string="Example: Text %property% some more text\n" options)</code></blockquote> +<p>Legacy example:</p> <blockquote><code>$template MyTemplateName,"\7Text %property% some more text\n",<options></code></blockquote> -<p>The "$template" is the template directive. It tells rsyslog +<p>The "template" (legacy: $template) is the template directive. It tells rsyslog that this line contains a template. "MyTemplateName" is the template name. All -other config lines refer to this name. The text within quotes is the +other config lines refer to this name. The text within "string" is the actual template text. The backslash is an escape character, much as it is in C. It does all these "cool" things. For example, \7 rings the bell (this is an ASCII value), \n is a new line. C programmers and perl @@ -69,24 +228,30 @@ on this is below, on some lines of the property replacer.<br> <br> The <options> part is optional. It carries options influencing the template as whole. See details below. Be sure NOT to -mistake template options with property options - the later ones are +mistake template options with property options - the latter ones are processed by the property replacer and apply to a SINGLE property, only (and not the whole template).<br> <br> Template options are case-insensitive. Currently defined are: </p> -<p><b>sql</b> - format the string suitable for a SQL +<p><b>option.sql</b> - format the string suitable for a SQL statement in MySQL format. This will replace single quotes ("'") and the backslash character by their backslash-escaped counterpart ("\'" and "\\") inside each field. Please note that in MySQL configuration, the <code class="literal">NO_BACKSLASH_ESCAPES</code> mode must be turned off for this format to work (this is the default).</p> -<p><b>stdsql</b> - format the string suitable for a +<p><b>option.stdsql</b> - format the string suitable for a SQL statement that is to be sent to a standards-compliant sql server. This will replace single quotes ("'") by two single quotes ("''") inside each field. You must use stdsql together with MySQL if in MySQL configuration the <code class="literal">NO_BACKSLASH_ESCAPES</code> is turned on.</p> +<p><b>option.json</b> - format the string suitable for a +json statement. +This will replace single quotes ("'") by two single quotes ("''") +inside each field.</p> +<p>At no time, multiple template option should be used. This can cause +unpredictable behaviour and is against all logic.</p> <p>Either the <b>sql</b> or <b>stdsql</b> option <b>must</b> be specified when a template is used for writing to a database, otherwise injection might occur. Please note @@ -120,56 +285,47 @@ $template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> <br> Properties can be accessed by the <a href="property_replacer.html">property replacer</a> (see there for details).</p> -<p><b>Please note that templates can also by +<p>Templates can be used in the form of a <b>list</b> as well. This has been +introduced with <b>6.5.0</b> The list consists of two parts which are either +a <b>constant</b> or a <b>property</b>. The constants +are taking the part of "text" that you usually enter in string-based templates. +The properties stay variable, as they are a substitute for different values of a +certain type. This type of template is extremely useful for complicated cases, +as it helps you to easily keep an overview over the template. Though, it has +the disadvantage of needing more effort to create it.</p> +<br>Config example: +<br><blockquote><code>template(name="MyTemplate" type="list" option.json="off") { + <br>constant(value="Test: ") + <br>property(name="msg" outname="mymessage") + <br>constant(value=" --!!!-- ") + <br>property(name="timereported" dateFormat="rfc3339" caseConversion="lower") + <br>constant(value="\n") + <br>} +</code></blockquote> +<p>First, the general template option will be defined. The values of the template +itself get defined in the curly brackets. As it can be seen, we have constants +and properties in exchange. Whereas constants will be filled with a value and probably +some options, properties do direct to a property and the options that could be needed +additional format definitions.</p> +<p>We suggest to use separate lines for all constants and properties. This +helps to keep a good overview over the different parts of the template. +Though, writing it in a single line will work, it is much harder to debug +if anything goes wrong with the template. </p> + + +<p><b>Please note that templates can also be used to generate selector lines with dynamic file names.</b> For example, if you would like to split syslog messages from different hosts to different files (one per host), you can define the following template:</p> +<blockquote><code>template (name="DynFile" type="string" string="/var/log/system-%HOSTNAME%.log")</code></blockquote> +<p>Legacy example:</p> <blockquote><code>$template DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote> <p>This template can then be used when defining an output selector line. It will result in something like "/var/log/system-localhost.log"</p> -<p>Template -names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT -use them if, otherwise you may receive a conflict in the future (and -quite unpredictable behaviour). There is a small set of pre-defined -templates that you can use without the need to define it:</p> -<ul> -<li><span style="font-weight: bold;">RSYSLOG_TraditionalFileFormat</span> -- the "old style" default log file format with low-precision timestamps</li> -<li><span style="font-weight: bold;">RSYSLOG_FileFormat</span> -- a modern-style logfile format similar to TraditionalFileFormat, buth -with high-precision timestamps and timezone information</li> -<li><span style="font-weight: bold;">RSYSLOG_TraditionalForwardFormat</span> -- the traditional forwarding format with low-precision timestamps. Most -useful if you send messages to other syslogd's or rsyslogd -below -version 3.12.5.</li> -<li><span style="font-weight: bold;">RSYSLOG_SysklogdFileFormat</span> -- sysklogd compatible log file format. If used with options: $SpaceLFOnReceive on; -$EscapeControlCharactersOnReceive off; $DropTrailingLFOnReception off, -the log format will conform to sysklogd log format.</li> -<li><span style="font-weight: bold;">RSYSLOG_ForwardFormat</span> -- a new high-precision forwarding format very similar to the -traditional one, but with high-precision timestamps and timezone -information. Recommended to be used when sending messages to rsyslog -3.12.5 or above.</li> -<li><span style="font-weight: bold;">RSYSLOG_SyslogProtocol23Format</span> -- the format specified in IETF's internet-draft -ietf-syslog-protocol-23, which is assumed to be come the new syslog -standard RFC. This format includes several improvements. The rsyslog -message parser understands this format, so you can use it together with -all relatively recent versions of rsyslog. Other syslogd's may get -hopelessly confused if receiving that format, so check before you use -it. Note that the format is unlikely to change when the final RFC comes -out, but this may happen.</li> -<li><span style="font-weight: bold;">RSYSLOG_DebugFormat</span> -- a special format used for troubleshooting property problems. This format -is meant to be written to a log file. Do <b>not</b> use for production or remote -forwarding.</li> -</ul> -<h3>String-based Template Samples</h3> +<h3>Legacy String-based Template Samples</h3> <p>This section provides some sample of what the default formats would look as a text-based template. Hopefully, their description is self-explanatory. Note that each $Template statement is on a <b>single</b> line, but probably broken @@ -192,7 +348,7 @@ $template StdSQLFormat,"insert into SystemEvents (Message, Facility, FromHost, P [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 2 or higher.</font></p> </body> diff --git a/doc/v4compatibility.html b/doc/v4compatibility.html index 72b0f5a9..2a51adea 100644 --- a/doc/v4compatibility.html +++ b/doc/v4compatibility.html @@ -60,7 +60,7 @@ restarting rsyslogd by HUPing it. and most other deamons require that a restart command is typed in if a restart is required. <p>Rsyslog will follow this paradigm in the next versions, resulting in many benefits. In v4, we provide some support for the old-style semantics. We introduced a setting $HUPisRestart -which may be set to "on" (tradional, heavy operationg) +which may be set to "on" (tradional, heavy operation) or "off" (new, lightweight "file close only" operation). The initial versions had the default set to traditional behavior, but starting with 4.5.1 we are now using the new behavior as the default. diff --git a/doc/v6compatibility.html b/doc/v6compatibility.html index c1799974..edb6dedf 100644 --- a/doc/v6compatibility.html +++ b/doc/v6compatibility.html @@ -112,6 +112,15 @@ to spot why things went wrong (and if at all). <p>Due to their positive effect on performance and comparatively low overhead, default batch sizes have been increased. Starting with 6.3.4, the action queues have a default batch size of 128 messages. +<h2>Default action queue enqueue timeout</h2> +<p>This timeout previously was 2seconds, and has been reduced to 50ms (starting with 6.5.0). This change +was made as a long timeout will caused delays in the associated main queue, something +that was quite unexpected to users. Now, this can still happen, but the effect is much +less harsh (but still considerable on a busy system). Also, 50ms should be fairly enough +for most output sources, except when they are really broken (like network disconnect). If +they are really broken, even a 2second timeout does not help, so we hopefully get the best +of both worlds with the new timeout. A specific timeout can of course still be configured, +it is just the timeout that changed. <h2>outchannels</h2> <p>Outchannels are a to-be-removed feature of rsyslog, at least as far as the config syntax is concerned. Nevertheless, v6 still supports it, but a new syntax is required @@ -127,6 +136,16 @@ Note that this syntax is available starting with rsyslog v4. It is important to mind that future versions of rsyslog will require different syntax and/or drop outchannel support completely. So if at all possible, avoid using this feature. If you must use it, be prepared for future changes and watch announcements very carefully. +<h2>ompipe default template</h2> +<p>Starting with 6.5.0, ompipe does no longer use the omfile default template. +Instead, the default template must be set via the module load statement. +An example is +<blockquote><code> +module(load="builtin:ompipe" template="myDefaultTemplate") +</code> </blockquote> +<p>For obvious reasons, the default template must be defined somewhere in +the config file, otherwise errors will happen during the config load +phase. <h2>omusrmsg</h2> <p>The omusrmsg module is used to send messages to users. In legacy-legacy config format (that is the very old sysklogd style), it was suffucient to use diff --git a/doc/v7compatibility.html b/doc/v7compatibility.html new file mode 100644 index 00000000..be89f666 --- /dev/null +++ b/doc/v7compatibility.html @@ -0,0 +1,33 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Compatibility notes for rsyslog v7</title> +</head> +<body> +<h1>Compatibility Notes for rsyslog v7</h1> +This document describes things to keep in mind when moving from v6 to v7. It +does not list enhancements nor does it talk about compatibility concerns introduced +by earlier versions (for this, see their respective compatibility documents). Its focus +is primarily on what you need to know if you used v6 and want to use v7 without hassle. +<p>Version 7 builds on the new config language introduced in v6 and extends it. +Other than v6, it not just only extends the config language, but provides +considerable changes to core elements as well. The result is much more power and +ease of use as well (this time that is not contradictionary). +</p> +<h2>BSD-Style blocks</h2> +BSD style blocks are no longer supported (for good reason). See the +<a href="http://www.rsyslog.com/g/BSD">rsyslog BSD blocks info</a> +page for more information and how to upgrade your config. +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> + +<h2>CEE-Properties</h2> +In rsyslog v6, CEE properties could not be used across disk-based queues. If this was +done, there content was reset. This was a missing feature in v6. In v7, this feature +has been implemented. Consequently, situations where the previous behaviour were +desired need now to be solved differently. We do not think that this will cause any +problems to anyone, especially as in v6 this was announced as a missing feature. + +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2011-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 2 or higher.</font></p> +</body></html> |