diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/Makefile.am | 8 | ||||
| -rw-r--r-- | doc/imfile.html | 19 | ||||
| -rw-r--r-- | doc/imkmsg.html | 50 | ||||
| -rw-r--r-- | doc/imptcp.html | 30 | ||||
| -rw-r--r-- | doc/imrelp.html | 2 | ||||
| -rw-r--r-- | doc/imtcp.html | 95 | ||||
| -rw-r--r-- | doc/imudp.html | 20 | ||||
| -rw-r--r-- | doc/imuxsock.html | 82 | ||||
| -rw-r--r-- | doc/manual.html | 5 | ||||
| -rw-r--r-- | doc/multi_ruleset.html | 132 | ||||
| -rw-r--r-- | doc/multi_ruleset_legacy_format.html | 192 | ||||
| -rw-r--r-- | doc/property_replacer.html | 10 | ||||
| -rw-r--r-- | doc/rainerscript.html | 8 | ||||
| -rw-r--r-- | doc/rsyslog_conf.html | 15 | ||||
| -rw-r--r-- | doc/rsyslog_conf_actions.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_basic_structure.html | 108 | ||||
| -rw-r--r-- | doc/rsyslog_conf_examples.html | 209 | ||||
| -rw-r--r-- | doc/rsyslog_conf_file_syntax_differences.html | 32 | ||||
| -rw-r--r-- | doc/rsyslog_conf_filter.html | 189 | ||||
| -rw-r--r-- | doc/rsyslog_conf_global.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_lines.html | 23 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_sysklogd_compatibility.html | 31 | ||||
| -rw-r--r-- | doc/rsyslog_conf_templates.html | 497 | ||||
| -rw-r--r-- | doc/v4compatibility.html | 2 | ||||
| -rw-r--r-- | doc/v7compatibility.html | 91 |
26 files changed, 1181 insertions, 675 deletions
diff --git a/doc/Makefile.am b/doc/Makefile.am index 1ae1c68d..e8cdba55 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -121,7 +121,6 @@ html_files = \ syslog_parsing.html \ troubleshoot.html \ rsyslog_conf_actions.html \ - rsyslog_conf_examples.html \ rsyslog_conf_filter.html \ rsyslog_conf_global.html \ rsyslog_conf_modules.html \ @@ -130,6 +129,7 @@ html_files = \ rsyslog_conf_nomatch.html \ queues_analogy.html \ multi_ruleset.html \ + multi_ruleset_legacy_format.html \ dev_oplugins.html \ free_support.html \ imudp.html \ @@ -140,11 +140,13 @@ html_files = \ rsconf1_abortonuncleanconfig.html \ rsconf1_maxopenfiles.html \ rsconf1_omfileforcechown.html \ - rsyslog_conf_file_syntax_differences.html \ - rsyslog_conf_lines.html \ rsyslog_queue_pointers.jpeg \ rsyslog_queue_pointers2.jpeg \ v6compatibility.html \ + v7compatibility.html \ + rsyslog_conf_basic_structure.html \ + rsyslog_conf_sysklogd_compatibility.html \ + imkmsg.html \ src/classes.dia grfx_files = \ diff --git a/doc/imfile.html b/doc/imfile.html index 1594cdce..0997e382 100644 --- a/doc/imfile.html +++ b/doc/imfile.html @@ -91,7 +91,6 @@ textual form (e.g. "info", "warning", ...) or as numbers (e.g. 4 for "info"). Textual form is suggested. <span style="font-weight: bold;">Default</span> is "notice".</li> <li><b>PersistStateInterval</b> [lines]</b><br> -Available in 4.7.3+, 5.6.2+<br> Specifies how often the state file shall be written when processing the input file. The default value is 0, which means a new state file is only written when the monitored files is being closed (end of rsyslogd execution). Any other @@ -101,9 +100,11 @@ to fatal errors (like power fail). Note that this setting affects imfile performance, especially when set to a low value. Frequently writing the state file is very time consuming. <li><b>ReadMode</b> [mode]</b><br> -Available in 5.7.5+ -<li><b>MaxLinesAtOnce</b> [number]</b><br> -Available in 5.9.0+ +This mode should defined when having multiline messages. The value can range from 0-2 and determines the multiline detection method. +<br>0 (default) - line based (Each line is a new message) +<br>1 - indented (New log messages start at the beginning of a line. If a line starts with a space it is part of the log message before it) +<br>2 - paragraph (There is a blank line between log messages) +<li><b>MaxLinesAtOnce</b> [number]</b> <br> This is useful if multiple files need to be monitored. If set to 0, each file will be fully processed and then processing switches to the next file @@ -112,16 +113,14 @@ will be fully processed and then processing switches to the next file switched. This provides a kind of mutiplexing the load of multiple files and probably leads to a more natural distribution of events when multiple busy files are monitored. The default is 1024. -<li><b>MaxSubmitAtOnce</b> [number]</b><br> -Available in 5.9.0+ +<li><b>MaxSubmitAtOnce</b> [number]</b> <br> This is an expert option. It can be used to set the maximum input batch size that imfile can generate. The default is 1024, which is suitable for a wide range of applications. Be sure to understand rsyslog message batch processing before you modify this option. If you do not know what this doc here talks about, this is a good indication that you should NOT modify the default. -<li><b>Ruleset</b> <ruleset><br> -Available in 5.7.5+, 6.1.5+ +<li><b>Ruleset</b> <ruleset> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> </ul> <b>Caveats/Known Bugs:</b> @@ -181,12 +180,16 @@ directive, no file monitoring will take place.</li> seconds</span><br> equivalent to: PollingInterva</li> <li><b>$InputFilePersistStateInterval</b> [lines]</b><br> +Available in 4.7.3+, 5.6.2+<br> equivalent to: PersistStateInterval <li><b>$InputFileReadMode</b> [mode]</b><br> +Available in 5.7.5+<br> equivalent to: ReadMode <li><b>$InputFileMaxLinesAtOnce</b> [number]</b><br> +Available in 5.9.0+<br> equivalent to: MaxLinesAtOnce <li>$InputFileBindRuleset <ruleset><br> +Available in 5.7.5+, 6.1.5+<br> equivalent to: Ruleset </li> </ul> <b>Caveats/Known Bugs:</b> diff --git a/doc/imkmsg.html b/doc/imkmsg.html new file mode 100644 index 00000000..23b96147 --- /dev/null +++ b/doc/imkmsg.html @@ -0,0 +1,50 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"><title>/dev/kmsg Log Input Module (imkmsg)</title> + +</head> +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>/dev/kmsg Log Input Module</h1> +<p><b>Module Name: imkmsg</b></p> +<p><b>Authors: </b>Rainer Gerhards +<rgerhards@adiscon.com><br /> +Milan Bartos +<mbartos@redhat.com></p> +<p><b>Description</b>:</p> +<p>Reads messages from the /dev/kmsg structured kernel log and submits them to the +syslog engine.</p> +<p> +The printk log buffer constains log records. These records are exported by /dev/kmsg +device as structured data in the following format:<br /> + "level,sequnum,timestamp;<message text>\n"<br /> +There could be continuation lines starting with space that contains key/value pairs.<br /> +<br /> +Log messages are parsed as necessary into rsyslog msg_t structure. Continuation lines are parsed +as json key/value pairs and added into rsyslog's message json representation. +</p> +<p><b>Configuration Directives</b>:</p> +<p>This module has no configuration directives.</p> +<b>Caveats/Known Bugs:</b> +<p>This module can't be used together with imklog module. When using one of them, make sure the other +one is not enabled.</p> +<p>This is Linux specific module and requires /dev/kmsg device with structured kernel logs.</p> +<p><b>Sample:</b></p> +<p>The following sample pulls messages from the /dev/kmsg log device. All +parameters are left by default, which is usually a good idea. Please +note that loading the plugin is sufficient to activate it. No directive +is needed to start pulling messages.<br> +</p> +<textarea rows="15" cols="60">$ModLoad imkmsg +</textarea> +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2008-2009 by <a href="http://www.gerhards.net/rainer">Rainer +Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/imptcp.html b/doc/imptcp.html index d301b76f..107dd306 100644 --- a/doc/imptcp.html +++ b/doc/imptcp.html @@ -24,7 +24,22 @@ specifying $InputPTCPServerRun multiple times. <p>This plugin has config directives similar named as imtcp, but they all have <b>P</b>TCP in their name instead of just TCP. Note that only a subset of the parameters are supported. <ul> -<li><b>AddTLFrameDelimiter</b> <Delimiter><br> + +<p><b>Module Parameters</b>:</p> +<p>These paramters can be used with the "module()" statement. They apply +globaly to all inputs defined by the module. +<ul> +<li>Threads <number><br> +Number of helper worker threads to process incoming messages. These +threads are utilized to pull data off the network. On a busy system, additional +helper threads (but not more than there are CPUs/Cores) can help improving +performance. The default value is two. +</ul> +<p><b>Input Parameters</b>:</p> +<p>These parameters can be used with the "input()" statement. They apply to the +input they are specified with. +<ul> +<li><b>AddtlFrameDelimiter</b> <Delimiter><br> This directive permits to specify an additional frame delimiter for plain tcp syslog. The industry-standard specifies using the LF character as frame delimiter. Some vendors, notable Juniper in their NetScreen products, use an invalid frame delimiter, in Juniper's @@ -78,11 +93,6 @@ name is not strictly necessary, but can be useful to apply filtering based on wh the message was received from. <li><b>Ruleset</b> <name><br> Binds specified ruleset to next server defined. -<!--<li>$InputPTCPHelperThreads <number><br> -Number of helper worker threads to process incoming messages. These -threads are utilized to pull data off the network. On a busy system, additional -helper threads (but not more than there are CPUs/Cores) can help improving -performance. The default value is two.--> <li><b>Address</b> <name><br> On multi-homed machines, specifies to which local address the listerner should be bound. </ul> @@ -93,13 +103,11 @@ On multi-homed machines, specifies to which local address the listerner should b <p><b>Sample:</b></p> <p>This sets up a TCP server on port 514:<br> </p> -<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imptcp/.libs/imptcp") # needs to be done just once +<textarea rows="4" cols="60">module(load="/folder/to/rsyslog/plugins/imptcp/.libs/imptcp") # needs to be done just once input(type="imptcp" port="514") </textarea> <p><b>Legacy Configuration Directives</b>:</p> -<p>This plugin has config directives similar named as imtcp, but they all have <b>P</b>TCP in -their name instead of just TCP. Note that only a subset of the parameters are supported. <ul> <li>$InputPTCPServerAddtlFrameDelimiter <Delimiter><br> Equivalent to: AddTLFrameDelimiter</li> @@ -137,7 +145,7 @@ Equivalent to: Address </li> <p><b>Sample:</b></p> <p>This sets up a TCP server on port 514:<br> </p> -<textarea rows="15" cols="60">$ModLoad imptcp # +<textarea rows="3" cols="60">$ModLoad imptcp # needs to be done just once $InputPTCPServerRun 514 </textarea> @@ -146,7 +154,7 @@ $InputPTCPServerRun 514 <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer +Copyright © 2010-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/imrelp.html b/doc/imrelp.html index 80ddfd53..856aff82 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -30,7 +30,7 @@ Clients send messages to the RELP server via omrelp.</p> <p><b>Configuration Directives</b>:</p> <ul> -<li><b>Ruleset</b> <name> (available in 6.3.6+)</br> +<li><b>Ruleset</b> <name></br> Binds the specified ruleset to all RELP listeners. <li><b>Port</b> <port><br> Starts a RELP server on selected port</li> diff --git a/doc/imtcp.html b/doc/imtcp.html index 649b08f8..feb0bdd7 100644 --- a/doc/imtcp.html +++ b/doc/imtcp.html @@ -21,9 +21,11 @@ can also be provided by using <a href="rsyslog_stunnel.html">stunnel</a> $InputTCPServerRun multiple times. This is available since version 4.3.1, earlier versions do NOT support it. </p> + <p><b>Configuration Directives</b>:</p> +<p><b>Global Directives</b>:</p> <ul> -<li><b>$InputTCPServerAddtlFrameDelimiter <Delimiter></b><br> +<li><b>AddtlFrameDelimiter</b> <Delimiter><br> This directive permits to specify an additional frame delimiter for plain tcp syslog. The industry-standard specifies using the LF character as frame delimiter. Some vendors, notable Juniper in their NetScreen products, use an invalid frame delimiter, in Juniper's @@ -43,7 +45,7 @@ very limited interest in fixing this issue. This directive <b>can not</b> fix th That would require much more code changes, which I was unable to do so far. Full details can be found at the <a href="http://www.rsyslog.com/Article321.phtml">Cisco tcp syslog anomaly</a> page. -<li><b>$InputTCPServerDisableLFDelimiter</b> <on/<b>off</b>> (available since 5.5.3)<br> +<li><b>DisableLFDelimiter</b> <on/<b>off</b>><br> Industry-strandard plain text tcp syslog uses the LF to delimit syslog frames. However, some users brought up the case that it may be useful to define a different delimiter and totally disable LF as a delimiter (the use case named were multi-line messages). This mode @@ -51,16 +53,14 @@ is non-standard and will probably come with a lot of problems. However, as there for it and it is relatively easy to support, we do so. Be sure to turn this setting to "on" only if you exactly know what you are doing. You may run into all sorts of troubles, so be prepared to wrangle with that! -<li><b>$InputTCPServerNotifyOnConnectionClose</b> [on/<b>off</b>] (available since 4.5.5)<br> +<li><b>NotifyOnConnectionClose</b> [on/<b>off</b>]<br> instructs imtcp to emit a message if the remote peer closes a connection.<br> <b>Important:</b> This directive is global to all listeners and must be given right after loading imtcp, otherwise it may have no effect.</li> -<li><b>$InputTCPServerKeepAlive</b> <on/<b>off</b>><br> +<li><b>KeepAlive</b> <on/<b>off</b>><br> enable of disable keep-alive packets at the tcp socket layer. The default is to disable them.</li> -<li><b>$InputTCPServerRun</b> <port><br> -Starts a TCP server on selected port</li> -<li><b>$InputTCPFlowControl</b> <<b>on</b>/off><br> +<li><b>FlowControl</b> <<b>on</b>/off><br> This setting specifies whether some message flow control shall be exercised on the related TCP input. If set to on, messages are handled as "light delayable", which means the sender is throttled a bit when the queue becomes near-full. This is done in order @@ -69,24 +69,32 @@ may have some undesired effect in some configurations. Still, we consider this a a useful setting and thus it is the default. To turn the handling off, simply configure that explicitely. </li> -<li><b>$InputTCPMaxListeners</b> <number><br> +<li><b>MaxListeners</b> <number><br> Sets the maximum number of listeners (server ports) supported. Default is 20. This must be set before the first $InputTCPServerRun directive.</li> -<li><b>$InputTCPMaxSessions</b> <number><br> Sets the maximum number of sessions supported. Default is 200. This must be set before the first $InputTCPServerRun directive</li> -<li><b>$InputTCPServerStreamDriverMode</b> <number><br> +<li><b>MaxSessions</b> <number><br> Sets the maximum number of sessions supported. Default is 200. This must be set before the first $InputTCPServerRun directive</li> +<li><b>StreamDriver.Mode</b> <number><br> Sets the driver mode for the currently selected <a href="netstream.html">network stream driver</a>. <number> is driver specifc.</li> -<li><b>$InputTCPServerInputName</b> <name><br> -Sets a name for the inputname property. If no name is set "imtcp" is used by default. Setting a -name is not strictly necessary, but can be useful to apply filtering based on which input -the message was received from. -<li><b>$InputTCPServerStreamDriverAuthMode</b> <mode-string><br> +<li><b>StreamDriver.AuthMode</b> <mode-string><br> Sets the authentication mode for the currently selected <a href="netstream.html">network stream driver</a>. <mode-string> is driver specifc.</li> -<li><b>$InputTCPServerStreamDriverPermittedPeer</b> <id-string><br> +<li><b>PermittedPeer</b> <id-string><br> Sets permitted peer IDs. Only these peers are able to connect to the listener. <id-string> semantics depend on the currently selected -AuthMode and <a href="netstream.html">network stream driver</a>. PermittedPeers may not be set in anonymous modes.</li> -<li><b>$InputTCPServerBindRuleset</b> <ruleset><br> +AuthMode and <a href="netstream.html">network stream driver</a>. PermittedPeer may not be set in anonymous modes. +<br>PermittedPeer may be set either to a single peer or an array of peers either of type IP or name, depending on the tls certificate. +<br>Single peer: PermittedPeer="127.0.0.1" +<br>Array of peers: PermittedPeer=["test1.example.net","10.1.2.3","test2.example.net","..."]</li> +</ul> +<p><b>Action Directives</b>:</p> +<ul> +<li><b>Port</b> <port><br> +Starts a TCP server on selected port</li> +<li><b>Name</b> <name><br> +Sets a name for the inputname property. If no name is set "imtcp" is used by default. Setting a +name is not strictly necessary, but can be useful to apply filtering based on which input +the message was received from. +<li><b>Ruleset</b> <ruleset><br> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> -<li><b>$InputTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> +<li><b>SupportOctetCountedFraming</b> <<b>on</b>|off><br> If set to "on", the legacy octed-counted framing (similar to RFC5425 framing) is activated. This is the default and should be left unchanged until you know very well what you do. It may be useful to turn it off, if you know this framing @@ -102,6 +110,55 @@ is not used and some senders emit multi-line messages into the message stream. <p><b>Example:</b></p> <p>This sets up a TCP server on port 514 and permits it to accept up to 500 connections:<br> </p> +<textarea rows="15" cols="60">module(load="imtcp" MaxSessions="500") +input(type="imtcp" port="514") +</textarea> +<p>Note that the global parameters (here: max sessions) need to be set when the module is loaded. Otherwise, the parameters will not apply. +</p> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> +<li><b>$InputTCPServerAddtlFrameDelimiter <Delimiter></b><br> +equivalent to: AddtlFrameDelimiter +<li><b>$InputTCPServerDisableLFDelimiter</b> <on/<b>off</b>> (available since 5.5.3)<br> +equivalent to: DisableLFDelimiter +<li><b>$InputTCPServerNotifyOnConnectionClose</b> [on/<b>off</b>] (available since 4.5.5)<br> +equivalent to: NotifyOnConnectionClose<br> +</li> +<li><b>$InputTCPServerKeepAlive</b> <on/<b>off</b>><br> +equivalent to: KeepAlive</li> +<li><b>$InputTCPServerRun</b> <port><br> +equivalent to: Port</li> +<li><b>$InputTCPFlowControl</b> <<b>on</b>/off><br> +equivalent to: FlowControl +</li> +<li><b>$InputTCPMaxListeners</b> <number><br> +equivalent to: MaxListeners</li> +<li><b>$InputTCPMaxSessions</b> <number><br> +equivalent to: MaxSessions</li> +<li><b>$InputTCPServerStreamDriverMode</b> <number><br> +equivalent to: StreamDriver.Mode</li> +<li><b>$InputTCPServerInputName</b> <name><br> +equivalent to: Name +<li><b>$InputTCPServerStreamDriverAuthMode</b> <mode-string><br> +equivalent to: StreamDriver.AuthMode</li> +<li><b>$InputTCPServerStreamDriverPermittedPeer</b> <id-string><br> +equivalent to: PermittedPeer.</li> +<li><b>$InputTCPServerBindRuleset</b> <ruleset><br> +equivalent to: Ruleset</a>.</li> +<li><b>$InputTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> +equivalent to: SupportOctetCountedFraming +</li> +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>module always binds to all interfaces</li> +<li>can not be loaded together with <a href="imgssapi.html">imgssapi</a> +(which includes the functionality of imtcp)</li> +</ul> +<p><b>Example:</b></p> +<p>This sets up a TCP server on port 514 and permits it to accept up to 500 connections:<br> +</p> <textarea rows="15" cols="60">$ModLoad imtcp # needs to be done just once $InputTCPMaxSessions 500 $InputTCPServerRun 514 diff --git a/doc/imudp.html b/doc/imudp.html index 3512d474..105e0b2a 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -33,27 +33,33 @@ the value, the less precise the timestamp. <li><b>SchedulingPolicy</b> <rr/fifo/other><br> Can be used the set the scheduler priority, if the necessary functionality is provided by the platform. Most useful to select "fifo" for real-time -processing under Linux (and thus reduce chance of packet loss). Available since 4.7.4+, 5.7.3+, 6.1.3+. +processing under Linux (and thus reduce chance of packet loss). <li><b>SchedulingPriority</b> <number><br> -Scheduling priority to use. Available since 4.7.4+, 5.7.3+, 6.1.3+. +Scheduling priority to use. </ul> <p><b>Action Directives</b>:</p> <ul> <li><b>Address</b> <IP><br> local IP address (or name) the UDP listens should bind to</li> <li><b>Port</b> <port><br> -default 514, start UDP server on this port</li> +default 514, start UDP server on this port. Either a single port can be specified or an array of ports. If multiple ports are specified, a listener will be automatically started for each port. Thus, no additional inputs need to be configured. +<br>Single port: Port="514" +<br>Array of ports: Port=["514","515","10514","..."]</li> <li><b>Ruleset</b> <ruleset><br> Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> </ul> <b>Caveats/Known Bugs:</b> <ul> -<li>currently none known</li> +<li>Scheduling parameters are set <b>after</b> privileges have been dropped. +In most cases, this means that setting them will not be possible after +privilege drop. This may be worked around by using a sufficiently-privileged +user account. +</li> </ul> <p><b>Sample:</b></p> <p>This sets up an UPD server on port 514:<br> </p> -<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imudp/.libs/imudp") # needs to be done just once +<textarea rows="15" cols="60">module(load="imudp") # needs to be done just once input(type="imudp" port="514") </textarea> @@ -70,9 +76,9 @@ equivalent to: Port </li> equivalent to: TimeRequery <li>$InputUDPServerBindRuleset <ruleset><br> equivalent to: Ruleset </li> -<li>$IMUDPSchedulingPolicy <rr/fifo/other><br> +<li>$IMUDPSchedulingPolicy <rr/fifo/other> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> equivalent to: SchedulingPolicy -<li>$IMUDPSchedulingPriority <number><br> +<li>$IMUDPSchedulingPriority <number> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> equivalent to: SchedulingPriority </ul> <b>Caveats/Known Bugs:</b> diff --git a/doc/imuxsock.html b/doc/imuxsock.html index bd207a37..d505604a 100644 --- a/doc/imuxsock.html +++ b/doc/imuxsock.html @@ -77,7 +77,7 @@ to the system log socket. <li><b>SysSock.UsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall be obtained from the log socket itself. If so, the TAG part of the message is rewritten. It is recommended to turn this option on, but the default is "off" to keep compatible -with earlier versions of rsyslog. This option was introduced in 5.7.0. +with earlier versions of rsyslog. </li> <li><b>SysSock.RateLimit.Interval</b> [number] - specifies the rate-limiting interval in seconds. Default value is 5 seconds. Set it to 0 to turn rate limiting off. @@ -92,6 +92,10 @@ messages that shall be rate-limited. </li> <li><b>SysSock.Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the system log socket.</li> +<li><b>SysSock.ParseTrusted</b> <on/<b>off</b>> if Annotation is turned on, create +JSON/lumberjack properties out of the trusted properties (which can be accessed +via RainerScript JSON Variables, e.g. "$!pid") instead of adding them to the message. +properties for the system log socket.</li> </ul> <p><b>Input Instance Parameters</b></p> @@ -102,7 +106,7 @@ properties for the system log socket.</li> to the next socket.</li> <li><b>RateLimit.Interval</b> [number] - specifies the rate-limiting interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number -of seconds (5 recommended) to activate rate-limiting. The default of 0 has been choosen in 5.9.6+, +of seconds (5 recommended) to activate rate-limiting. The default of 0 has been choosen as people experienced problems with this feature activated by default. Now it needs an explicit opt-in by setting this parameter. </li> @@ -112,7 +116,7 @@ burst in number of messages. Default is 200. <li><b>RateLimit.Severity</b> [numerical severity] - specifies the severity of messages that shall be rate-limited. </li> -<!--<li><b>LocalIPIF</b> [interface name] - (available since 5.9.6) - if provided, the IP of the specified +<!--<li><b>LocalIPIF</b> [interface name] - if provided, the IP of the specified interface (e.g. "eth0") shall be used as fromhost-ip for imuxsock-originating messages. If this directive is not given OR the interface cannot be found (or has no IP address), the default of "127.0.0.1" is used. @@ -120,7 +124,7 @@ the default of "127.0.0.1" is used. <li><b>UsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall be obtained from the log socket itself. If so, the TAG part of the message is rewritten. It is recommended to turn this option on, but the default is "off" to keep compatible -with earlier versions of rsyslog. This option was introduced in 5.7.0.</li> +with earlier versions of rsyslog. </li> <li><b>UseSysTimeStamp</b> [<b>on</b>/off] instructs imuxsock to obtain message time from the system (via control messages) insted of using time recorded inside the message. This may be most useful in combination with systemd. Note: @@ -139,7 +143,7 @@ being reset to "off" after the Socket directive, so if you would have for two additional listen sockets, you need to specify it in front of each one. This option is primarily considered useful for defining additional sockets that reside on non-permanent file systems. As rsyslogd probably starts up before the daemons that create these sockets, it is a vehicle to enable rsyslogd to listen to those -sockets even though their directories do not yet exist. [available since 4.7.0 and 5.3.0]</li> +sockets even though their directories do not yet exist.</li> <li><b>Socket</b> <name-of-socket> adds additional unix socket, default none -- former -a option</li> <li><b>HostName</b> <hostname> permits to override the hostname that shall be used inside messages taken from the <b>next</b> Socket socket. Note that @@ -148,6 +152,8 @@ will only affect the next one and then automatically be reset. This functionalit that the local hostname can be overridden in cases where that is desired.</li> <li><b>Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the non-system log socket in question.</li> +<li><b>ParseTrusted</b> <on/<b>off</b>> equivalent to the SysSock.ParseTrusted module +parameter, but applies to the input that is being defined. </ul> <b>Caveats/Known Bugs:</b><br> @@ -160,12 +166,20 @@ change the array size in imuxsock.c. <p>The following sample is the minimum setup required to accept syslog messages from applications running on the local system.<br> </p> -<textarea rows="2" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="2" cols="70">module(load="imuxsock" # needs to be done just once SysSock.FlowControl="on") # enable flow control (use if needed) </textarea> + +<p>The following sample is similiar to the first one, but enables trusted +properties, which are put into JSON/lumberjack variables. +<br> +</p> +<textarea rows="2" cols="70">module(load="imuxsock" SysSock.Annotate="on" SysSock.ParseTrusted="on") +</textarea> + <p>The following sample is a configuration where rsyslogd pulls logs from two jails, and assigns different hostnames to each of the jails: </p> -<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once +<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once input(type="imuxsock" HostName="jail1.example.net" Socket="/jail/1/dev/log") input(type="imuxsock" HostName="jail2.example.net" Socket="/jail/2/dev/log") @@ -176,18 +190,18 @@ system. As rsyslogd starts up before the sshd, it needs to create the socket directories, because it otherwise can not open the socket and thus not listen to openssh messages. Note that it is vital not to place any other socket between the CreatePath and the Socket.</p> -<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once +<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once input(type="imuxsock" Socket="/var/run/sshd/dev/log" CreatePath="on") </textarea> <p>The following sample is used to turn off input rate limiting on the system log socket. -<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once SysSock.RateLimit.Interval="0") # turn off rate limiting </textarea> <p>The following sample is used activate message annotation and thus trusted properties on the system log socket. -<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once SysSock.Annotate="on") </textarea> @@ -195,39 +209,43 @@ SysSock.Annotate="on") <p><b>Legacy Configuration Directives</b>:</p> <ul> <li><b>$InputUnixListenSocketIgnoreMsgTimestamp</b> [<b>on</b>/off] -<br>Please see: IgnoreTimestamp.</li> -<li><b>$InputUnixListenSocketFlowControl</b> [on/<b>off</b>] - Please see: FlowControl .</li> -<li><b>$IMUXSockRateLimitInterval</b> [number] - Please see: RateLimit.Interval +<br>equivalent to: IgnoreTimestamp.</li> +<li><b>$InputUnixListenSocketFlowControl</b> [on/<b>off</b>] - equivalent to: FlowControl .</li> +<li><b>$IMUXSockRateLimitInterval</b> [number] - equivalent to: RateLimit.Interval </li> -<li><b>$IMUXSockRateLimitBurst</b> [number] - Please see: RateLimit.Burst +<li><b>$IMUXSockRateLimitBurst</b> [number] - equivalent to: RateLimit.Burst </li> -<li><b>$IMUXSockRateLimitSeverity</b> [numerical severity] - Please see: RateLimit.Severity +<li><b>$IMUXSockRateLimitSeverity</b> [numerical severity] - equivalent to: RateLimit.Severity </li> <li><b>$IMUXSockLocalIPIF</b> [interface name] - (available since 5.9.6) - if provided, the IP of the specified interface (e.g. "eth0") shall be used as fromhost-ip for imuxsock-originating messages. If this directive is not given OR the interface cannot be found (or has no IP address), the default of "127.0.0.1" is used. </li> -<li><b>$InputUnixListenSocketUsePIDFromSystem</b> [on/<b>off</b>] - Please see: UsePIDFromSystem.</li> -<li><b>$InputUnixListenSocketUseSysTimeStamp</b> [<b>on</b>/off] Please see: UseSysTimeStamp .<br> +<li><b>$InputUnixListenSocketUsePIDFromSystem</b> [on/<b>off</b>] - equivalent to: UsePIDFromSystem. +<br>This option was introduced in 5.7.0.</li> +<li><b>$InputUnixListenSocketUseSysTimeStamp</b> [<b>on</b>/off] equivalent to: UseSysTimeStamp .<br> <li><b>$SystemLogSocketIgnoreMsgTimestamp</b> [<b>on</b>/off]<br> -Please see: SysSock.IgnoreTimestamp.</li> -<li><b>$OmitLocalLogging</b> (imuxsock) [on/<b>off</b>] Please see: SysSock.Use</li> -<li><b>$SystemLogSocketName</b> <name-of-socket> Please see: SysSock.Name</li> -<li><b>$SystemLogFlowControl</b> [on/<b>off</b>] - Please see: SysSock.FlowControl.</li> -<li><b>$SystemLogUsePIDFromSystem</b> [on/<b>off</b>] - Please see: SysSock.UsePIDFromSystem.</li> -<li><b>$SystemLogRateLimitInterval</b> [number] - Please see: SysSock.RateLimit.Interval. +equivalent to: SysSock.IgnoreTimestamp.</li> +<li><b>$OmitLocalLogging</b> (imuxsock) [on/<b>off</b>] equivalent to: SysSock.Use</li> +<li><b>$SystemLogSocketName</b> <name-of-socket> equivalent to: SysSock.Name</li> +<li><b>$SystemLogFlowControl</b> [on/<b>off</b>] - equivalent to: SysSock.FlowControl.</li> +<li><b>$SystemLogUsePIDFromSystem</b> [on/<b>off</b>] - equivalent to: SysSock.UsePIDFromSystem. +<br>This option was introduced in 5.7.0.</li> +<li><b>$SystemLogRateLimitInterval</b> [number] - equivalent to: SysSock.RateLimit.Interval. </li> -<li><b>$SystemLogRateLimitBurst</b> [number] - Please see: SysSock.RateLimit.Burst +<li><b>$SystemLogRateLimitBurst</b> [number] - equivalent to: SysSock.RateLimit.Burst </li> -<li><b>$SystemLogRateLimitSeverity</b> [numerical severity] - Please see: SysSock.RateLimit.Severity +<li><b>$SystemLogRateLimitSeverity</b> [numerical severity] - equivalent to: SysSock.RateLimit.Severity </li> -<li><b>$SystemLogUseSysTimeStamp</b> [<b>on</b>/off] Please see: SysSock.UseSysTimeStamp. -<li><b>$InputUnixListenSocketCreatePath</b> [on/<b>off</b>] - Please see: CreatePath</li> -<li><b>$AddUnixListenSocket</b> <name-of-socket> Please see: Socket </li> -<li><b>$InputUnixListenSocketHostName</b> <hostname> Please see: HostName.</li> -<li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> Please see: Annotate.</li> -<li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> Please see: SysSock.Annotate.</li> +<li><b>$SystemLogUseSysTimeStamp</b> [<b>on</b>/off] equivalent to: SysSock.UseSysTimeStamp. +<li><b>$InputUnixListenSocketCreatePath</b> [on/<b>off</b>] - equivalent to: CreatePath +<br>[available since 4.7.0 and 5.3.0]</li> +<li><b>$AddUnixListenSocket</b> <name-of-socket> equivalent to: Socket </li> +<li><b>$InputUnixListenSocketHostName</b> <hostname> equivalent to: HostName.</li> +<li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> equivalent to: Annotate.</li> +<li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> equivalent to: SysSock.Annotate.</li> +<li><b>$SystemLogSocketParseTrusted</b> <on/<b>off</b>> equivalent to: SysSock.ParseTrusted.</li> </ul> <b>Caveats/Known Bugs:</b><br> @@ -280,7 +298,7 @@ $SystemLogSocketAnnotate on <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/manual.html b/doc/manual.html index 9c7c677e..8b54bd0b 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 6.6.0 (v6-stable branch) of rsyslog.</b> +<p><b>This documentation is for version 7.2.6 (v7.2-stable branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might @@ -35,12 +35,13 @@ if you upgrade from v4, read the <a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and if you upgrade from v5, read the <a href="v6compatibility.html">rsyslog v6 compatibility notes</a>. +if you upgrade from v6, read the +<a href="v7compatibility.html">rsyslog v7 compatibility notes</a>. <p>Rsyslog will work even if you do not read the doc, but doing so will definitely improve your experience.</p> <p><b>Follow the links below for the</b></p> <ul> <li><a href="troubleshoot.html">troubleshooting rsyslog problems</a></li> -<li><a href="http://www.rsyslog.com/doc/node1.html">rsyslog.conf, new RainerScript-based format (v6+)</a></li> <li><a href="rsyslog_conf.html">configuration file format (rsyslog.conf)</a></li> <li><a href="http://www.rsyslog.com/tool-regex">a regular expression checker/generator tool for rsyslog</a></li> <li> <a href="property_replacer.html">property replacer, an important core component</a></li> diff --git a/doc/multi_ruleset.html b/doc/multi_ruleset.html index da65b4ba..37c54065 100644 --- a/doc/multi_ruleset.html +++ b/doc/multi_ruleset.html @@ -31,7 +31,7 @@ You can think of a traditional config file just as a single default rule set, wh automatically bound to each of the inputs. This is even what actually happens. When rsyslog.conf is processed, the config file parser looks for the directive -<pre>$RuleSet <name> +<pre>ruleset(name="rulesetname"); </pre> <p>Where name is any name the user likes (but must not start with "RSYSLOG_", which @@ -63,7 +63,7 @@ to seperate the messages by any other method. <p>Binding to rulesets is input-specifc. For imtcp, this is done via the -<pre>$InputTCPServerBindRuleset <name> +<pre>input(type="imptcp" port="514" ruleset="rulesetname"); </pre> directive. Note that "name" must be the name of a ruleset that is already defined @@ -116,8 +116,12 @@ filters on the message, processes it and then discards it: <pre> # ... module loading ... # process remote messages -:fromhost-ip, isequal, "192.0.2.1" /var/log/remotefile -& ~ +if $fromhost-ip == '192.168.152.137' then { + action(type="omfile" file="/var/log/remotefile02") + stop + } + + # only messages not from 192.0.21 make it past this point # The authpriv file has restricted access. @@ -131,7 +135,7 @@ cron.* /var/log/cron ... more ... </pre> -<p>Note the tilde character, which is the discard action!. Also note that we assume that +<p>Note that "stop" is the discard action!. Also note that we assume that 192.0.2.1 is the sole remote sender (to keep it simple). <p>With multiple rulesets, we can simply define a dedicated ruleset for the remote reception @@ -141,66 +145,15 @@ case and bind it to the receiver. This may be written as follows: # ... module loading ... # process remote messages # define new ruleset and add rules to it: -$RuleSet remote -*.* /var/log/remotefile +ruleset(name="remote"){ + action(type="omfile" file="/var/log/remotefile") +} # only messages not from 192.0.21 make it past this point -# bind ruleset to tcp listener -$InputTCPServerBindRuleset remote -# and activate it: -$InputTCPServerRun 10514 - -# switch back to the default ruleset: -$RuleSet RSYSLOG_DefaultRuleset -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... +# bind ruleset to tcp listener and activate it: +input(type="imptcp" port="10514" ruleset="remote") </pre> -<p>Here, we need to switch back to the default ruleset after we have defined our custom -one. This is why I recommend a different ordering, which I find more intuitive. The sample -below has it, and it leads to the same results: - -<pre> -# ... module loading ... -# at first, this is a copy of the unmodified rsyslog.conf -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -# end of the "regular" rsyslog.conf. Now come the new definitions: - -# process remote messages -# define new ruleset and add rules to it: -$RuleSet remote -*.* /var/log/remotefile - -# bind ruleset to tcp listener -$InputTCPServerBindRuleset remote -# and activate it: -$InputTCPServerRun 10514 -</pre> - -<p>Here, we do not switch back to the default ruleset, because this is not needed as it is -completely defined when we begin the "remote" ruleset. - -<p>Now look at the examples and compare them to the single-ruleset solution. You will notice -that we do <b>not</b> need a real filter in the multi-ruleset case: we can simply use -"*.*" as all messages now means all messages that are being processed by this -rule set and all of them come in via the TCP receiver! This is what makes using multiple -rulesets so much easier. - <h3>Split local and remote logging for three different ports</h3> <p>This example is almost like the first one, but it extends it a little bit. While it is very similar, I hope it is different enough to provide a useful example why you may want @@ -217,47 +170,34 @@ written to 10516's general log file. <pre> # ... module loading ... -# at first, this is a copy of the unmodified rsyslog.conf -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -# end of the "regular" rsyslog.conf. Now come the new definitions: - # process remote messages -#define rulesets first -$RuleSet remote10514 -*.* /var/log/remote10514 - -$RuleSet remote10515 -*.* /var/log/remote10515 +ruleset(name="remote10514"){ + action(type="omfile" file="/var/log/remote10514") +} -$RuleSet remote10516 -mail.* /var/log/mail10516 -& ~ -# note that the discard-action will prevent this messag from -# being written to the remote10516 file - as usual... -*.* /var/log/remote10516 +ruleset(name="remote10515"){ + action(type="omfile" file="/var/log/remote10515") +} -# and now define listners bound to the relevant ruleset -$InputTCPServerBindRuleset remote10514 -$InputTCPServerRun 10514 +ruleset(name="test1"){ + if prifilt("mail.*") then { + /var/log/mail10516 + stop + # note that the stop-command will prevent this message from + # being written to the remote10516 file - as usual... + } + /var/log/remote10516 +} -$InputTCPServerBindRuleset remote10515 -$InputTCPServerRun 10515 -$InputTCPServerBindRuleset remote10516 -$InputTCPServerRun 10516 +# and now define listners bound to the relevant ruleset +input(type="imptcp" port="10514" ruleset="remote10514") +input(type="imptcp" port="10515" ruleset="remote10515") +input(type="imptcp" port="10516" ruleset="remote10516") </pre> -<p>Note that the "mail.*" rule inside the "remote10516" ruleset does -not affect processing inside any other rule set, including the default rule set. + <h2>Performance</h2> @@ -289,10 +229,6 @@ dedicated queue for each of the inputs. <p>By default, rulesets do <b>not</b> have their own queue. It must be activated via the <a href="rsconf1_rulesetcreatemainqueue.html">$RulesetCreateMainQueue</a> directive. -<h3>Future Enhancements</h3> -<p>In the long term, multiple rule sets will probably lay the foundation for even better -optimizations. So it is not a bad idea to get aquainted with them. - <p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> diff --git a/doc/multi_ruleset_legacy_format.html b/doc/multi_ruleset_legacy_format.html new file mode 100644 index 00000000..5a9e7a4a --- /dev/null +++ b/doc/multi_ruleset_legacy_format.html @@ -0,0 +1,192 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<title>Multiple Rulesets in legacy format</title></head> +<body> +<h1>Multiple Rulesets in rsyslog</h1> +<p>Starting with version 4.5.0 and 5.1.1, <a href="http://www.rsyslog.com">rsyslog</a> supports +multiple rulesets within a single configuration. +This is especially useful for routing the recpetion of remote messages to a set of specific rules. +Note that the input module must support binding to non-standard rulesets, so the functionality +may not be available with all inputs.<p> +<b>Attention: this guide is shortened and only contains the samples in legacy format.</b> +Please follow this link to the full guide in the new config format "list": <a href="http://www.rsyslog.com/doc/multi_ruleset.html">http://www.rsyslog.com/doc/multi_ruleset.html<a> + + +<h2>Examples</h2> +<h3>Split local and remote logging</h3> +<p>Let's say you have a pretty standard system that logs its local messages to the usual +bunch of files that are specified in the default rsyslog.conf. As an example, your rsyslog.conf +might look like this: + +<pre> +# ... module loading ... +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +</pre> + +<p>Now, you want to add receive messages from a remote system and log these to +a special file, but you do not want to have these messages written to the files +specified above. The traditional approach is to add a rule in front of all others that +filters on the message, processes it and then discards it: + +<pre> +# ... module loading ... +# process remote messages +:fromhost-ip, isequal, "192.0.2.1" /var/log/remotefile +& ~ +# only messages not from 192.0.21 make it past this point + +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +</pre> + +<p>Note the tilde character, which is the discard action!. Also note that we assume that +192.0.2.1 is the sole remote sender (to keep it simple). + +<p>With multiple rulesets, we can simply define a dedicated ruleset for the remote reception +case and bind it to the receiver. This may be written as follows: + +<pre> +# ... module loading ... +# process remote messages +# define new ruleset and add rules to it: +$RuleSet remote +*.* /var/log/remotefile +# only messages not from 192.0.21 make it past this point + +# bind ruleset to tcp listener +$InputTCPServerBindRuleset remote +# and activate it: +$InputTCPServerRun 10514 + +# switch back to the default ruleset: +$RuleSet RSYSLOG_DefaultRuleset +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +</pre> + +<p>Here, we need to switch back to the default ruleset after we have defined our custom +one. This is why I recommend a different ordering, which I find more intuitive. The sample +below has it, and it leads to the same results: + +<pre> +# ... module loading ... +# at first, this is a copy of the unmodified rsyslog.conf +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +# end of the "regular" rsyslog.conf. Now come the new definitions: + +# process remote messages +# define new ruleset and add rules to it: +$RuleSet remote +*.* /var/log/remotefile + +# bind ruleset to tcp listener +$InputTCPServerBindRuleset remote +# and activate it: +$InputTCPServerRun 10514 +</pre> + +<p>Here, we do not switch back to the default ruleset, because this is not needed as it is +completely defined when we begin the "remote" ruleset. + +<p>Now look at the examples and compare them to the single-ruleset solution. You will notice +that we do <b>not</b> need a real filter in the multi-ruleset case: we can simply use +"*.*" as all messages now means all messages that are being processed by this +rule set and all of them come in via the TCP receiver! This is what makes using multiple +rulesets so much easier. + +<h3>Split local and remote logging for three different ports</h3> +<p>This example is almost like the first one, but it extends it a little bit. While it is +very similar, I hope it is different enough to provide a useful example why you may want +to have more than two rulesets. + +<p>Again, we would like to use the "regular" log files for local logging, only. But +this time we set up three syslog/tcp listeners, each one listening to a different +port (in this example 10514, 10515, and 10516). Logs received from these receivers shall go into +different files. Also, logs received from 10516 (and only from that port!) with +"mail.*" priority, shall be written into a specif file and <b>not</b> be +written to 10516's general log file. + +<p>This is the config: + +<pre> +# ... module loading ... +# at first, this is a copy of the unmodified rsyslog.conf +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +# end of the "regular" rsyslog.conf. Now come the new definitions: + +# process remote messages + +#define rulesets first +$RuleSet remote10514 +*.* /var/log/remote10514 + +$RuleSet remote10515 +*.* /var/log/remote10515 + +$RuleSet remote10516 +mail.* /var/log/mail10516 +& ~ +# note that the discard-action will prevent this messag from +# being written to the remote10516 file - as usual... +*.* /var/log/remote10516 + +# and now define listners bound to the relevant ruleset +$InputTCPServerBindRuleset remote10514 +$InputTCPServerRun 10514 + +$InputTCPServerBindRuleset remote10515 +$InputTCPServerRun 10515 + +$InputTCPServerBindRuleset remote10516 +$InputTCPServerRun 10516 +</pre> + +<p>Note that the "mail.*" rule inside the "remote10516" ruleset does +not affect processing inside any other rule set, including the default rule set. + + +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/property_replacer.html b/doc/property_replacer.html index dc09d33c..c7624b2d 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -228,7 +228,15 @@ for filtering in a generic way)</td> <td>This is the "bridge" to syslog message normalization (via <a href="mmnormalize.html">mmnormalize</a>): name is a name defined inside the normalization rule. It has the value selected by the rule -or none if no rule with this field did match. +or none if no rule with this field did match. You can also use these +properties to specify JSON fields from the CEE-enhanced syslog +message, once you parse it with <a href="mmjsonparse.html">mmjsonparse</a> +</td> +</tr> +<tr> +<td><b>$!all-json</b></td> +<td>This is the JSON part of the CEE-enhanced syslog message, which +can be parsed with <a href="mmjsonparse.html">mmjsonparse</a> </td> </tr> </tbody> diff --git a/doc/rainerscript.html b/doc/rainerscript.html index fcc2674d..84261bdd 100644 --- a/doc/rainerscript.html +++ b/doc/rainerscript.html @@ -61,6 +61,14 @@ variable, if it exists. Returns an empty string if it does not exist. <li>cstr(expr) - converts expr to a string value <li>cnum(expr) - converts expr to a number (integer) <li>re_match(expr, re) - returns 1, if expr matches re, 0 otherwise +<li>field(str, delim, matchnbr) - returns a field-based substring. str is the string +to search, delim is the numerical ascii value of the field delimiter (so that +non-printable characters can by specified) and matchnbr is the match to search +for (the first match starts at 1). This works similar as the field based +property-replacer option. +<li>prifilt(constant) - mimics a traditional PRI-based filter (like "*.*" or +"mail.info"). The traditional filter string must be given as a <b>constant string</b>. +Dynamic string evaluation is not permitted (for performance reasons). </ul> <p>The following example can be used to build a dynamic filter based on some environment variable: diff --git a/doc/rsyslog_conf.html b/doc/rsyslog_conf.html index 6aa2e460..c5f4d2e3 100644 --- a/doc/rsyslog_conf.html +++ b/doc/rsyslog_conf.html @@ -21,16 +21,15 @@ especially useful while you are migrating from syslogd to rsyslogd.</p> <p><b>Follow the links below to learn more about specific topics:</b></p> <ul> -<li><a href="rsyslog_conf_modules.html">Modules</a></li> -<li><a href="rsyslog_conf_lines.html">Lines</a></li> -<li><a href="rsyslog_conf_global.html">Configuration Directives</a></li> <li><a href="rsyslog_conf_basic_structure.html">Basic Structure</a></li> +<li><a href="rsyslog_conf_modules.html">Modules</a></li> <li><a href="rsyslog_conf_templates.html">Templates</a></li> -<li><a href="rsyslog_conf_output.html">Output Channels</a></li> <li><a href="rsyslog_conf_filter.html">Filter Conditions</a></li> -<li><a href="rsyslog_conf_actions.html">Actions</a></li> -<li><a href="rsyslog_conf_file_syntax_differences.html">Configuration File Syntax Differences</a></li> -<li><a href="rsyslog_conf_examples.html">Examples</a></li> +<li><a href="rsyslog_conf_actions.html">Actions (legacy format)</a></li> +<li><a href="rsyslog_conf_output.html">Output Channels</a></li> +<!--<li><a href="rsyslog_conf_examples.html">Examples</a></li>--> +<li><a href="rsyslog_conf_global.html">Legacy Configuration Directives</a></li> +<li><a href="rsyslog_conf_sysklogd_compatibility.html">sysklogd compatibility</a></li> </ul> <p>[<a href="rsyslog_conf.html">back to top</a>] @@ -38,7 +37,7 @@ especially useful while you are migrating from syslogd to rsyslogd.</p> [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body> diff --git a/doc/rsyslog_conf_actions.html b/doc/rsyslog_conf_actions.html index 2e2293ce..0c7705f8 100644 --- a/doc/rsyslog_conf_actions.html +++ b/doc/rsyslog_conf_actions.html @@ -3,7 +3,7 @@ <body> <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">back</a> -<h2>Actions</h2> +<h2>Actions (legacy format)</h2> <p>The action field of a rule describes what to do with the message. In general, message content is written to a kind of "logfile". But also other actions might be done, like writing to a database table diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html index 4ce78de0..00a700d4 100644 --- a/doc/rsyslog_conf_basic_structure.html +++ b/doc/rsyslog_conf_basic_structure.html @@ -1,33 +1,101 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html><head><title>Basic Structure - rsyslog.conf</title></head> <body> +<h1>Basic rsyslog.conf Structure</h1> <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> -<h1>Basic Structure</h1> -<p>Rsyslog supports standard sysklogd's configuration file format -and extends it. So in general, you can take a "normal" syslog.conf and -use it together with rsyslogd. It will understand everything. However, -to use most of rsyslogd's unique features, you need to add extended -configuration directives.</p> -<p>Rsyslogd supports the classical, selector-based rule lines. -They are still at the heart of it and all actions are initiated via -rule lines. A rule lines is any line not starting with a $ or the -comment sign (#). Lines starting with $ carry rsyslog-specific -directives.</p> -<p>Every rule line consists of two fields, a selector field and -an action field. These two fields are separated by one or more spaces -or tabs. The selector field specifies a pattern of facilities and -priorities belonging to the specified action.<br> -<br> -Lines starting with a hash mark ("#'') and empty lines are ignored. -</p> +<p>Rsyslog supports three different types of configuration statements +concurrently: +<ul> +<li><b>sysklogd</b> - this is the plain old format, thaught everywhere +and still pretty useful for simple use cases. Note that some very +few constructs are no longer supported because they are incompatible +with newer features. These are mentioned in the compatibility docs. +<li><b>legacy rsyslog</b> - these are statements that begin with a dollar +sign. They set some configuration parameters and modify e.g. the way +actions operate. This is the only format supported in pre-v6 versions of +rsyslog. It is still fully supported in v6 and above. Note that some +plugins and features may still only be available through legacy format +(because plugins need to be explicitely upgraded to use the new style +format, and this hasn't happened to all plugins). +<li><b>RainerScript</b> - the new style format. This is the best and most +precise format to be used for more complex cases. The rest of this page +assumes RainerScript based rsyslog.conf. +</ul> +<p>The rsyslog.conf files consists of statements. For old style (sysklogd & legacy +rsyslog), lines do matter. For new style (RainerScript) line spacing is irrelevant. +Most importantly, this means with new style actions and all other objects can split +across lines as users want to. +<h2>Comments</h2> +<p>There are two types of comments: +<ul> +<li><b>#-Comments</b> - start with a hash sign (#) and run to the end of the line +<li><b>C-style Comments</b> - start with /* and end with */, just like in the C +programming language. They can be used to comment out multiple lines at one. Comment +nesting is not supported, but #-Comments can be contained inside a C-style comment. +</ul> + +<h2>Processing Order</h2> +<p>Directives are processed from the top of rsyslog.conf to the bottom. Sequence +matters. For example, if you stop processing of a message, obviously all statements +after the stop statement are never evaluated. + +<h3>Flow Control Statements</h3> +<ul> +<li><b>if expr then ... else ...</b> - conditional execution +<li><b>stop</b> - stops processing the current message +<li><b>call</b> - calls a ruleset (just like a subroutine call) +<li><b>continue</b> - a NOP, useful e.g. inside the then part of an if +</ul> + +<h3>Data Manipulation Statements</h3> +<ul> +<li><b>set</b> - <a href="http://www.rsyslog.com/how-to-set-variables-in-rsyslog-v7/">sets</a> +a user variable +<li><b>unset</b> - deletes a previously set user variable +</ul> + +<h2>Inputs</h2> +<p>Every input requires an input module to be loaded and a listener defined for it. +Full details can be found inside the <a href="rsyslog_conf_modules.html">rsyslog +modules</a> documentation. Once loaded, inputs are defined via the +<b>input()</b> object. + +<h2>Outputs</h2> +<p>Outputs are also called "actions". A small set of actions is pre-loaded (like +the output file writer, which is used in almost every rsyslog.conf), others must +be loaded just like inputs. +<p>An action is invoked via the <b>action(type="type" ...)</b> object. Type is +mandatory and must contain the name of the plugin to be called (e.g. "omfile" or +"ommongodb"). Other paramters may be present. Their type and use depends on +the output plugin in question. + +<h2>Rulesets and Rules</h2> +<p>Rulesets and rules form the basis of rsyslog processing. In short, a rule +is a way how rsyslog shall process a specific message. Usually, there is a type +of filter (if-statement) in front of the rule. Complex nesting of rules is possible, +much like in a programming language. +<p>Rulesets are containers for rules. A single ruleset can contain many rules. In +the programming language analogy, one may think of a ruleset like being a program. +A ruleset can be "bound" (assigned) to a specific input. In the analogy, this means that when +a message comes in via that input, the "program" (ruleset) bound to it will be executed +(but not any other!). +<p>There is detail documentation available for +<a href="multi_ruleset">rsyslog rulesets</a>. +<p>For quick reference, rulesets are defined as follows: +<pre> +ruleset(name="rulesetname") { + action(type="omfile" file="/path/to/file") + action(type="..." ...) + /* and so on... */ +} +</pre> <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body> diff --git a/doc/rsyslog_conf_examples.html b/doc/rsyslog_conf_examples.html deleted file mode 100644 index b46460e5..00000000 --- a/doc/rsyslog_conf_examples.html +++ /dev/null @@ -1,209 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Examples - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h2>Examples</h2> -<p>Below are example for templates and selector lines. I hope -they are self-explanatory. If not, please see -www.monitorware.com/rsyslog/ for advise.</p> -<h3>TEMPLATES</h3> -<p>Please note that the samples are split across multiple lines. -A template MUST NOT actually be split across multiple lines.<br> -<br> -A template that resembles traditional syslogd file output:<br> -$template TraditionalFormat,"%timegenerated% %HOSTNAME%<br> -%syslogtag%%msg:::drop-last-lf%\n"<br> -<br> -A template that tells you a little more about the message:<br> -$template -precise,"%syslogpriority%,%syslogfacility%,%timegenerated%,%HOSTNAME%,<br> -%syslogtag%,%msg%\n"<br> -<br> -A template for RFC 3164 format:<br> -$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% -%syslogtag%%msg%"<br> -<br> -A template for the format traditonally used for user messages:<br> -$template usermsg," XXXX%syslogtag%%msg%\n\r"<br> -<br> -And a template with the traditonal wall-message format:<br> -$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at -%timegenerated%<br> -<br> -A template that can be used for the database write (please note the SQL<br> -template option)<br> -$template MySQLInsert,"insert iut, message, receivedat values<br> -('%iut%', '%msg:::UPPERCASE%', '%timegenerated:::date-mysql%')<br> -into systemevents\r\n", SQL<br> -<br> -The following template emulates <a href="http://www.winsyslog.com/en/">WinSyslog</a> -format (it's an <a href="http://www.adiscon.com/en/">Adiscon</a> -format, you do not feel bad if you don't know it ;)). It's interesting -to see how it takes different parts out of the date stamps. What -happens is that the date stamp is split into the actual date and time -and the these two are combined with just a comma in between them.<br> -<br> -$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,<br> -%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,<br> -%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,<br> -%syslogtag%%msg%\n"</p> -<h3>SELECTOR LINES</h3> -<p># Store critical stuff in critical<br> -#<br> -*.=crit;kern.none /var/adm/critical<br> -<br> -This will store all messages with the priority crit in the file -/var/adm/critical, except for any kernel message.<br> -<br> -<br> -# Kernel messages are first, stored in the kernel<br> -# file, critical messages and higher ones also go<br> -# to another host and to the console. Messages to<br> -# the host finlandia are forwarded in RFC 3164<br> -# format (using the template defined above).<br> -#<br> -kern.* /var/adm/kernel<br> -kern.crit @finlandia;RFC3164fmt<br> -kern.crit /dev/console<br> -kern.info;kern.!err /var/adm/kernel-info<br> -<br> -The first rule direct any message that has the kernel facility to the -file /var/adm/kernel.<br> -<br> -The second statement directs all kernel messages of the priority crit -and higher to the remote host finlandia. This is useful, because if the -host crashes and the disks get irreparable errors you might not be able -to read the stored messages. If they're on a remote host, too, you -still can try to find out the reason for the crash.<br> -<br> -The third rule directs these messages to the actual console, so the -person who works on the machine will get them, too.<br> -<br> -The fourth line tells rsyslogd to save all kernel messages that come -with priorities from info up to warning in the file -/var/adm/kernel-info. Everything from err and higher is excluded.<br> -<br> -<br> -# The tcp wrapper loggs with mail.info, we display<br> -# all the connections on tty12<br> -#<br> -mail.=info /dev/tty12<br> -<br> -This directs all messages that uses mail.info (in source LOG_MAIL | -LOG_INFO) to /dev/tty12, the 12th console. For example the tcpwrapper -tcpd(8) uses this as it's default.<br> -<br> -<br> -# Store all mail concerning stuff in a file<br> -#<br> -mail.*;mail.!=info /var/adm/mail<br> -<br> -This pattern matches all messages that come with the mail facility, -except for the info priority. These will be stored in the file -/var/adm/mail.<br> -<br> -<br> -# Log all mail.info and news.info messages to info<br> -#<br> -mail,news.=info /var/adm/info<br> -<br> -This will extract all messages that come either with mail.info or with -news.info and store them in the file /var/adm/info.<br> -<br> -<br> -# Log info and notice messages to messages file<br> -#<br> -*.=info;*.=notice;\<br> -mail.none /var/log/messages<br> -<br> -This lets rsyslogd log all messages that come with either the info or -the notice facility into the file /var/log/messages, except for all<br> -messages that use the mail facility.<br> -<br> -<br> -# Log info messages to messages file<br> -#<br> -*.=info;\<br> -mail,news.none /var/log/messages<br> -<br> -This statement causes rsyslogd to log all messages that come with the -info priority to the file /var/log/messages. But any message coming -either with the mail or the news facility will not be stored.<br> -<br> -<br> -# Emergency messages will be displayed using wall<br> -#<br> -*.=emerg *<br> -<br> -This rule tells rsyslogd to write all emergency messages to all -currently logged in users. This is the wall action.<br> -<br> -<br> -# Messages of the priority alert will be directed<br> -# to the operator<br> -#<br> -*.alert root,rgerhards<br> -<br> -This rule directs all messages with a priority of alert or higher to -the terminals of the operator, i.e. of the users "root'' and -"rgerhards'' if they're logged in.<br> -<br> -<br> -*.* @finlandia<br> -<br> -This rule would redirect all messages to a remote host called -finlandia. This is useful especially in a cluster of machines where all -syslog messages will be stored on only one machine.<br> -<br> -In the format shown above, UDP is used for transmitting the message. -The destination port is set to the default auf 514. Rsyslog is also -capable of using much more secure and reliable TCP sessions for message -forwarding. Also, the destination port can be specified. To select TCP, -simply add one additional @ in front of the host name (that is, @host -is UPD, @@host is TCP). For example:<br> -<br> -<br> -*.* @@finlandia<br> -<br> -To specify the destination port on the remote machine, use a colon -followed by the port number after the machine name. The following -forwards to port 1514 on finlandia:<br> -<br> -<br> -*.* @@finlandia:1514<br> -<br> -This syntax works both with TCP and UDP based syslog. However, you will -probably primarily need it for TCP, as there is no well-accepted port -for this transport (it is non-standard). For UDP, you can usually stick -with the default auf 514, but might want to modify it for security rea-<br> -sons. If you would like to do that, it's quite easy:<br> -<br> -<br> -*.* @finlandia:1514<br> -<br> -<br> -<br> -*.* >dbhost,dbname,dbuser,dbpassword;dbtemplate<br> -<br> -This rule writes all message to the database "dbname" hosted on -"dbhost". The login is done with user "dbuser" and password -"dbpassword". The actual table that is updated is specified within the -template (which contains the insert statement). The template is called -"dbtemplate" in this case.</p> -<p>:msg,contains,"error" @errorServer</p> -<p>This rule forwards all messages that contain the word "error" -in the msg part to the server "errorServer". Forwarding is via UDP. -Please note the colon in fron</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_file_syntax_differences.html b/doc/rsyslog_conf_file_syntax_differences.html deleted file mode 100644 index bfac8926..00000000 --- a/doc/rsyslog_conf_file_syntax_differences.html +++ /dev/null @@ -1,32 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Configuration File Syntax Differences - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> -<h1>Configuration File Syntax Differences</h1> -<p>Rsyslogd uses a slightly different syntax for its -configuration file than the original BSD sources. Originally all -messages of a specific priority and above were forwarded to the log -file. The modifiers "='', "!'' and "!-'' were added to make rsyslogd -more flexible and to use it in a more intuitive manner.<br> -<br> -The original BSD syslogd doesn't understand spaces as separators -between the selector and the action field.<br> -<br> -When compared to syslogd from sysklogd package, rsyslogd offers -additional -<a href="features.html">features</a> (like template -and database support). For obvious reasons, the syntax for defining -such features is available in rsyslogd, only.</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index fbced4a3..a795193f 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -4,38 +4,95 @@ <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">back</a> <h2>Filter Conditions</h2> -<p>Rsyslog offers four different types "filter conditions":</p> +<p>Rsyslog offers three different types "filter conditions":</p> <ul> -<li>BSD-style blocks</li> +<li><a href="http://www.rainerscript.com/">RainerScript</a>-based filters</li> <li>"traditional" severity and facility based selectors</li> <li>property-based filters</li> -<li>expression-based filters</li> </ul> -<h3>Blocks</h3> -<p>Rsyslogd supports BSD-style blocks inside rsyslog.conf. Each -block of lines is separated from the previous block by a program or -hostname specification. A block will only log messages corresponding to -the most recent program and hostname specifications given. Thus, a -block which selects ‘ppp’ as the program, directly followed by a block -that selects messages from the hostname ‘dialhost’, then the second -block will only log messages from the ppp program on dialhost. -</p> -<p>A program specification is a line beginning with ‘!prog’ and -the following blocks will be associated with calls to syslog from that -specific program. A program specification for ‘foo’ will also match any -message logged by the kernel with the prefix ‘foo: ’. Alternatively, a -program specification ‘-foo’ causes the following blocks to be applied -to messages from any program but the one specified. A hostname -specification of the form ‘+hostname’ and the following blocks will be -applied to messages received from the specified hostname. -Alternatively, a hostname specification ‘-hostname’ causes the -following blocks to be applied to messages from any host but the one -specified. If the hostname is given as ‘@’, the local hostname will be -used. (NOT YET IMPLEMENTED) A program or hostname specification may be -reset by giving the program or hostname as ‘*’.</p> -<p>Please note that the "#!prog", "#+hostname" and "#-hostname" -syntax available in BSD syslogd is not supported by rsyslogd. By -default, no hostname or program is set.</p> +<h3>RainerScript-Based Filters</h3> +RainerScript based filters are the prime means of creating complex rsyslog configuration. +The permit filtering on arbitrary complex expressions, which can include boolean, +arithmetic and string operations. They also support full nesting of filters, just +as you know from other scripting environments. +<br> +Scripts based filters are indicated by the keyword "if", as usual. +They have this format:<br> +<br> +if expr then block else block +<br> +"If" and "then" are fixed keywords that mus be present. "expr" is a +(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for +details. +The keyword "else" and its associated block is optional. Note that a block can contain either +a single action (chain), or an arbitrary complex script enclosed in curly braces, e.g.: +<br> +<pre> +if $programname == 'prog1' then { + action(type="omfile" file="/var/log/prog1.log") + if $msg contains 'test' then + action(type="omfile" file="/var/log/prog1test.log") + else + action(type="omfile" file="/var/log/prog1notest.log") +} +</pre> +<br> +Other types of filtes can also be combined with the pure RainerScript ones. This makes +it particularly easy to migrate from early config files to RainerScript. Also, the traditional +syslog PRI-based filters are a good and easy to use addition. While they are legacy, we still +recommend there use where they are up to the job. We do NOT, however, recommend property-based +filters any longer. As an example, the following is perfectly valid: +<br> +<pre> +if $fromhost == 'host1' then { + mail.* action(type="omfile" file="/var/log/host1/mail.log") + *.err /var/log/host1/errlog # this is also still valid + # + # more "old-style rules" ... + # +} else { + mail.* action(type="omfile" file="/var/log/mail.log") + *.err /var/log/errlog + # + # more "old-style rules" ... + # +} +</pre> +<br> + +Right now, you need to specify numerical values if you would like to +check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. +If you don't like that, you can of course also use the textual property +- just be sure to use the right one. As expression support is enhanced, +this will change. For example, if you would like to filter on message +that have facility local0, start with "DEVNAME" and have either +"error1" or "error0" in their message content, you could use the +following filter:<br> +<br> +<code> +if $syslogfacility-text == 'local0' and $msg +startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains +'error0') then /var/log/somelog<br> +</code> +<br> +Please note that the above <span style="font-weight: bold;">must +all be on one line</span>! And if you would like to store all +messages except those that contain "error1" or "error0", you just need +to add a "not":<br> +<br> +<code> +if $syslogfacility-text == 'local0' and $msg +startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> +($msg contains 'error1' or $msg contains +'error0') then /var/log/somelog<br> +</code> +<br> +If you would like to do case-insensitive comparisons, use +"contains_i" instead of "contains" and "startswith_i" instead of +"startswith".<br> +<br> +Regular expressions are supported via functions (see function list). + <h3>Selectors</h3> <p><b>Selectors are the traditional way of filtering syslog messages.</b> They have been kept in rsyslog with their original @@ -140,9 +197,14 @@ of the property value. For example, if you search for "val" with <p>it will be a match if msg contains "values are in this message" but it won't match if the msg contains "There are values in this message" (in the later case, contains would match). Please note -that "startswith" is by far faster than regular expressions. So even -once they are implemented, it can make very much sense -(performance-wise) to use "startswith".</p> +that "startswith" is by far faster than regular expressions. So +it makes very much sense (performance-wise) to use "startswith".</p> +<p>Note: when processing syslog messages, please note that $msg usually +starts with a space. The reason for this is RFC3164. Please read the +<a href="http://www.rsyslog.com/log-normalization-and-the-leading-space/">detail +description</a> of what that means to you. In short, you need to make sure +that you include the first space if you use "startswith", otherwise you will +not get matches. </td> </tr> <tr> @@ -213,71 +275,6 @@ supported (except for "not" as outlined above). Please note that while it is possible to query facility and severity via property-based filters, it is far more advisable to use classic selectors (see above) for those cases.</p> -<h3>Expression-Based Filters</h3> -Expression based filters allow -filtering on arbitrary complex expressions, which can include boolean, -arithmetic and string operations. Expression filters will evolve into a -full configuration scripting language. Unfortunately, their syntax will -slightly change during that process. So if you use them now, you need -to be prepared to change your configuration files some time later. -However, we try to implement the scripting facility as soon as possible -(also in respect to stage work needed). So the window of exposure is -probably not too long.<br> -<br> -Expression based filters are indicated by the keyword "if" in column 1 -of a new line. They have this format:<br> -<br> -if expr then action-part-of-selector-line<br> -<br> -"If" and "then" are fixed keywords that mus be present. "expr" is a -(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for -details. "action-part-of-selector-line" is an action, just as you know -it (e.g. "/var/log/logfile" to write to that file).<br> -<br> -A few quick samples:<br> -<br> -<code> -*.* /var/log/file1 # the traditional way<br> -if $msg contains 'error' then /var/log/errlog # the expression-based way<br> -</code> -<br> -Right now, you need to specify numerical values if you would like to -check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. -If you don't like that, you can of course also use the textual property -- just be sure to use the right one. As expression support is enhanced, -this will change. For example, if you would like to filter on message -that have facility local0, start with "DEVNAME" and have either -"error1" or "error0" in their message content, you could use the -following filter:<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -Please note that the above <span style="font-weight: bold;">must -all be on one line</span>! And if you would like to store all -messages except those that contain "error1" or "error0", you just need -to add a "not":<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> -($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -If you would like to do case-insensitive comparisons, use -"contains_i" instead of "contains" and "startswith_i" instead of -"startswith".<br> -<br> -Note that regular expressions are currently NOT -supported in expression-based filters. These will be added later when -function support is added to the expression engine (the reason is that -regular expressions will be a separate loadable module, which requires -some more prequisites before it can be implemented).<br> - <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> diff --git a/doc/rsyslog_conf_global.html b/doc/rsyslog_conf_global.html index a4d760eb..651808f6 100644 --- a/doc/rsyslog_conf_global.html +++ b/doc/rsyslog_conf_global.html @@ -67,7 +67,7 @@ default 0 (no delay). Simple rate-limiting!]</li> <li>$ActionQueueDiscardMark <number> [default 9750]</li> <li>$ActionQueueDiscardSeverity <number> -[*numerical* severity! default 4 (warning)]</li> +[*numerical* severity! default 8 (nothing discarded)]</li> <li>$ActionQueueFileName <name></li> <li>$ActionQueueHighWaterMark <number> [default 8000]</li> diff --git a/doc/rsyslog_conf_lines.html b/doc/rsyslog_conf_lines.html deleted file mode 100644 index 0e6cc0d3..00000000 --- a/doc/rsyslog_conf_lines.html +++ /dev/null @@ -1,23 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Lines - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> -<h1>Lines</h1> -<p>Lines can be continued by specifying a backslash ("\") as the last -character of the line. There is a hard-coded maximum line length of 4K.<br> -If you need lines larger than that, you need to change compile-time -settings inside rsyslog and recompile. -</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index cbd60faf..4186dac4 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -35,7 +35,7 @@ to message generators. <ul> <li><a href="imfile.html">imfile</a> - input module for text files</li> <li><a href="imrelp.html">imrelp</a> - RELP input module</li> -<li>imudp - udp syslog message input</li> +<li><a href="imudp.html">imudp</a> - udp syslog message input</li> <li><a href="imtcp.html">imtcp</a> - input plugin for tcp syslog</li> <li><a href="imptcp.html">imptcp</a> - input plugin for plain tcp syslog (no TLS but faster)</li> <li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li> diff --git a/doc/rsyslog_conf_sysklogd_compatibility.html b/doc/rsyslog_conf_sysklogd_compatibility.html new file mode 100644 index 00000000..c95d6fda --- /dev/null +++ b/doc/rsyslog_conf_sysklogd_compatibility.html @@ -0,0 +1,31 @@ +<html><head><title>sysklogdcompatibility - rsyslog.conf</title></head> +<body> +<h1>sysklogd compatibility</h1> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<p>Rsyslog supports standard sysklogd's configuration file format +and extends it. So in general, you can take a "normal" syslog.conf and +use it together with rsyslogd. It will understand everything. However, +to use most of rsyslogd's unique features, you need to add extended +configuration directives.</p> +<p>Rsyslogd supports the classical, selector-based rule lines. +They are still at the heart of it and all actions are initiated via +rule lines. +However, there are ample new directives, either in rsyslog traditional +format (starting with a dollar sign) or in RainerScript format. These +work together with sysklogd statements. A few select statements are +no longer supported and may generate error messages. They are mentioned +in the compatibility notes. +</p> + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html index b97f6609..0c189100 100644 --- a/doc/rsyslog_conf_templates.html +++ b/doc/rsyslog_conf_templates.html @@ -3,7 +3,7 @@ <body> <p>This is a part of the rsyslog.conf - documentation.</p> <a href="rsyslog_conf.html">back</a> -<h2>Templates</h2> +<h1>Templates</h1> <p>Templates are a key feature of rsyslog. They allow to specify any format a user might want. They are also used for dynamic file name @@ -16,67 +16,200 @@ compatible with the stock syslogd formats are hardcoded into rsyslogd. So if no template is specified, we use one of these hardcoded templates. Search for "template_" in syslogd.c and you will find the hardcoded ones.</p> -<p>Starting with 5.5.6, there are actually two differnt types of template: +<p>Templates are specified by template() statements. They can also be specified +via $Template legacy statements. Note that these are scheduled for removal in +later versions of rsyslog, so it is probably a good idea to avoid them +for new uses. +<h2>The template() statement</h2> +<p>The template() statement is used to define templates. Note that it is a +<b>static</b> statement, that means all templates are defined when rsyslog +reads the config file. As such, templates are not affected by if-statements +or config nesting. +<p>The basic structure of the template statement is as follows: +<br><br> +<code>template(parameters)</code> +<br><br> +In addition to this simpler syntax, list templates (to be described below) +support an extended syntax: +<br><br> +<code>template(parameters) { list-descriptions }</code> +<p>Each template has a parameter <b>name</b>, which specifies the templates +name, and a parameter <b>type</b>, which specifies the template type. The name +parameter must be unique, and behaviour is unpredictable if it is not. The <b>type</b> +parameter specifies different template types. Different types simply enable +different ways to specify the template content. The template type <b>does not</b> +affect what an (output) plugin can do with it. So use the type that best fits your +needs (from a config writing point of view!). The following types are available: <ul> -<li>string based -<li>string-generator module based +<li>list +<li>subtree +<li>string +<li>plugin </ul> -<p><a href="rsyslog_conf_modules.html#sm">String-generator module</a> based templates -have been introduced in 5.5.6. They permit a string generator, actually a C "program", -the generate a format. Obviously, it is more work required to code such a generator, -but the reward is speed improvement. If you do not need the ultimate throughput, you -can forget about string generators (so most people never need to know what they are). -You may just be interested in learning that for the most important default formats, -rsyslog already contains highly optimized string generators and these are called -without any need to configure anything. But if you have written (or purchased) a -string generator module, you need to know how to call it. Each such module has a name, -which you need to know (look it up in the module doc or ask the developer). Let's assume -that "mystrgen" is the module name. Then you can define a template for that strgen -in the following way: +The various types are described below. -<blockquote><code>template(name="MyTemplateName" type="plugin" string="mystrgen")</code></blockquote> -<p>Legacy example:</p> -<blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote> -(Of course, you must have first loaded the module via $ModLoad). -<p>The important part is the equal sign in the legacy format: it tells the rsyslog config parser that -no string follows but a strgen module name. -<p>There are no additional parameters but the module name supported. This is because -there is no way to customize anything inside such a "template" other than by -modifying the code of the string generator. +<h3>list</h3> +<p>In this case, the template is generated by a list of constant and +variable statements. These follow the template spec in curly braces. This type is +also primarily meant for use with structure-aware outputs, like ommongodb. However, +it also works perfectly with text-based outputs. We recommend to use this mode +if more complex property substitutions needs to be done. In that case, the list-based +template syntax is much clearer than the simple string-based one. +<p>The list template contains the template header (with <b>type="list"</b>) and is followed +by <b>constant</b> and <b>property</b> statements, given in curly braces to signify +the template statement they belong to. As the name says, <b>constant</b> statements +describe constant text and <b>property</b> describes property access. There are many options +to <b>property</b>, described further below. Most of these options are used to extract +only partial property contents or to modify the text obtained (like to change its case +to upper or lower case, only). +<p>To grasp the idea, an actual sample is: +<br><pre><code>template(name="tpl1" type="list") { + constant(value="Syslog MSG is: '") + property(name="msg") + constant(value="', ") + property(name="timereported" dateFormat="rfc3339" caseConversion="lower") + constant(value="\n") + } +</code></pre> +<br>This sample is probably primarily targeted at the usual file-based output.</p> -<p>So for most use cases, string-generator module based templates are <b>not</b> -the route to take. Usually, we use <b>string based templates</b> instead. -This is what the rest of the documentation now talks about. -<p>A template consists of a template directive, a name, the -actual template text and optional options. A sample is:</p> -<blockquote><code>template(name="MyTemplateName" type="string" string="Example: Text %property% some more text\n" options)</code></blockquote> -<p>Legacy example:</p> -<blockquote><code>$template MyTemplateName,"\7Text -%property% some more text\n",<options></code></blockquote> -<p>The "template" (legacy: $template) is the template directive. It tells rsyslog -that this line contains a template. "MyTemplateName" is the template -name. All -other config lines refer to this name. The text within "string" is the -actual template text. The backslash is an escape character, much as it -is in C. It does all these "cool" things. For example, \7 rings the -bell (this is an ASCII value), \n is a new line. C programmers and perl -coders have the advantage of knowing this, but the set in rsyslog is a -bit restricted currently. -</p> -<p>All text in the template is used literally, except for things -within percent signs. These are properties and allow you access to the -contents of the syslog message. Properties are accessed via the -<a href="property_replacer.html">property replacer</a> -(nice name, huh) and it can do cool things, too. For -example, it can pick a substring or do date-specific formatting. More -on this is below, on some lines of the property replacer.<br> -<br> +<h4>constant statement</h4> +<p>This provides a way to specify constant text. The text is used literally. It is +primarily intended for text-based output, so that some constant text can be included. For +example, if a complex template is build for file output, one usually needs to finish it +by a newline, which can be introduced by a constant statement. Here is an actual sample +of that use case from the rsylsog testbench: +<br><pre><code>template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +}</code></pre> +The following escape sequences are recogniced inside the constant text: +<ul> +<li>\\ - single backslash +<li>\n - LF +<li>\ooo - (three octal digits) - represents character with this numerical value (e.g. \101 +equals "A"). Note that three +octal digits must be given (in contrast to some languagues, where between one and three are valid). +While we support octal notation, we recommend to use hex notation as this is better known. +<li>\xhh - (where h is a hex digit) - represents character with this numerical value (e.g. \x41 +equals "A"). Note that two hexadecimal digits must be given (in contrast to some languagues +where one or two are valid). +<li>... some others ... list needs to be extended +</ul> +<p>Note: if an unsupported character follows a backslash, this is treated as an error. Behaviour +is unpredictable in this case. +<p>To aid usage of the same template both for text-based outputs and structured ones, constant +text without an "outname" parameter will be ignored when creating the name/value tree +for structured outputs. So if you want to supply some constant text e.g. to mongodb, you must +include an outname, as can be seen here: +<br><pre><code>template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n" <b>outname="IWantThisInMyDB"</b>) +}</code></pre> + +The "constant" statement supports the following parameters: +<ul> +<li>value - the constant value to use +<li>outname - output field name (for structured outputs) +</ul> + + +<h4>property statement</h4> +<p>This statement is used to include property text. It can access all properties. Also, +options permit to specify picking only part of a property or modifying it. +It supports the following parameters: +<ul> +<li>name - the name of the property to access +<li>outname - output field name (for structured outputs) +<li>dateformat - date format to use (only for date-related properties) +<li>caseconversion - permits to convert case of the text. supported values are +"lower" and "upper" +<li>controlcharacters - specifies how to handle control characters. Supported values are +"escape", which escapes them, "space", which replaces them by a single space, and +"drop", which simply removes them from the string. +<li>securepath - used for creating pathnames suitable for use in dynafile templates +<li>format - specifiy format on a field basis. Supported values are "csv", for use when +csv-data is generated, "json", which formats proper json content (but without a field +header) and "jsonf", which formats as a complete json field. +<li>position.from - obtain substring starting from this position (1 is the first position) +<li>position.to - obtain substring up to this position +<li>field.number - obtain this field match +<li>field.delimiter - decimal value of delimiter character for field extraction +<li>regex.expression - expression to use +<li>regex.type - either ERE or BRE +<li>regex.nomatchmode - what to do if we have no match +<li>regex.match - match to use +<li>regex.submatch - submatch to use +<li>droplastlf - drop a trailing LF, if it is present +<li>mandatory - signifies a field as mandatory. If set to "on", this field will always +be present in data passed to structured outputs, even if it is empty. If "off" (the default) +empty fields will not be passed to structured outputs. This is especially useful for outputs +that support dynamic schemas (like ommongodb). +<li>spifno1stsp - expert options for RFC3164 template processing +</ul> + + +<h3>subtree</h3> +<p>Available since rsyslog 7.1.4 +<p> +In this case, the template is generated based on a complete +(CEE) subtree. This type of template is most useful for outputs that know how to +process hierarchical structure, like ommongodb. With that type, the parameter +<b>subtree</b> must be specified, which tells which subtree to use. For example +template(name="tpl1" type="subtree" subtree="$!") includes all CEE data, while +template(name="tpl2" type="subtree" subtree="$!usr!tpl2") includes only the +subtree starting at $!usr!tpl2. The core idea when using this type of template +is that the actual data is prefabricated via set and unset script statements, +and the resulting strucuture is then used inside the template. This method MUST +be used if a complete subtree needs to be placed <i>directly</i> into the +object's root. With all other template types, only subcontainers can be generated. +Note that subtree type can also be used with text-based outputs, like omfile. HOWEVER, +you do not have any capability to specify constant text, and as such cannot include +line breaks. As a consequence, using this template type for text outputs is usually +only useful for debugging or very special cases (e.g. where the text is interpreted +by a JSON parser later on). +<h4>Use case</h4> +<p>A typical use case is to first create a custom subtree and then include it into +the template, like in this small example: +<br><blockquote><code>set $!usr!tpl2!msg = $msg; +<br>set $!usr!tpl2!dataflow = field($msg, 58, 2); +<br>template(name="tpl2" type="subtree" subtree="$!usr!tpl2") +</code></blockquote> +<p>Here, we assume that $msg contains various fields, and the data from a field +is to be extracted and stored - together with the message - as field content. +<h3>string</h3> +<p>This closely resembles the legacy template statement. It +has a mandatory parameter <b>string</b>, which holds the template string to be +applied. A template string is a mix of constant text and replacement variables +(see property replacer). These variables are taken from message or other dynamic +content when the final string to be passed to a plugin is generated. String-based +templates are a great way to specify textual content, especially if no complex +manipulation to properties is necessary. Full details on how to specify template +text can be found below. +<br>Config example: +<br><blockquote><code>template(name="tpl3" type="string" string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n") +</code></blockquote> +<h3>plugin</h3> +In this case, the template is generated by a plugin (which +is then called +a "strgen" or "string generator"). The format is fix as it is coded. While this +is inflexible, it provides superior performance, and is often used for that +reason (not that "regular" templates are slow - but in very demanding environments +that "last bit" can make a difference). Refer to the plugin's documentation +for further details. For this type, the paramter <b>plugin</b> must be specified and +must contain the name of the plugin as it identifies itself. Note that the +plugin must be loaded prior to being used inside a template. +<br>Config example: +<br><blockquote><code>template(name="tpl4" type="plugin" plugin="mystrgen") +</code></blockquote> + +<h3>options</h3> The <options> part is optional. It carries options -influencing the template as whole. See details below. Be sure NOT to -mistake template options with property options - the latter ones are -processed by the property replacer and apply to a SINGLE property, only -(and not the whole template).<br> +influencing the template as whole and is part of the template parameters. +See details below. Be sure NOT to mistake template options with property +options - the latter ones are processed by the property replacer and +apply to a SINGLE property, only (and not the whole template).<br> <br> Template options are case-insensitive. Currently defined are: </p> <p><b>option.sql</b> - format the string suitable for a SQL @@ -127,50 +260,102 @@ option. Otherwise you will become vulnerable to SQL injection. <br> To escape:<br> % = \%<br> \ = \\ --> '\' is used to escape (as in C)<br> -$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> +template (name="TraditionalFormat" type="string" string="%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> <br> -Properties can be accessed by the <a href="property_replacer.html">property -replacer</a> (see there for details).</p> -<p>Templates can be used in the form of a <b>list</b> as well. This has been -introduced with <b>6.5.0</b> The list consists of two parts which are either -a <b>constant</b> or a <b>property</b>. The constants -are taking the part of "text" that you usually enter in string-based templates. -The properties stay variable, as they are a substitute for different values of a -certain type. This type of template is extremely useful for complicated cases, -as it helps you to easily keep an overview over the template. Though, it has -the disadvantage of needing more effort to create it.</p> -<br>Config example: -<br><blockquote><code>template(name="MyTemplate" type="list" option.json="off") { - <br>constant(value="Test: ") - <br>property(name="msg" outname="mymessage") - <br>constant(value=" --!!!-- ") - <br>property(name="timereported" dateFormat="rfc3339" caseConversion="lower") - <br>constant(value="\n") - <br>} -</code></blockquote> -<p>First, the general template option will be defined. The values of the template -itself get defined in the curly brackets. As it can be seen, we have constants -and properties in exchange. Whereas constants will be filled with a value and probably -some options, properties do direct to a property and the options that could be needed -additional format definitions.</p> -<p>We suggest to use separate lines for all constants and properties. This -helps to keep a good overview over the different parts of the template. -Though, writing it in a single line will work, it is much harder to debug -if anything goes wrong with the template. </p> +<h3>Examples</h3> +<h4>Standard Template for Writing to Files</h4> +<p><pre><code>template(name="FileFormat" type="list") { + property(name="timestamp" dateFormat="rfc3339") + constant(value=" ") + property(name="hostname") + constant(value=" ") + property(name="syslogtag") + constant(value=" ") + property(name="msg" spifno1stsp="on" ) + property(name="msg" droplastlf="on" ) + constant(value="\n") + } +</code></pre> +<p>The equivalent string template looks like this: +<br><pre><code>template(name="FileFormat" type="string" + string= "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" +)</code></pre> +Note that the template string itself must be on a single line. -<p><b>Please note that templates can also be -used to generate selector lines with dynamic file names.</b> For -example, if you would like to split syslog messages from different -hosts to different files (one per host), you can define the following -template:</p> -<blockquote><code>template (name="DynFile" type="string" string="/var/log/system-%HOSTNAME%.log")</code></blockquote> -<p>Legacy example:</p> -<blockquote><code>$template -DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote> -<p>This template can then be used when defining an output -selector line. It will result in something like -"/var/log/system-localhost.log"</p> +<h4>Standard Template for Forwarding to a Remote Host (RFC3164 mode)</h4> +<p><pre><code>template(name="ForwardFormat" type="list") { + constant(value="<") + property(name="PRI") + constant(value="<") + property(name="timestamp" dateFormat="rfc3339") + constant(value=" ") + property(name="hostname") + constant(value=" ") + property(name="syslogtag" position.from="1" position.to="32") + constant(value=" ") + property(name="msg" spifno1stsp="on" ) + } +</code></pre> +<p>The equivalent string template looks like this: +<br><pre><code>template(name="forwardFormat" type="string" + string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" +)</code></pre> +Note that the template string itself must be on a single line. + +<h4>Standard Template for write to the MySQL database</h4> +<p><pre><code>template(name="StdSQLformat" type="list" option.sql="on") { + constant(value="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag)") + constant(value=" values ('") + property(name="msg") + constant(value="', ") + property(name="syslogfacility") + constant(value=", '") + property(name="hostname") + constant(value="', ") + property(name="syslogpriority") + constant(value=", '") + property(name="timereported" dateFormat="mysql") + constant(value="', '") + property(name="timegenerated" dateFormat="mysql") + constant(value="', ") + property(name="iut") + constant(value=", '") + property(name="syslogtag") + constant(value="')") + } +</code></pre> +<p>The equivalent string template looks like this: +<br><pre><code>template(name="stdSQLformat" type="string" option.sql="on" + string="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')" +)</code></pre> +Note that the template string itself must be on a single line. + +<h2>legacy format</h2> +<p>In pre v6-versions of rsyslog, you need to use the <code>$template</code> +statement to configure templates. They provide the equivalent to string- and +plugin-based templates. The legacy syntax continous to work in v7, however +we recommend to avoid legacy format for newly written config files. Legacy and +current config statements can coexist within the same config file. +<p>The general format is +<br><br><code>$template name,param[,options]</code></br></br> +where "name" is the template name and "param" is a single parameter +that specifies template content. The optional "options" part is used to +set template options. +<h3>string</h3> +The parameter is the same string that with the current-style format you +specify in the <b>string</b> parameter, for example: +<br><br><code>$template strtpl,"PRI: %pri%, MSG: %msg%\n"</code> +<p>Note that list templates are not available in legacy format, so you need +to use complex property replacer constructs to do complex things. + +<h3>plugin</h3> +This is equivalent to the "plugin"-type template directive. Here, the +parameter is the plugin name, with an equal sign prepended. An example +is: +<br><br><code>$template plugintpl,=myplugin</code> + +<h2>Reserved Template Names</h2> <p>Template names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT use them if, otherwise you may receive a conflict in the future (and @@ -210,12 +395,122 @@ out, but this may happen.</li> is meant to be written to a log file. Do <b>not</b> use for production or remote forwarding.</li> </ul> + +<h2>The following is legacy documentation soon to be integrated.</h2> + +<!--<table> +<tr><td>param name</td><td>meaning</td></tr> +<tr><td>name</td><td>name of the template</td></tr> +</table> +--> + +<p>Starting with 5.5.6, there are actually two different types of template: +<ul> +<li>string based +<li>string-generator module based +</ul> +<p><a href="rsyslog_conf_modules.html#sm">String-generator module</a> based templates +have been introduced in 5.5.6. They permit a string generator, actually a C "program", +the generate a format. Obviously, it is more work required to code such a generator, +but the reward is speed improvement. If you do not need the ultimate throughput, you +can forget about string generators (so most people never need to know what they are). +You may just be interested in learning that for the most important default formats, +rsyslog already contains highly optimized string generators and these are called +without any need to configure anything. But if you have written (or purchased) a +string generator module, you need to know how to call it. Each such module has a name, +which you need to know (look it up in the module doc or ask the developer). Let's assume +that "mystrgen" is the module name. Then you can define a template for that strgen +in the following way: + +<blockquote><code>template(name="MyTemplateName" type="plugin" string="mystrgen")</code></blockquote> +<p>Legacy example:</p> +<blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote> +(Of course, you must have first loaded the module via $ModLoad). +<p>The important part is the equal sign in the legacy format: it tells the rsyslog config parser that +no string follows but a strgen module name. +<p>There are no additional parameters but the module name supported. This is because +there is no way to customize anything inside such a "template" other than by +modifying the code of the string generator. + +<p>So for most use cases, string-generator module based templates are <b>not</b> +the route to take. Usually, we use <b>string based templates</b> instead. +This is what the rest of the documentation now talks about. + +<p>A template consists of a template directive, a name, the +actual template text and optional options. A sample is:</p> +<blockquote><code>template(name="MyTemplateName" type="string" string="Example: Text %property% some more text\n" options)</code></blockquote> +<p>Legacy example:</p> +<blockquote><code>$template MyTemplateName,"\7Text +%property% some more text\n",<options></code></blockquote> +<p>The "template" (legacy: $template) is the template directive. It tells rsyslog +that this line contains a template. "MyTemplateName" is the template +name. All +other config lines refer to this name. The text within "string" is the +actual template text. The backslash is an escape character, much as it +is in C. It does all these "cool" things. For example, \7 rings the +bell (this is an ASCII value), \n is a new line. C programmers and perl +coders have the advantage of knowing this, but the set in rsyslog is a +bit restricted currently. +</p> +<p>All text in the template is used literally, except for things +within percent signs. These are properties and allow you access to the +contents of the syslog message. Properties are accessed via the +<a href="property_replacer.html">property replacer</a> +(nice name, huh) and it can do cool things, too. For +example, it can pick a substring or do date-specific formatting. More +on this is below, on some lines of the property replacer.<br> +<br> + +<br> +Properties can be accessed by the <a href="property_replacer.html">property +replacer</a> (see there for details).</p> +<p>Templates can be used in the form of a <b>list</b> as well. This has been +introduced with <b>6.5.0</b> The list consists of two parts which are either +a <b>constant</b> or a <b>property</b>. The constants +are taking the part of "text" that you usually enter in string-based templates. +The properties stay variable, as they are a substitute for different values of a +certain type. This type of template is extremely useful for complicated cases, +as it helps you to easily keep an overview over the template. Though, it has +the disadvantage of needing more effort to create it.</p> +<br>Config example: +<br><blockquote><code>template(name="MyTemplate" type="list" option.json="off") { + <br>constant(value="Test: ") + <br>property(name="msg" outname="mymessage") + <br>constant(value=" --!!!-- ") + <br>property(name="timereported" dateFormat="rfc3339" caseConversion="lower") + <br>constant(value="\n") + <br>} +</code></blockquote> +<p>First, the general template option will be defined. The values of the template +itself get defined in the curly brackets. As it can be seen, we have constants +and properties in exchange. Whereas constants will be filled with a value and probably +some options, properties do direct to a property and the options that could be needed +additional format definitions.</p> +<p>We suggest to use separate lines for all constants and properties. This +helps to keep a good overview over the different parts of the template. +Though, writing it in a single line will work, it is much harder to debug +if anything goes wrong with the template. </p> + +<p><b>Please note that templates can also be +used to generate selector lines with dynamic file names.</b> For +example, if you would like to split syslog messages from different +hosts to different files (one per host), you can define the following +template:</p> +<blockquote><code>template (name="DynFile" type="string" string="/var/log/system-%HOSTNAME%.log")</code></blockquote> +<p>Legacy example:</p> +<blockquote><code>$template +DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote> +<p>This template can then be used when defining an output +selector line. It will result in something like +"/var/log/system-localhost.log"</p> <h3>Legacy String-based Template Samples</h3> -<p>This section provides some sample of what the default formats would -look as a text-based template. Hopefully, their description is self-explanatory. +<p>This section provides some default templates in legacy format, as used in rsyslog +previous to version 6. Note that this format is still supported, so there is no hard need +to upgrade existing configurations. However, it is strongly recommended that the legacy +constructs are not used when crafting new templates. Note that each $Template statement is on a <b>single</b> line, but probably broken accross several lines for display purposes by your browsers. Lines are separated by -empty lines. +empty lines. Keep in mind, that line breaks are important in legacy format. <p><code> $template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" <br><br> @@ -233,7 +528,7 @@ $template StdSQLFormat,"insert into SystemEvents (Message, Facility, FromHost, P [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 2 or higher.</font></p> </body> diff --git a/doc/v4compatibility.html b/doc/v4compatibility.html index 72b0f5a9..2a51adea 100644 --- a/doc/v4compatibility.html +++ b/doc/v4compatibility.html @@ -60,7 +60,7 @@ restarting rsyslogd by HUPing it. and most other deamons require that a restart command is typed in if a restart is required. <p>Rsyslog will follow this paradigm in the next versions, resulting in many benefits. In v4, we provide some support for the old-style semantics. We introduced a setting $HUPisRestart -which may be set to "on" (tradional, heavy operationg) +which may be set to "on" (tradional, heavy operation) or "off" (new, lightweight "file close only" operation). The initial versions had the default set to traditional behavior, but starting with 4.5.1 we are now using the new behavior as the default. diff --git a/doc/v7compatibility.html b/doc/v7compatibility.html new file mode 100644 index 00000000..01faacac --- /dev/null +++ b/doc/v7compatibility.html @@ -0,0 +1,91 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Compatibility notes for rsyslog v7</title> +</head> +<body> +<h1>Compatibility Notes for rsyslog v7</h1> +This document describes things to keep in mind when moving from v6 to v7. It +does not list enhancements nor does it talk about compatibility concerns introduced +by earlier versions (for this, see their respective compatibility documents). Its focus +is primarily on what you need to know if you used v6 and want to use v7 without hassle. +<p>Version 7 builds on the new config language introduced in v6 and extends it. +Other than v6, it not just only extends the config language, but provides +considerable changes to core elements as well. The result is much more power and +ease of use as well (this time that is not contradictionary). +</p> +<h2>BSD-Style blocks</h2> +BSD style blocks are no longer supported (for good reason). See the +<a href="http://www.rsyslog.com/g/BSD">rsyslog BSD blocks info</a> +page for more information and how to upgrade your config. +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> + +<h2>CEE-Properties</h2> +In rsyslog v6, CEE properties could not be used across disk-based queues. If this was +done, there content was reset. This was a missing feature in v6. In v7, this feature +has been implemented. Consequently, situations where the previous behaviour were +desired need now to be solved differently. We do not think that this will cause any +problems to anyone, especially as in v6 this was announced as a missing feature. + +<h2>omusrmsg: using just a username or "*" is deprecated</h2> +<p>In legacy config format, the asterisk denotes writing the message to all users. +This is usually used for emergency messages and configured like this: +<pre> +*.emerg * +</pre> +<p>Unfortunately, the use of this single character conflicts with other uses, for +example with the multiplication operator. While rsyslog up to versions v7.4 preserves the meaning of +asterisk as an action, it is deprecated and will probably be removed in future versions. +Consequently, a warning message is emitted. To make this warning go away, the action must +be explicitly given, as follows: +<pre> +*.emerg :omusrmsg:* +</pre> +<p>The same holds true for user names. For example +<pre> +*.emerg john +</pre> +<p>at a minimum should be rewritten as +<pre> +*.emerg :omusrmsg:john +</pre> +<p>Of course, for even more clarity the new RainerScript style of action can +also be used: +<pre> +*.emerg action(type="omusrmsg" users="john") +</pre> +<p>In Rainer's blog, there is more +<a href="http://blog.gerhards.net/2011/07/why-omusrmsg-is-evil-and-how-it-is.html">background +information on why omusrmsg needed to be changed</a> available. + +<h2>omruleset and discard (~) action are deprecated</h2> +<p>Both continue to work, but have been replaced by better alternatives. +<p>The discard action (tilde character) has been replaced by the "stop" +RainerScript directive. It is considered more intuitive and offers slightly +better performance. +<p>The omruleset module has been replaced by the "call" RainerScript directive. +Call permits to execute a ruleset like a subroutine, and does so with much +higher performance than omruleset did. Note that omruleset could be run off +an async queue. This was more a side than a desired effect and is not supported +by the call statement. If that effect was needed, it can simply be simulated by +running the called rulesets actions asynchronously (what in any case is the right +way to handle this). +<p>Note that the deprecated modules emit warning messages when being used. +They tell that the construct is deprecated and which statement is to be used +as replacement. This does <b>not</b> affect operations: both modules are still +fully operational and will not be removed in the v7 timeframe. + +<h2>Retries of output plugins that do not do proper replies</h2> +<p>Some output plugins may not be able to detect if their target is capable of +accepting data again after an error (technically, they always return OK when +TryResume is called). Previously, the rsyslog core engine suspended such an action +after 1000 succesive failures. This lead to potentially a large amount of +errors and error messages. Starting with 7.2.1, this has been reduced to 10 +successive failures. This still gives the plugin a chance to recover. In extreme +cases, a plugin may now enter suspend mode where it previously did not do so. +In practice, we do NOT expect that. + +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2011-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 2 or higher.</font></p> +</body></html> |