diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/imjournal.html | 68 | ||||
| -rw-r--r-- | doc/imptcp.html | 5 | ||||
| -rw-r--r-- | doc/imrelp.html | 51 | ||||
| -rw-r--r-- | doc/manual.html | 2 | ||||
| -rw-r--r-- | doc/mmcount.html | 58 | ||||
| -rw-r--r-- | doc/omfile.html | 8 | ||||
| -rw-r--r-- | doc/omfwd.html | 33 | ||||
| -rw-r--r-- | doc/omjournal.html | 3 | ||||
| -rw-r--r-- | doc/omrelp.html | 27 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 1 |
10 files changed, 236 insertions, 20 deletions
diff --git a/doc/imjournal.html b/doc/imjournal.html index dbf9279e..5a18d5d6 100644 --- a/doc/imjournal.html +++ b/doc/imjournal.html @@ -1,6 +1,6 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html><head> -<meta http-equiv="Content-Language" content="en"><title>Text File Input Monitor</title></head> +<meta http-equiv="Content-Language" content="en"><title>Systemd Journal Input Module</title></head> <body> <a href="rsyslog_conf_modules.html">back</a> @@ -11,28 +11,84 @@ <p><b>Description</b>:</p> <p>Provides the ability to import structured log messages from systemd journal to syslog.</p> +<p>Note that this module reads the journal database, what is considered a +relativly performance-intense operation. As such, the performance of a +configuration utilizing this +module may be notably slower then when using +<a href="imuxsock.html">imuxsock</a>. The journal provides imuxsock with a +copy of all "classical" syslog messages, however, it does not provide +structured data. If the latter is needed, imjournal must be used. Otherwise, +imjournal may be simply replaced by imuxsock. +<p>We suggest to check out our short presentation on +<a href="http://youtu.be/GTS7EuSdFKE">rsyslog journal integration</a> to +learn more details of anticipated use cases. + +<p><b>Warning:</b> Some versions of systemd journal have problems with database +corruption, which leads to the journal to return the same data endlessly +in a thight loop. This results in massive message duplication inside rsyslog +probably resulting in a denial-of-service when the system ressouces get +exhausted. This can be somewhat mitigated by using proper rate-limiters, but +even then there are spikes of old data which are endlessly repeated. By default, +ratelimiting is activated and permits to process 20,000 messages within 10 +seconds, what should be well enough for most use cases. If insufficient, use +the parameters described below to adjust the permitted volume. +<b>It is strongly recommended to use this plugin only if there +is hard need to do so.</b> <p><b>Configuration Directives</b>:</p> <p><b>Module Directives</b></p> <ul> -<li><span style="font-weight: bold;">PersistStateInterval -messages</span><br> +<li><b>PersistStateInterval</b> messages<br> This is a global setting. It specifies how often should the journal state be persisted. This option is useful for rsyslog to start reding from the last journal message it read. -<li><span style="font-weight: bold;">StateFile -/path/to/file</span><br> +<li><b>StateFile</b> /path/to/file<br> This is a global setting. It specifies where the state file for persisting journal state is located. + +<li><b>ratelimit.interval</b> seconds (default: 600)<br> +Specifies the interval in seconds onto which rate-limiting is to be applied. +If more than ratelimit.burst messages are read during that interval, further +messages up to the end of the interval are discarded. The number of messages +discarded is emitted at the end of the interval (if there were any discards). +<br>Setting this to value zero turns off ratelimiting. Note that it is +<b>not recommended to turn of ratelimiting</b>, except that you know for +sure journal database entries will never be corrupted. Without ratelimiting, +a corrupted systemd journal database may cause a kind of denial of service (we +are stressing this point as multiple users have reported us such problems +with the journal database - information current as of June 2013). + +<li><b>ratelimit.burst</b> messages (default: 20000)<br> +Specifies the maximum number of messages that can be emitted within the +ratelimit.interval interval. For futher information, see description there. + +</ul> + +<p><b>Legacy Configuration Directives</b>:</p> +<ul> +<li>$imjournalPersistStateInterval <Delimiter><br> +Equivalent to: ratelimit.PersistStateInterval</li> +<li>$imjournalStateFile <Delimiter><br> +Equivalent to: ratelimit.StateFile</li> +<li>$imjournalRatelimitInterval <Delimiter><br> +Equivalent to: ratelimit.interval</li> +<li>$imjournalRatelimitBurst <Delimiter><br> +Equivalent to: ratelimit.burst</li> </ul> + <b>Caveats/Known Bugs:</b> <p> +<ul> +<li>As stated above, a corrupted systemd journal database can cause major +problems, depending on what the corruption results in. This is beyond the +control of the rsyslog team. +</ul> </p> <p><b>Sample:</b></p> <p> The following example shows pulling structured imjournal messages and saving them into /var/log/ceelog </p> -<textarea rows="15" cols="60"> +<textarea rows="11" cols="60"> module(load="imjournal" PersistStateInterval="100" StateFile="/path/to/file") #load imjournal module module(load="mmjsonparse") #load mmjsonparse module for structured logs diff --git a/doc/imptcp.html b/doc/imptcp.html index b5bd0970..3efcce27 100644 --- a/doc/imptcp.html +++ b/doc/imptcp.html @@ -99,6 +99,11 @@ of seconds (5 recommended) to activate rate-limiting. </li> <li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting burst in number of messages. Default is 10,000. +<li><b>compression.mode</b><i>mode</i><br> +<i>mode</i> is one of "none" or "stream:always". +It is the counterpart to the compression modes set in +<a href="omfile.html">omfile</a>. +Please see it's documentation for details. </li> </ul> <b>Caveats/Known Bugs:</b> diff --git a/doc/imrelp.html b/doc/imrelp.html index 9f3e4875..73af2659 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -28,12 +28,50 @@ nits outlined above, is a much more reliable solution than plain tcp syslog and so it is highly suggested to use RELP instead of plain tcp. Clients send messages to the RELP server via omrelp.</p> -<p><b>Configuration Directives</b>:</p> +<p><b>Module Parameters</b>:</p> +<ul> + <li><b>Ruleset</b> <name></br> + Binds the specified ruleset to <b>all</b> RELP listeners. +</ul> +<p><b>Input Parameters</b>:</p> <ul> -<li><b>Ruleset</b> <name></br> -Binds the specified ruleset to all RELP listeners. <li><b>Port</b> <port><br> Starts a RELP server on selected port</li> +<li><b>tls</b> (not mandatory, values "on","off", default "off")<br> +If set to "on", the RELP connection will be encrypted by TLS, +so that the data is protected against observers. Please note +that both the client and the server must have set TLS to +either "on" or "off". Other combinations lead to unpredictable +results. +</li> +<li><b>tls.compression</b> (not mandatory, values "on","off", default "off")<br> +The controls if the TLS stream should be compressed (zipped). While this +increases CPU use, the network bandwidth should be reduced. Note that +typical text-based log records usually compress rather well. +</li> +<li><b>tls.dhbits</b> (not mandatory, integer)<br> +This setting controls how many bits are used for Diffie-Hellman key +generation. If not set, the librelp default is used. For secrity +reasons, at least 1024 bits should be used. Please note that the number +of bits must be supported by GnuTLS. If an invalid number is given, rsyslog +will report an error when the listener is started. We do this to be transparent +to changes/upgrades in GnuTLS (to check at config processing time, we would need +to hardcode the supported bits and keep them in sync with GnuTLS - this is +even impossible when custom GnuTLS changes are made...). +</li> +<li><b>tls.prioritystring</b> (not mandatory, string)<br> +This parameter permits to specify the so-called "priority string" to +GnuTLS. This string gives complete control over all crypto parameters, +including compression setting. For this reason, when the prioritystring +is specified, the "tls.compression" parameter has no effect and is +ignored. +<br>Full information about how to construct a priority string can be +found in the GnuTLS manual. At the time of this writing, this +information was contained in +<a href="http://gnutls.org/manual/html_node/Priority-Strings.html">section 6.10 of the GnuTLS manual</a>. +<br><b>Note: this is an expert parameter.</b> Do not use if you do +not exactly know what you are doing. +</li> </ul> <b>Caveats/Known Bugs:</b> <ul> @@ -47,7 +85,7 @@ not specific ones. This is due to a currently existing limitation in librelp. <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> </p> -<textarea rows="15" cols="60">module(load="imrelp") # needs to be done just once +<textarea rows="5" cols="60">module(load="imrelp") # needs to be done just once input(type="imrelp" port="20514") </textarea> @@ -60,7 +98,6 @@ equivalent to: Port</li> </ul> <b>Caveats/Known Bugs:</b> <ul> -<li>see description</li> <li>To obtain the remote system's IP address, you need to have at least librelp 1.0.0 installed. Versions below it return the hostname instead of the IP address.</li> @@ -70,14 +107,14 @@ not specific ones. This is due to a currently existing limitation in librelp. <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> </p> -<textarea rows="15" cols="60">$ModLoad imrelp # needs to be done just once +<textarea rows="5" cols="60">$ModLoad imrelp # needs to be done just once $InputRELPServerRun 20514 </textarea> <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/manual.html b/doc/manual.html index dea0b82d..8cb519a5 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ professional services</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 7.4.0 (v7.4-stable branch) of rsyslog.</b> +<p><b>This documentation is for version 7.5.0 (devel branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might diff --git a/doc/mmcount.html b/doc/mmcount.html new file mode 100644 index 00000000..1d06340d --- /dev/null +++ b/doc/mmcount.html @@ -0,0 +1,58 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>mmcount</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>mmcount</h1> +<p><b>Module Name: mmcount</b></p> +<p><b>Author: </b>Bala.FA <barumuga@redhat.com></p> +<p><b>Status: </b>Non project-supported module - contact author +or rsyslog mailing list for questions +<p><b>Available since</b>: 7.5.0</p> +<p><b>Description</b>:</p> +<p> +<pre> + mmcount: message modification plugin which counts messages + + This module provides the capability to count log messages by severity + or json property of given app-name. The count value is added into the + log message as json property named 'mmcount' + + Example usage of the module in the configuration file + + module(load="mmcount") + + # count each severity of appname gluster + action(type="mmcount" appname="gluster") + + # count each value of gf_code of appname gluster + action(type="mmcount" appname="glusterd" key="!gf_code") + + # count value 9999 of gf_code of appname gluster + action(type="mmcount" appname="glusterfsd" key="!gf_code" value="9999") + + # send email for every 50th mmcount + if $app-name == 'glusterfsd' and $!mmcount <> 0 and $!mmcount % 50 == 0 then { + $ActionMailSMTPServer smtp.example.com + $ActionMailFrom rsyslog@example.com + $ActionMailTo glusteradmin@example.com + $template mailSubject,"50th message of gf_code=9999 on %hostname%" + $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" + $ActionMailSubject mailSubject + $ActionExecOnlyOnceEveryInterval 30 + :ommail:;RSYSLOG_SyslogProtocol23Format + } +</pre> + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omfile.html b/doc/omfile.html index a5c3bb3b..438d694c 100644 --- a/doc/omfile.html +++ b/doc/omfile.html @@ -111,8 +111,12 @@ File="/var/log/messages") <p><b>Legacy Configuration Directives</b>:</p> <ul> - <li><strong>$DynaFileCacheSize </strong>(not mandatory, default will be used)<br> - Defines a template to be used for the output. <br></li><br> + <li><strong>$DynaFileCacheSize </strong>(not mandatory, default 10)<br> + defines the maximum size of the dynafile cache for <b>this</b> + action. Cache size greatly affects performance and should be + set so that it matches the actual need. Note that files + inside the cache are kept open until either rsyslogd + is HUPed or the file is evicted from the cache.<br></li><br> <li><strong>$OMFileZipLevel </strong>0..9 [default 0]<br> if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> diff --git a/doc/omfwd.html b/doc/omfwd.html index 53f9e527..51aa58b5 100644 --- a/doc/omfwd.html +++ b/doc/omfwd.html @@ -35,7 +35,38 @@ Framing-Mode to be for forwarding. This affects only TCP-based protocols. It is ignored for UDP. In protocol engineering, ``framing'' means how multiple messages over the same connection are separated. Usually, this is transparent to users. Unfortunately, the early syslog protocol evolved, and so there are cases where users need to specify the framing. The traditional framing is nontransparent. With it, messages are end when a LF (aka ``line break'', ``return'') is encountered, and the next message starts immediately after the LF. If multi-line messages are received, these are essentially broken up into multiple message, usually with all but the first message segment being incorrectly formatted. The octet-counting framing solves this issue. With it, each message is prefixed with the actual message length, so that a receivers knows exactly where the message ends. Multi-line messages cause no problem here. This mode is very close to the method described in RFC5425 for TLS-enabled syslog. Unfortunately, only few syslogd implementations support octet-counted framing. As such, the traditional framing is set as default, even though it has defects. If it is known that the receiver supports octet-counted framing, it is suggested to use that framing mode. <br></li><br> <li><strong>ZipLevel </strong>0..9 [default 0]<br> - Compression level for messages. Rsyslog implements a proprietary capability to zip transmitted messages. Note that compression happens on a message-per-message basis. As such, there is a performance gain only for larger messages. Before compressing a message, rsyslog checks if there is some gain by compression. If so, the message is sent compressed. If not, it is sent uncompressed. As such, it is totally valid that compressed and uncompressed messages are intermixed within a conversation. <br>The compression level is specified via the usual factor of 0 to 9, with 9 being the strongest compression (taking up most processing time) and 0 being no compression at all (taking up no extra processing time). <br></li><br> + Compression level for messages. + <br>Up until rsyslog 7.5.1, this was the only compression setting that + rsyslog understood. Starting with 7.5.1, we have different compression + modes. All of them are affected by the ziplevel. If, however, no mode + is explicitely set, setting ziplevel also turns on "single" + compression mode, so pre 7.5.1 configuration will continue to work + as expected. + <br>The compression level is specified via the usual factor of 0 to 9, with 9 being the strongest compression (taking up most processing time) and 0 being no compression at all (taking up no extra processing time). <br></li><br> + <li><b>compression.mode</b><i>mode</i><br> + <i>mode</i> is one of "none", "single", or "stream:always". The + default is "none", in which no compression happens at all. + <br>In "single" compression mode, Rsyslog implements a proprietary + capability to zip transmitted messages. That compression happens + on a message-per-message basis. As such, there is a performance gain + only for larger messages. Before compressing a message, rsyslog checks + if there is some gain by compression. If so, the message is sent + compressed. If not, it is sent uncompressed. As such, it is totally + valid that compressed and uncompressed messages are intermixed + within a conversation. + <br>In "stream:always" compression mode the full stream is being + compressed. This also uses non-standard protocol and is compatible + only with receives that have the same abilities. This mode offers + potentially very high compression ratios. With typical syslog + messages, it can be as high as 95+% compression (so only one twentieth + of data is actually transmitted!). Note that this mode introduces + extra latency, as data is only sent when the compressor emits new + compressed data. For typical syslog messages, this can mean that + some hundered messages may be held in local buffers before they are + actually sent. This mode has been introduced in 7.5.1. + <br><b>Note: currently only imptcp supports receiving stream-compressed + data.</b> + <br></li><br> <li><strong>RebindInterval </strong>integer<br> Permits to specify an interval at which the current connection is broken and re-established. This setting is primarily an aid to load balancers. After the configured number of messages has been transmitted, the current connection is terminated and a new one started. Note that this setting applies to both TCP and UDP traffic. For UDP, the new ``connection'' uses a different source port (ports are cycled and not reused too frequently). This usually is perceived as a ``new connection'' by load balancers, which in turn forward messages to another physical target system. <br></li><br> diff --git a/doc/omjournal.html b/doc/omjournal.html index c42d9841..6124e40c 100644 --- a/doc/omjournal.html +++ b/doc/omjournal.html @@ -19,6 +19,9 @@ and processed by its tools. <p>A typical use case we had on our mind is a SOHO environment, where the user wants to include syslog data obtained from the local router to be part of the journal data. +<p>We suggest to check out our short presentation on +<a href="http://youtu.be/GTS7EuSdFKE">rsyslog journal integration</a> to +learn more details of anticipated use cases. <p> </p> <p><b>Module Configuration Parameters</b>:</p> diff --git a/doc/omrelp.html b/doc/omrelp.html index 8858f884..a44ec319 100644 --- a/doc/omrelp.html +++ b/doc/omrelp.html @@ -24,16 +24,37 @@ implementation).</p> rsyslog 7.3.10. For older versions, legacy configuration directives must be used. <ul> - <li><b>target </b>(mandatory)<br> + <li><b>target</b> (mandatory)<br> The target server to connect to. </li> - <li><b>template </b>(not mandatory, default "RSYSLOG_ForwardFormat")<br> + <li><b>template</b> (not mandatory, default "RSYSLOG_ForwardFormat")<br> Defines the template to be used for the output. </li> - <li><b>timeout </b>(not mandatory, default 90)<br> + <li><b>timeout</b> (not mandatory, default 90)<br> Timeout for relp sessions. If set too low, valid sessions may be considered dead and tried to recover. </li> + <li><b>tls</b> (not mandatory, values "on","off", default "off")<br> + If set to "on", the RELP connection will be encrypted by TLS, so that the data is protected against observers. Please note that both the client and the server must have set TLS to either "on" or "off". Other combinations lead to unpredictable results. + </li> + <li><b>tls.compression</b> (not mandatory, values "on","off", default "off")<br> + The controls if the TLS stream should be compressed (zipped). While this + increases CPU use, the network bandwidth should be reduced. Note that + typical text-based log records usually compress rather well. + </li> + <li><b>tls.prioritystring</b> (not mandatory, string)<br> + This parameter permits to specify the so-called "priority string" to + GnuTLS. This string gives complete control over all crypto parameters, + including compression setting. For this reason, when the prioritystring + is specified, the "tls.compression" parameter has no effect and is + ignored. + <br>Full information about how to construct a priority string can be + found in the GnuTLS manual. At the time of this writing, this + information was contained in + <a href="http://gnutls.org/manual/html_node/Priority-Strings.html">section 6.10 of the GnuTLS manual</a>. + <br><b>Note: this is an expert parameter.</b> Do not use if you do + not exactly know what you are doing. + </li> </ul> <p><b>Sample:</b></p> <p>The following sample sends all messages to the central server diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 18d6b8a1..c8c1c5d7 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -111,6 +111,7 @@ probably an excellent starting base for writing a new module. Currently, the fol modules exist inside the source tree: <ul> <li><a href="mmanon.html">mmanon</a> - used to anonymize log messages. +<li><a href="mmcount.html">mmcount</a> - message modification plugin which counts messages <li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages. Note that this actually is a <b>generic</b> module. <li><a href="mmjsonparse.html">mmjsonparse</a> - used to interpret CEE/lumberjack |