diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/Makefile.am | 5 | ||||
| -rw-r--r-- | doc/debug.html | 144 | ||||
| -rw-r--r-- | doc/imklog.html | 19 | ||||
| -rw-r--r-- | doc/imrelp.html | 3 | ||||
| -rw-r--r-- | doc/imtcp.html | 14 | ||||
| -rw-r--r-- | doc/man_rsyslogd.html | 438 | ||||
| -rw-r--r-- | doc/manual.html | 22 | ||||
| -rw-r--r-- | doc/multi_ruleset.html | 275 | ||||
| -rw-r--r-- | doc/omoracle.html | 122 | ||||
| -rw-r--r-- | doc/professional_support.html | 112 | ||||
| -rw-r--r-- | doc/property_replacer.html | 13 | ||||
| -rw-r--r-- | doc/queues.html | 6 | ||||
| -rw-r--r-- | doc/rsconf1_markmessageperiod.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_actions.html | 21 | ||||
| -rw-r--r-- | doc/rsyslog_conf_filter.html | 6 | ||||
| -rw-r--r-- | doc/rsyslog_conf_global.html | 51 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 63 | ||||
| -rw-r--r-- | doc/rsyslog_conf_templates.html | 3 | ||||
| -rw-r--r-- | doc/rsyslog_pgsql.html | 336 | ||||
| -rw-r--r-- | doc/rsyslog_php_syslog_ng.html | 16 | ||||
| -rw-r--r-- | doc/src/rsyslog_pgsql.odt | bin | 0 -> 41755 bytes | |||
| -rw-r--r-- | doc/tls_cert_machine.html | 12 | ||||
| -rw-r--r-- | doc/v4compatibility.html | 77 |
23 files changed, 1121 insertions, 639 deletions
diff --git a/doc/Makefile.am b/doc/Makefile.am index 0703b8fc..ca2ee71c 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -13,13 +13,13 @@ html_files = \ ipv6.html \ log_rotation_fix_size.html \ manual.html \ - man_rsyslogd.html \ modules.html \ property_replacer.html \ rsyslog_ng_comparison.html \ rsyslog_conf.html \ rsyslog-example.conf \ rsyslog_mysql.html \ + rsyslog_pgsql.html \ rsyslog_packages.html \ rsyslog_high_database_rate.html \ rsyslog_php_syslog_ng.html \ @@ -41,7 +41,6 @@ html_files = \ imrelp.html \ imuxsock.html \ imklog.html \ - professional_support.html \ queues.html \ src/queueWorkerLogic.dia \ queueWorkerLogic.jpg \ @@ -91,6 +90,7 @@ html_files = \ rsconf1_resetconfigvariables.html \ rsconf1_umask.html \ v3compatibility.html \ + v4compatibility.html \ im3195.html \ netstream.html \ ns_gtls.html \ @@ -111,6 +111,7 @@ html_files = \ rsyslog_conf_templates.html \ rsyslog_conf_nomatch.html \ queues_analogy.html \ + multi_ruleset.html \ src/classes.dia grfx_files = \ diff --git a/doc/debug.html b/doc/debug.html index de77f04a..46759986 100644 --- a/doc/debug.html +++ b/doc/debug.html @@ -1,41 +1,145 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>Debug Support</title></head> +<html> +<head> +<meta http-equiv="Content-Language" content="en"> +<title>Rsyslog Debug Support</title></head> <body> -<h1>Debug Support</h1> +<h1>Rsyslog Debug Support</h1> <p> -Rsyslog provides a number of debug aids. Some of them are activated by +Rsyslog provides a number of debug aides. Some of them are activated by adding the --enable-rtinst ./configure option ("rtinst" means runtime instrumentation). Turning debugging on obviously costs some performance (in some cases considerable). </p> <p>This is document is just being created and thus terse.</p> -<p style="font-weight: bold;">Signals supported</p> -<p>SIGUSR1 - turns debug messages on and off (expect this signal -to go away over time)</p> -<p>SIGUSR2 - outputs debug information (including active threads +<h2>Signals supported</h2> +<p><b>SIGUSR1</b> - turns debug messages on and off. Note that for this +signal to work, rsyslogd must be running with debugging enabled, either +via the -d command line switch or the environment options specified below. +It is <b>not</b> required that rsyslog was compiled with debugging enabled +(but depending on the settings this may lead to better debug info). +<p><b>SIGUSR2</b> - outputs debug information (including active threads and a call stack) for the state when SIGUSR2 was received. This is a -one-time output. Can be sent as often as the user likes.</p> -<p style="font-weight: bold;">Environment Variables</p> -<p>There are two environment variables that set several debug settings. The "RSYSLOG_DEBUGLOG" (sample: RSYSLOG_DEBUGLOG="/path/to/debuglog/") +one-time output. Can be sent as often as the user likes. +<p><b>Note:</b> this signal <b>may go away</b> in later releases and may +be replaced by something else.</p> +<h2>Environment Variables</h2> +<p>There are two environment variables that set several debug settings: +<ul> +<li>The "RSYSLOG_DEBUGLOG" (sample: RSYSLOG_DEBUGLOG="/path/to/debuglog/") writes (allmost) all debug message to the specified log file in addition to stdout. Some system messages (e.g. segfault or abort message) are not written to the -file as we can not capture them. Runtime debug support is controlled by -"RSYSLOG_DEBUG". It contains an option string with the following -options possible (all are case insensitive):</p><ul><li><span style="font-weight: bold;">LogFuncFlow</span> - print out the logical flow of functions (entering and exiting them)</li><li><span style="font-weight: bold;">FileTrace</span> - specifies which files to trace LogFuncFlow. If <span style="font-weight: bold;">not</span> +file as we can not capture them. +<li>Runtime debug support is controlled by "RSYSLOG_DEBUG". +<p>The "RSYSLOG_DEBUG" environment variable contains an option string with the following +options possible (all are case insensitive):</p> +<ul> +<li><b>LogFuncFlow</b> - print out the logical flow of functions (entering and exiting them)</li> +<li><b>FileTrace</b> - specifies which files to trace LogFuncFlow. If <b>not</b> set (the default), a LogFuncFlow trace is provided for all files. Set to limit it to the files specified. FileTrace may be specified multiple times, one file each (e.g. export RSYSLOG_DEBUG="LogFuncFlow -FileTrace=vm.c FileTrace=expr.c"</li><li><span style="font-weight: bold;">PrintFuncDB</span> - print the content of the debug function database whenever debug information is printed (e.g. abort case)!</li><li><span style="font-weight: bold;">PrintAllDebugInfoOnExit</span> - print all debug information immediately before rsyslogd exits (<span style="font-weight: bold; font-style: italic;">currently not implemented!</span>)</li><li><span style="font-weight: bold;">PrintMutexAction</span> - print mutex action as it happens. Useful for finding deadlocks and such.</li><li><span style="font-weight: bold;">NoLogTimeStamp</span> - do not prefix log lines with a timestamp (default is to do that).</li><li><span style="font-weight: bold;">NoStdOut</span> - do not emit debug messages to stdout. If RSYSLOG_DEBUGLOG is not set, this means no messages will be displayed at all.</li><li><span style="font-weight: bold;">help</span> - display a very short list of commands - hopefully a life saver if you can't access the documentation...</li></ul> +FileTrace=vm.c FileTrace=expr.c"</li> +<li><b>PrintFuncDB</b> - print the content of the debug function database whenever debug information is printed (e.g. abort case)!</li> +<li><b>PrintAllDebugInfoOnExit</b> - print all debug information immediately before rsyslogd exits (<span style="font-weight: bold; font-style: italic;">currently not implemented!</span>)</li> +<li><b>PrintMutexAction</b> - print mutex action as it happens. Useful for finding deadlocks and such.</li> +<li><b>NoLogTimeStamp</b> - do not prefix log lines with a timestamp (default is to do that).</li> +<li><b>NoStdOut</b> - do not emit debug messages to stdout. If RSYSLOG_DEBUGLOG is not set, this means no messages will be displayed at all.</li> +<li><b>Debug</b> - if present, turns on the debug system and enables debug output +<li><b>DebugOnDemand</b> - if present, turns on the debug system but does not enable +debug output itself. You need to send SIGUSR1 to turn it on when desired. +<li><b>help</b> - display a very short list of commands - hopefully a life saver if you can't access the documentation...</li> +</ul> +</ul> +<h3>Why Environment Variables?</h3> +<p>You may ask why we use environment variables for debug-system parameters and not +the usual rsyslog.conf configuration commands. After all, environment variables force one +to change distro-specific configuration files, whereas regular configuration directives +would fit nicely into the one central rsyslog.conf. +<p>The problem here is that many settings of the debug system must be initialized +before the full rsyslog engine starts up. At that point, there is no such thing like +rsyslog.conf or the objects needed to process it present in an running instance. +And even if we would enable to change settings some time later, that would mean that +we can not correctly monitor (and debug) the initial startup phase of rsyslogd. What +makes matters worse is that during this startup phase (and never again later!) some +of the base debug structure needs to be created, at least if the build is +configured for that (many of these things only happen in --enable-rtinst mode). So +if we do not initialize the debug system <b>before</b> actually startig up the +rsyslog core, we get a number of data structures wrong. +<p>For these reasons, we utilize environment variables to initialize and configure +the debugging system. We understand this may be somewhat painful, but now you know +there are at least some good reasons for doing so. +<h2>Getting debug information from a running Instance</h2> +<p>It is possible to obtain debugging information from a running instance, but this requires +some setup. We assume that the instance runs in the background, so debug output to +stdout is not desired. As such, all debug information needs to go into a log file. +<p>To create this setup, you need to <ul> +<li>point the RSYSLOG_DEBUGLOG environment variable to a file that is accessible +during the while runtime (we strongly suggest a file in the local file system!) +<li>set RSYSLOG_DEBUG at least to "DebugOnDeman NoStdOut" +<li>make sure these environment variables are set in the correct (distro-specifc) +startup script if you do not run rsyslogd interactively </ul> +<p>These settings enable the capability to react to SIGUSR1. The signal will toggle +debug status when received. So send it one to turn debug loggin on, and send it again +to turn debug logging off again. The third time it will be turned on again ... and so on. +<p>On a typical system, you can signal rsyslogd as follows: +<pre> +kill -USR1 `cat /var/run/rsyslogd.pid` +</pre> +Important: there are backticks around the "cat"-command. If you use the regular +quote it won't work. The debug log will show whether debug logging has been turned +on or off. There is no other indication of the status. +<p>Note: running with DebugOnDemand by itself does in practice not have any performance +toll. However, switching debug logging on has a severe performance toll. Also, debug +logging synchronizes much of the code, removing a lot of concurrency and thus +potential race conditions. As such, the very same running instance may behave +very differently with debug logging turned on vs. off. The on-demand debug log +functionality is considered to be very valuable to analyze hard-to-find bugs that +only manifest after a long runtime. Turning debug logging on a failing instance +may reveal the cause of the failure. However, depending on the failure, debug logging +may not even be successfully be turned on. Also note that with this rsyslog version we cannot +obtain any debug information on events that happened <i>before</i> debug logging was +turned on. +<p>If an instance hangs, it is possible to obtain some useful information about the current +threads and their calling stack by sending SIGUSR2. However, the usefulness of that +information is very much depending on rsyslog compile-time settings, must importantly +the --enable-rtinst configure flag. Note that activating this option causes additional overhead +and slows down rsyslgod considerable. So if you do that, you need to check if it is +capable to handle the workload. Also, threading behavior is modified by the +runtime instrumentation. +<p>Sending SIGUSR2 writes new process state information to the log file each time +it is sent. So it may be useful to do that from time to time. It probably is most +useful if the process seems to hang, in which case it may (may!) be able to output +some diagnostic information on the current processing state. In that case, turning +on the mutex debugging options (see above) is probably useful. +<h2>Interpreting the Logs</h2> +<p>Debug logs are primarily meant for rsyslog developers. But they may still provide valuable +information to users. Just be warned that logs sometimes contains informaton the looks like +an error, but actually is none. We put a lot of extra information into the logs, and there +are some cases where it is OK for an error to happen, we just wanted to record it inside +the log. The code handles many cases automatically. So, in short, the log may not make sense to +you, but it (hopefully) makes sense to a developer. Note that we developers often need +many lines of the log file, it is relatively rare that a problem can be diagnosed by +looking at just a couple of (hundered) log records. +<h2>Security Risks</h2> +<p>The debug log will reveal potentially sensible information, including user accounts and +passwords, to anyone able to read the log file. As such, it is recommended to properly +guard access to the log file. Also, an instance running with debug log enabled runs much +slower than one without. An attacker may use this to place carry out a denial-of-service +attack or try to hide some information from the log file. As such, it is suggested to +enable DebugOnDemand mode only for a reason. Note that when no debug mode is enabled, +SIGUSR1 and SIGUSR2 are completely ignored. +<p>When running in any of the debug modes (including on demand mode), an interactive +instance of rsyslogd can be aborted by pressing ctl-c. +<p> <p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> -</body></html>
\ No newline at end of file +</body> +</html> diff --git a/doc/imklog.html b/doc/imklog.html index 9166bae6..5bfab5ce 100644 --- a/doc/imklog.html +++ b/doc/imklog.html @@ -34,9 +34,9 @@ handled. The default is "off", in which case these messages are ignored. Switch it to on to submit non-kernel messages to rsyslog processing.<span style="font-weight: bold;"></span></li> <li><span style="font-weight: bold;"></span>$DebugPrintKernelSymbols -(imklog) [on/<b>off</b>]<br> +[on/<b>off</b>]<br> Linux only, ignored on other platforms (but may be specified)</li> -<li>$klogSymbolLookup (imklog) [on/<b>off</b>] -- +<li>$klogSymbolLookup [on/<b>off</b>] -- disables imklog kernel symbol translation (former klogd -x option). NOTE that this option is counter-productive on recent kernels (>= 2.6) because the kernel already does the symbol translation and this option breaks the information.<br> @@ -44,10 +44,19 @@ kernel already does the symbol translation and this option breaks the informatio it except if you have a very good reason. If you have one, let us know because otherwise new versions will no longer support it.<br> Linux only, ignored on other platforms (but may be specified)</li> -<li>$klogUseSyscallInterface (imklog) [on/<b>off</b>] +<li><b>$klogConsoleLogLevel</b> [<i>number</i>] +(former klogd -c option) -- sets the console log level. If specified, only messages with +up to the specified level are printed to the console. The default is -1, which means that +the current settings are not modified. To get this behavior, do not specify +$klogConsoleLogLevel in the configuration file. Note that this is a global parameter. Each time +it is changed, the previous definition is re-set. The one activate will be that one that is +active when imklog actually starts processing. In short words: do not specify this +directive more than once! +<br><b>Linux only</b>, ignored on other platforms (but may be specified)</li> +<li><b>$klogUseSyscallInterface</b> [on/<b>off</b>] -- former klogd -s option<br> Linux only, ignored on other platforms (but may be specified)</li> -<li>$klogSymbolsTwice (imklog) [on/<b>off</b>] -- +<li>$klogSymbolsTwice [on/<b>off</b>] -- former klogd -2 option<br> Linux only, ignored on other platforms (but may be specified)<br style="font-weight: bold;"> </li> @@ -69,7 +78,7 @@ is needed to start pulling kernel messages.<br> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer +Copyright © 2008-2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/imrelp.html b/doc/imrelp.html index 53826ac2..2cf9c1f7 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -35,6 +35,9 @@ Starts a RELP server on selected port</li> <b>Caveats/Known Bugs:</b> <ul> <li>see description</li> +<li>To obtain the remote system's IP address, you need to have at least +librelp 1.0.0 installed. Versions below it return the hostname instead +of the IP address.</li> </ul> <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> diff --git a/doc/imtcp.html b/doc/imtcp.html index 9ea7efa1..0ccdecc7 100644 --- a/doc/imtcp.html +++ b/doc/imtcp.html @@ -41,17 +41,25 @@ very limited interest in fixing this issue. This directive <b>can not</b> fix th That would require much more code changes, which I was unable to do so far. Full details can be found at the <a href="http://www.rsyslog.com/Article321.phtml">Cisco tcp syslog anomaly</a> page. +<li>$InputTCPServerNotifyOnConnectionClose [on/<b>off</b>] (available since 4.5.5)<br> +instructs imtcp to emit a message if the remote peer closes a connection.<br> +<b>Important:</b> This directive is global to all listeners and must be given right +after loading imtcp, otherwise it may have no effect.</li> <li>$InputTCPServerRun <port><br> Starts a TCP server on selected port</li> -<li><ul><li>$InputTCPMaxSessions <number></li></ul> -Sets the maximum number of sessions supported</li><li>$InputTCPServerStreamDriverMode <number><br> +<li>$InputTCPMaxListeners <number><br> +Sets the maximum number of listeners (server ports) supported. Default is 20. This must be set before the first $InputTCPServerRun directive.</li> +<li>$InputTCPMaxSessions <number><br> +Sets the maximum number of sessions supported. Default is 200. This must be set before the first $InputTCPServerRun directive</li> +<li>$InputTCPServerStreamDriverMode <number><br> Sets the driver mode for the currently selected <a href="netstream.html">network stream driver</a>. <number> is driver specifc.</li> <li>$InputTCPServerInputName <name><br> Sets a name for the inputname property. If no name is set "imtcp" is used by default. Setting a name is not strictly necessary, but can be useful to apply filtering based on which input the message was received from. <li>$InputTCPServerStreamDriverAuthMode <mode-string><br> -Sets the authentication mode for the currently selected <a href="netstream.html">network stream driver</a>. <mode-string> is driver specifc.</li><li>$InputTCPServerStreamDriverPermittedPeer <id-string><br> +Sets the authentication mode for the currently selected <a href="netstream.html">network stream driver</a>. <mode-string> is driver specifc.</li> +<li>$InputTCPServerStreamDriverPermittedPeer <id-string><br> Sets permitted peer IDs. Only these peers are able to connect to the listener. <id-string> semantics depend on the currently selected AuthMode and <a href="netstream.html">network stream driver</a>. PermittedPeers may not be set in anonymous modes.</li> diff --git a/doc/man_rsyslogd.html b/doc/man_rsyslogd.html deleted file mode 100644 index d18fd88a..00000000 --- a/doc/man_rsyslogd.html +++ /dev/null @@ -1,438 +0,0 @@ -<BODY><PRE> -RSYSLOGD(8) Linux System Administration RSYSLOGD(8) - - - -<B>NAME</B> - rsyslogd - reliable and extended syslogd - -<B>SYNOPSIS</B> - <B>rsyslogd </B>[ <B>-4 </B>] [ <B>-6 </B>] [ <B>-A </B>] [ <B>-a </B><I>socket </I>] [ <B>-d </B>] [ <B>-e </B>] - [ <B>-f </B><I>config file </I>] [ <B>-h </B>] [ <B>-i </B><I>pid file </I>] [ <B>-l </B><I>hostlist </I>] - [ <B>-m </B><I>interval </I>] [ <B>-n </B>] [ <B>-o </B>] [ <B>-p </B><I>socket </I>] - [ <B>-r </B><I>[port] </I>] [ <B>-s </B><I>domainlist </I>] [ <B>-t </B><I>port,max-nbr-of-sessions </I>] - [ <B>-v </B>] [ <B>-w </B>] [ <B>-x </B>] - - -<B>DESCRIPTION</B> - <B>Rsyslogd </B>is a system utility providing support for message logging. - Support of both internet and unix domain sockets enables this utility - to support both local and remote logging (via UDP and TCP). - - <B>Rsyslogd</B>(8) is derived from the sysklogd package which in turn is - derived from the stock BSD sources. - - <B>Rsyslogd </B>provides a kind of logging that many modern programs use. - Every logged message contains at least a time and a hostname field, - normally a program name field, too, but that depends on how trusty the - logging program is. The rsyslog package supports free definition of - output formats via templates. It also supports precise timestamps and - writing directly to MySQL databases. If the database option is used, - tools like phpLogCon can be used to view the log data. - - While the <B>rsyslogd </B>sources have been heavily modified a couple of notes - are in order. First of all there has been a systematic attempt to - insure that rsyslogd follows its default, standard BSD behavior. Of - course, some configuration file changes are necessary in order to sup- - port the template system. However, rsyslogd should be able to use a - standard syslog.conf and act like the original syslogd. However, an - original syslogd will not work correctly with a rsyslog-enhanced con- - figuration file. At best, it will generate funny looking file names. - The second important concept to note is that this version of rsyslogd - interacts transparently with the version of syslog found in the stan- - dard libraries. If a binary linked to the standard shared libraries - fails to function correctly we would like an example of the anomalous - behavior. - - The main configuration file <I>/etc/rsyslog.conf </I>or an alternative file, - given with the <B>-f </B>option, is read at startup. Any lines that begin - with the hash mark (‘‘#’’) and empty lines are ignored. If an error - occurs during parsing the error element is ignored. It is tried to - parse the rest of the line. - - For details and configuration examples, see the <B>rsyslog.conf (5) </B>man - page. - - - -<B>OPTIONS</B> - <B>-A </B>When sending UDP messages, there are potentially multiple paths - to the target destination. By default, <B>rsyslogd </B>only sends to - the first target it can successfully send to. If -A is given, - messages are sent to all targets. This may improve reliability, - but may also cause message duplication. This option should - enabled only if it is fully understood. - - <B>-4 </B>Causes <B>rsyslogd </B>to listen to IPv4 addresses only. If neither -4 - nor -6 is given, <B>rsyslogd </B>listens to all configured addresses of - the system. - - <B>-6 </B>Causes <B>rsyslogd </B>to listen to IPv6 addresses only. If neither -4 - nor -6 is given, <B>rsyslogd </B>listens to all configured addresses of - the system. - - <B>-a </B><I>socket</I> - Using this argument you can specify additional sockets from that - <B>rsyslogd </B>has to listen to. This is needed if you’re going to - let some daemon run within a chroot() environment. You can use - up to 19 additional sockets. If your environment needs even - more, you have to increase the symbol <B>MAXFUNIX </B>within the sys- - logd.c source file. An example for a chroot() daemon is - described by the people from OpenBSD at - http://www.psionic.com/papers/dns.html. - - <B>-d </B>Turns on debug mode. Using this the daemon will not proceed a - <B>fork</B>(2) to set itself in the background, but opposite to that - stay in the foreground and write much debug information on the - current tty. See the DEBUGGING section for more information. - - <B>-e </B>Set the default of $RepeatedMsgReduction config option to "off". - Hine: "e" like "every message". For further information, see - there. - - <B>-f </B><I>config file</I> - Specify an alternative configuration file instead of <I>/etc/rsys-</I> - <I>log.conf</I>, which is the default. - - <B>-h </B>By default rsyslogd will not forward messages it receives from - remote hosts. Specifying this switch on the command line will - cause the log daemon to forward any remote messages it receives - to forwarding hosts which have been defined. - - <B>-i </B><I>pid file</I> - Specify an alternative pid file instead of the default one. - This option must be used if multiple instances of rsyslogd - should run on a single machine. - - <B>-l </B><I>hostlist</I> - Specify a hostname that should be logged only with its simple - hostname and not the fqdn. Multiple hosts may be specified - using the colon (‘‘:’’) separator. - - <B>-m </B><I>interval</I> - The <B>rsyslogd </B>logs a mark timestamp regularly. The default - <I>interval </I>between two <I>-- MARK -- </I>lines is 20 minutes. This can - be changed with this option. Setting the <I>interval </I>to zero turns - it off entirely. - - <B>-n </B>Avoid auto-backgrounding. This is needed especially if the - <B>rsyslogd </B>is started and controlled by <B>init</B>(8). - - <B>-o </B>Omit reading the standard local log socket. This option is most - useful for running multiple instances of rsyslogd on a single - machine. When specified, no local log socket is opened at all. - - <B>-p </B><I>socket</I> - You can specify an alternative unix domain socket instead of - <I>/dev/log</I>. - - <B>-r </B><I>["port"]</I> - Activates the syslog/udp listener service. The listener will - listen to the specified port. If no port is specified, 0 is - used as port number, which in turn will lead to a lookup of the - system default syslog port. If there is no system default, 514 - is used. Please note that the port must immediately follow the - -r option. Thus "-r514" is valid while "-r 514" is invalid (note - the space). - - <B>-s </B><I>domainlist</I> - Specify a domainname that should be stripped off before logging. - Multiple domains may be specified using the colon (‘‘:’’) sepa- - rator. Please be advised that no sub-domains may be specified - but only entire domains. For example if <B>-s north.de </B>is speci- - fied and the host logging resolves to satu.infodrom.north.de no - domain would be cut, you will have to specify two domains like: - <B>-s north.de:infodrom.north.de</B>. - - <B>-t </B><I>port,max-nbr-of-sessions</I> - Activates the syslog/tcp listener service. The listener will - listen to the specified port. If max-nbr-of-sessions is speci- - fied, that becomes the maximum number of concurrent tcp ses- - sions. If not specified, the default is 200. Please note that - syslog/tcp is not standardized, but the implementation in rsys- - logd follows common practice and is compatible with e.g. Cisco - PIX, syslog-ng and MonitorWare (Windows). Please note that the - port must immediately follow the -t option. Thus "-t514" is - valid while "-t 514" is invalid (note the space). - - <B>-v </B>Print version and exit. - - <B>-w </B>Supress warnings issued when messages are received from non- - authorized machines (those, that are in no AllowedSender list). - - <B>-x </B>Disable DNS for remote messages. - - -<B>SIGNALS</B> - <B>Rsyslogd </B>reacts to a set of signals. You may easily send a signal to - <B>rsyslogd </B>using the following: - - kill -SIGNAL ‘cat /var/run/rsyslogd.pid‘ - - - <B>SIGHUP </B>This lets <B>rsyslogd </B>perform a re-initialization. All open files - are closed, the configuration file (default is <I>/etc/rsys-</I> - <I>log.conf</I>) will be reread and the <B>rsyslog</B>(3) facility is started - again. - - <B>SIGTERM</B> - <B>Rsyslogd </B>will die. - - <B>SIGINT</B>, <B>SIGQUIT</B> - If debugging is enabled these are ignored, otherwise <B>rsyslogd</B> - will die. - - <B>SIGUSR1</B> - Switch debugging on/off. This option can only be used if <B>rsys-</B> - <B>logd </B>is started with the <B>-d </B>debug option. - - <B>SIGCHLD</B> - Wait for childs if some were born, because of wall’ing messages. - - -<B>SUPPORT FOR REMOTE LOGGING</B> - <B>Rsyslogd </B>provides network support to the syslogd facility. Network - support means that messages can be forwarded from one node running - rsyslogd to another node running rsyslogd (or a compatible syslog - implementation) where they will be actually logged to a disk file. - - To enable this you have to specify either the <B>-r </B>or <B>-t </B>option on the - command line. The default behavior is that <B>rsyslogd </B>won’t listen to - the network. You can also combine these two options if you want rsys- - logd to listen to both TCP and UDP messages. - - The strategy is to have rsyslogd listen on a unix domain socket for - locally generated log messages. This behavior will allow rsyslogd to - inter-operate with the syslog found in the standard C library. At the - same time rsyslogd listens on the standard syslog port for messages - forwarded from other hosts. To have this work correctly the <B>ser-</B> - <B>vices</B>(5) files (typically found in <I>/etc</I>) must have the following entry: - - syslog 514/udp - - If this entry is missing <B>rsyslogd </B>will use the well known port of 514 - (so in most cases, it’s not really needed). - - To cause messages to be forwarded to another host replace the normal - file line in the <I>rsyslog.conf </I>file with the name of the host to which - the messages is to be sent prepended with an @ (for UDP delivery) or - the sequence @@ (for TCP delivery). The host name can also be followed - by a colon and a port number, in which case the message is sent to the - specified port on the remote host. - - For example, to forward <B>ALL </B>messages to a remote host use the - following <I>rsyslog.conf </I>entry: - - # Sample rsyslogd configuration file to - # messages to a remote host forward all. - *.* @hostname - More samples can be found in sample.conf. - - If the remote hostname cannot be resolved at startup, because - the name-server might not be accessible (it may be started after - rsyslogd) you don’t have to worry. <B>Rsyslogd </B>will retry to - resolve the name ten times and then complain. Another possibil- - ity to avoid this is to place the hostname in <I>/etc/hosts</I>. - - With normal <B>syslogd</B>s you would get syslog-loops if you send out - messages that were received from a remote host to the same host - (or more complicated to a third host that sends it back to the - first one, and so on). - - To avoid this no messages that were received from a remote host - are sent out to another (or the same) remote host. You can dis- - able this feature by the <B>-h </B>option. - - If the remote host is located in the same domain as the host, - <B>rsyslogd </B>is running on, only the simple hostname will be logged - instead of the whole fqdn. - - In a local network you may provide a central log server to have - all the important information kept on one machine. If the net- - work consists of different domains you don’t have to complain - about logging fully qualified names instead of simple hostnames. - You may want to use the strip-domain feature <B>-s </B>of this server. - You can tell <B>rsyslogd </B>to strip off several domains other than - the one the server is located in and only log simple hostnames. - - Using the <B>-l </B>option there’s also a possibility to define single - hosts as local machines. This, too, results in logging only - their simple hostnames and not the fqdns. - - -<B>OUTPUT TO DATABASES</B> - <B>Rsyslogd </B>has support for writing data to MySQL database tables. The - exact specifics are described in the <B>rsyslog.conf (5) </B>man page. Be sure - to read it if you plan to use database logging. - - While it is often handy to have the data in a database, you must be - aware of the implications. Most importantly, database logging takes far - longer than logging to a text file. A system that can handle a large - log volume when writing to text files can most likely not handle a sim- - ilar large volume when writing to a database table. - - -<B>OUTPUT TO NAMED PIPES (FIFOs)</B> - <B>Rsyslogd </B>has support for logging output to named pipes (fifos). A fifo - or named pipe can be used as a destination for log messages by prepend- - ing a pipy symbol (‘‘|’’) to the name of the file. This is handy for - debugging. Note that the fifo must be created with the mkfifo command - before <B>rsyslogd </B>is started. - - The following configuration file routes debug messages from the - kernel to a fifo: - - # Sample configuration to route kernel debugging - # messages ONLY to /usr/adm/debug which is a - # named pipe. - kern.=debug |/usr/adm/debug - - -<B>INSTALLATION CONCERNS</B> - There is probably one important consideration when installing rsyslogd. - It is dependent on proper formatting of messages by the syslog func- - tion. The functioning of the syslog function in the shared libraries - changed somewhere in the region of libc.so.4.[2-4].n. The specific - change was to null-terminate the message before transmitting it to the - <I>/dev/log </I>socket. Proper functioning of this version of rsyslogd is - dependent on null-termination of the message. - - This problem will typically manifest itself if old statically linked - binaries are being used on the system. Binaries using old versions of - the syslog function will cause empty lines to be logged followed by the - message with the first character in the message removed. Relinking - these binaries to newer versions of the shared libraries will correct - this problem. - - The <B>rsyslogd</B>(8) can be run from <B>init</B>(8) or started as part of the rc.* - sequence. If it is started from init the option <I>-n </I>must be set, other- - wise you’ll get tons of syslog daemons started. This is because - <B>init</B>(8) depends on the process ID. - - -<B>SECURITY THREATS</B> - There is the potential for the rsyslogd daemon to be used as a conduit - for a denial of service attack. A rogue program(mer) could very easily - flood the rsyslogd daemon with syslog messages resulting in the log - files consuming all the remaining space on the filesystem. Activating - logging over the inet domain sockets will of course expose a system to - risks outside of programs or individuals on the local machine. - - There are a number of methods of protecting a machine: - - 1. Implement kernel firewalling to limit which hosts or networks - have access to the 514/UDP socket. - - 2. Logging can be directed to an isolated or non-root filesystem - which, if filled, will not impair the machine. - - 3. The ext2 filesystem can be used which can be configured to limit - a certain percentage of a filesystem to usage by root only. - <B>NOTE </B>that this will require rsyslogd to be run as a non-root - process. <B>ALSO NOTE </B>that this will prevent usage of remote log- - ging since rsyslogd will be unable to bind to the 514/UDP - socket. - - 4. Disabling inet domain sockets will limit risk to the local - machine. - - 5. Use step 4 and if the problem persists and is not secondary to a - rogue program/daemon get a 3.5 ft (approx. 1 meter) length of - sucker rod* and have a chat with the user in question. - - Sucker rod def. — 3/4, 7/8 or 1in. hardened steel rod, male - threaded on each end. Primary use in the oil industry in West- - ern North Dakota and other locations to pump ’suck’ oil from oil - wells. Secondary uses are for the construction of cattle feed - lots and for dealing with the occasional recalcitrant or bel- - ligerent individual. - - <B>Message replay and spoofing</B> - If remote logging is enabled, messages can easily be spoofed and - replayed. As the messages are transmitted in clear-text, an attacker - might use the information obtained from the packets for malicious - things. Also, an attacker might reply recorded messages or spoof a - sender’s IP address, which could lead to a wrong perception of system - activity. Be sure to think about syslog network security before - enabling it. - - -<B>DEBUGGING</B> - When debugging is turned on using <B>-d </B>option then <B>rsyslogd </B>will be very - verbose by writing much of what it does on stdout. Whenever the con- - figuration file is reread and re-parsed you’ll see a tabular, corre- - sponding to the internal data structure. This tabular consists of four - fields: - - <I>number </I>This field contains a serial number starting by zero. This num- - ber represents the position in the internal data structure (i.e. - the array). If one number is left out then there might be an - error in the corresponding line in <I>/etc/rsyslog.conf</I>. - - <I>pattern</I> - This field is tricky and represents the internal structure - exactly. Every column stands for a facility (refer to <B>sys-</B> - <B>log</B>(3)). As you can see, there are still some facilities left - free for former use, only the left most are used. Every field - in a column represents the priorities (refer to <B>syslog</B>(3)). - - <I>action </I>This field describes the particular action that takes place - whenever a message is received that matches the pattern. Refer - to the <B>syslog.conf</B>(5) manpage for all possible actions. - - <I>arguments</I> - This field shows additional arguments to the actions in the last - field. For file-logging this is the filename for the logfile; - for user-logging this is a list of users; for remote logging - this is the hostname of the machine to log to; for console-log- - ging this is the used console; for tty-logging this is the spec- - ified tty; wall has no additional arguments. - - - <B>templates</B> - There will also be a second internal structure which lists all - defined templates and there contents. This also enables you to - see the internally-defined, hardcoded templates. - -<B>FILES</B> - <I>/etc/rsyslog.conf</I> - Configuration file for <B>rsyslogd</B>. See <B>rsyslog.conf</B>(5) for exact - information. - <I>/dev/log</I> - The Unix domain socket to from where local syslog messages are - read. - <I>/var/run/rsyslogd.pid</I> - The file containing the process id of <B>rsyslogd</B>. - -<B>BUGS</B> - Please review the file BUGS for up-to-date information on known bugs - and annoyances. - -<B>Further Information</B> - Please visit <B>http://www.rsyslog.com/doc </B>for additional information, - tutorials and a support forum. - -<B>SEE ALSO</B> - <B>rsyslog.conf</B>(5), <B>logger</B>(1), <B>syslog</B>(2), <B>syslog</B>(3), <B>services</B>(5), - <B>savelog</B>(8) - - -<B>COLLABORATORS</B> - <B>rsyslogd </B>is derived from sysklogd sources, which in turn was taken from - the BSD sources. Special thanks to Greg Wettstein (greg@wind.enjel- - lic.com) and Martin Schulze (joey@linux.de) for the fine sysklogd pack- - age. - - Rainer Gerhards - Adiscon GmbH - Grossrinderfeld, Germany - rgerhards@adiscon.com - - Michael Meckelein - Adiscon GmbH - mmeckelein@adiscon.com - - - -Version 1.16.1 (devel) 17 July 2007 RSYSLOGD(8) -</PRE></BODY> diff --git a/doc/manual.html b/doc/manual.html index 5d43cb25..aff23c8b 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -14,13 +14,13 @@ replacement. Its <a href="features.html"> advanced features</a> make it suitable for enterprise-class, <a href="rsyslog_tls.html">encryption protected syslog</a> relay chains while at the same time being very easy to setup for the novice user. And as we know what enterprise users really need, there is -also <a href="professional_support.html">professional +also <a href="http://www.rsyslog.com/professional-services">professional rsyslog support</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 4.4.2 (v4-stable) of rsyslog.</b> -Visit the <i> <a href="http://www.rsyslog.com/doc-status.html">rsyslog status page</a></i></b> to obtain current +<p><b>This documentation is for version 4.6.4 (v4-stable branch) of rsyslog.</b> +Visit the <i> <a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might want to lend us a helping hand. </b>It doesn't require a lot of @@ -28,12 +28,13 @@ time - even a single mouse click helps. Learn <a href="how2help.html">how to hel Due to popular demand, there is now a <a href="rsyslog_ng_comparison.html">side-by-side comparison between rsyslog and syslog-ng</a>.</p> <p>If you are upgrading from rsyslog v2 or stock sysklogd, -<a href="v3compatibility.html">be -sure to read the rsyslog v3 compatibility document!</a> It will work even +<a href="v3compatibility.html">be sure to read the rsyslog v3 compatibility document</a>, +and if you are upgrading from v3, read the +<a href="v4compatibility.html">rsyslog v4 compatibility document</a>. +<p>Rsyslog will work even if you do not read the doc, but doing so will definitely improve your experience.</p> -<p><span style="font-weight: bold;"></span><b>Follow -the links below for the</b><br></p><ul> - +<p><b>Follow the links below for the</b></p> +<ul> <li><a href="troubleshoot.html">troubleshooting rsyslog problems</a></li> <li><a href="rsyslog_conf.html">configuration file syntax (rsyslog.conf)</a></li> <li><a href="http://www.rsyslog.com/tool-regex">a regular expression checker/generator tool for rsyslog</a></li> @@ -42,8 +43,9 @@ the links below for the</b><br></p><ul> <li><a href="bugs.html">rsyslog bug list</a></li> <li><a href="rsyslog_packages.html"> rsyslog packages</a></li> <li><a href="generic_design.html">backgrounder on -generic syslog application design</a><!-- not good as it currently is ;) <li><a href="contributors.html">contributor "Hall of Fame"</a>--></li> +generic syslog application design</a> <li><a href="modules.html">description of rsyslog modules</a></li> +<li><a href="http://cookbook.rsyslog.com">the rsyslog "cookbook"</a> - a set of configurations ready to use</li> </ul> <p><b>We have some in-depth papers on</b></p> <ul> @@ -51,8 +53,10 @@ generic syslog application design</a><!-- not good as it currently is ;) <li><a <li><a href="build_from_repo.html">obtaining rsyslog from the source repository</a></li> <li><a href="ipv6.html">rsyslog and IPv6</a> (which is fully supported)</li> <li><a href="rsyslog_secure_tls.html">native TLS encryption for syslog</a></li> +<li><a href="multi_ruleset.html">using multiple rule sets in rsyslog</a></li> <li><a href="rsyslog_stunnel.html">ssl-encrypting syslog with stunnel</a></li> <li><a href="rsyslog_mysql.html">writing syslog messages to MySQL (and other databases as well)</a></li> +<li><a href="rsyslog_pgsql.html">writing syslog messages to PostgreSQL (and other databases as well)</a></li> <li><a href="rsyslog_high_database_rate.html">writing massive amounts of syslog messages to a database</a></li> <li><a href="rsyslog_reliable_forwarding.html">reliable forwarding to a remote server</a></li> <li><a href="rsyslog_php_syslog_ng.html">using diff --git a/doc/multi_ruleset.html b/doc/multi_ruleset.html new file mode 100644 index 00000000..8d8c614f --- /dev/null +++ b/doc/multi_ruleset.html @@ -0,0 +1,275 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<title>Multiple Rulesets in rsyslog</title></head> +<body> +<h1>Multiple Rulesets in rsyslog</h1> +<p>Starting with version 4.5.0 and 5.1.1, <a href="http://www.rsyslog.com">rsyslog</a> supports +multiple rulesets within a single configuration. +This is especially useful for routing the recpetion of remote messages to a set of specific rules. +Note that the input module must support binding to non-standard rulesets, so the functionality +may not be available with all inputs. +<p>In this document, I am using <a href="imtcp.html">imtcp</a>, an input module +that supports binding to non-standard rulesets since rsyslog started to support them. +<h2>What is a Ruleset?</h2> +If you have worked with (r)syslog.conf, you know that it is made up of what I call rules (others +tend to call them selectors, a sysklogd term). Each rule consist of a filter and one or more +actions to be carried out when the filter evaluates to true. A filter may be as simple as a +traditional +syslog priority based filter (like "*.*" or "mail.info" or a as complex as a +script-like expression. Details on that are covered in the config file documentation. After the +filter come action specifiers, and an action is something that does something to a message, e.g. +write it to a file or forward it to a remote logging server. + +<p>A traditional configuration file is made up of one or more of these rules. When a new +message arrives, its processing starts with the first rule (in order of appearance in +rsyslog.conf) and continues for each rule until either all rules have been processed or +a so-called "e;discard" action happens, in which case processing stops and the +message is thrown away (what also happens after the last rule has been processed). + +<p>The <b>multi-ruleset</b> support now permits to specify more than one such rule sequence. +You can think of a traditional config file just as a single default rule set, which is +automatically bound to each of the inputs. This is even what actually happens. When +rsyslog.conf is processed, the config file parser looks for the directive + +<pre>$RuleSet <name> +</pre> + +<p>Where name is any name the user likes (but must not start with "RSYSLOG_", which +is the name space reserved for rsyslog use). If it finds this directive, it begins a new +rule set (if the name was not yet know) or switches to an already-existing one (if the name +was known). All rules defined between this $RuleSet directive and the next one are appended +to the named ruleset. Note that the reserved name "RSYSLOG_DefaultRuleset" is used to +specify rsyslogd's default ruleset. You can use that name whereever you can use a ruleset name, +including when binding an input to it. + +<p>Inside a ruleset, messages are processed as described above: they start with the first rule +and rules are processed in the order of appearance of the configuration file until either +there are no more rules or the discard action is executed. Note that with multiple rulesets +no longer <b>all</b> rsyslog.conf rules are executed but <b>only</b> those that are +contained within the specific ruleset. + +<p>Inputs must explicitely bind to rulesets. If they don't do, the default ruleset is bound. + +<p>This brings up the next question: + +<h2>What does "To bind to a Ruleset" mean?</h2> +<p>This term is used in the same sense as "to bind an IP address to an interface": +it means that a specific input, or part of an input (like a tcp listener) will use a specific +ruleset to "pass its messages to". So when a new message arrives, it will be processed +via the bound ruleset. Rule from all other rulesets are irrelevant and will never be processed. +<p>This makes multiple rulesets very handy to process local and remote message via +seperate means: bind the respective receivers to different rule sets, and you do not need +to seperate the messages by any other method. + +<p>Binding to rulesets is input-specifc. For imtcp, this is done via the + +<pre>$InputTCPServerBindRuleset <name> +</pre> + +directive. Note that "name"e; must be the name of a ruleset that is already defined +at the time the bind directive is given. There are many ways to make sure this happens, but +I personally think that it is best to define all rule sets at the top of rsyslog.conf and +define the inputs at the bottom. This kind of reverses the traditional recommended ordering, but +seems to be a really useful and straightforward way of doing things. +<h2>Can I use a different Ruleset as the default?</h2> +<p>This is possible by using the + +<pre>$DefaultRuleset <name> +</pre> + +Directive. Please note, however, that this directive is actually global: that is, it does not +modify the ruleset to which the next input is bound but rather provides a system-wide +default rule set for those inputs that did not explicitly bind to one. As such, the directive +can not be used as a work-around to bind inputs to non-default rulesets that do not support +ruleset binding. +<h2>Examples</h2> +<h3>Split local and remote logging</h3> +<p>Let's say you have a pretty standard system that logs its local messages to the usual +bunch of files that are specified in the default rsyslog.conf. As an example, your rsyslog.conf +might look like this: + +<pre> +# ... module loading ... +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +</pre> + +<p>Now, you want to add receive messages from a remote system and log these to +a special file, but you do not want to have these messages written to the files +specified above. The traditional approach is to add a rule in front of all others that +filters on the message, processes it and then discards it: + +<pre> +# ... module loading ... +# process remote messages +:fromhost-ip, isequal, "192.0.2.1" /var/log/remotefile +& ~ +# only messages not from 192.0.21 make it past this point + +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +</pre> + +<p>Note the tilde character, which is the discard action!. Also note that we assume that +192.0.2.1 is the sole remote sender (to keep it simple). + +<p>With multiple rulesets, we can simply define a dedicated ruleset for the remote reception +case and bind it to the receiver. This may be written as follows: + +<pre> +# ... module loading ... +# process remote messages +# define new ruleset and add rules to it: +$RuleSet remote +*.* /var/log/remotefile +# only messages not from 192.0.21 make it past this point + +# bind ruleset to tcp listener +$InputTCPServerBindRuleset remote +# and activate it: +$InputTCPServerRun 10514 + +# switch back to the default ruleset: +$RuleSet RSYSLOG_DefaultRuleset +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +</pre> + +<p>Here, we need to switch back to the default ruleset after we have defined our custom +one. This is why I recommend a different ordering, which I find more intuitive. The sample +below has it, and it leads to the same results: + +<pre> +# ... module loading ... +# at first, this is a copy of the unmodified rsyslog.conf +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +# end of the "regular" rsyslog.conf. Now come the new definitions: + +# process remote messages +# define new ruleset and add rules to it: +$RuleSet remote +*.* /var/log/remotefile + +# bind ruleset to tcp listener +$InputTCPServerBindRuleset remote +# and activate it: +$InputTCPServerRun 10514 +</pre> + +<p>Here, we do not switch back to the default ruleset, because this is not needed as it is +completely defined when we begin the "remote" ruleset. + +<p>Now look at the examples and compare them to the single-ruleset solution. You will notice +that we do <b>not</b> need a real filter in the multi-ruleset case: we can simply use +"*.*" as all messages now means all messages that are being processed by this +rule set and all of them come in via the TCP receiver! This is what makes using multiple +rulesets so much easier. + +<h3>Split local and remote logging for three different ports</h3> +<p>This example is almost like the first one, but it extends it a little bit. While it is +very similar, I hope it is different enough to provide a useful example why you may want +to have more than two rulesets. + +<p>Again, we would like to use the "regular" log files for local logging, only. But +this time we set up three syslog/tcp listeners, each one listening to a different +port (in this example 10514, 10515, and 10516). Logs received from these receivers shall go into +different files. Also, logs received from 10516 (and only from that port!) with +"mail.*" priority, shall be written into a specif file and <b>not</b> be +written to 10516's general log file. + +<p>This is the config: + +<pre> +# ... module loading ... +# at first, this is a copy of the unmodified rsyslog.conf +# The authpriv file has restricted access. +authpriv.* /var/log/secure +# Log all the mail messages in one place. +mail.* /var/log/maillog +# Log cron stuff +cron.* /var/log/cron +# Everybody gets emergency messages +*.emerg * +... more ... +# end of the "regular" rsyslog.conf. Now come the new definitions: + +# process remote messages + +#define rulesets first +$RuleSet remote10514 +*.* /var/log/remote10514 + +$RuleSet remote10515 +*.* /var/log/remote10515 + +$RuleSet remote10516 +mail.* /var/log/mail10516 +& ~ +# note that the discard-action will prevent this messag from +# being written to the remote10516 file - as usual... +*.* /var/log/remote10516 + +# and now define listners bound to the relevant ruleset +$InputTCPServerBindRuleset remote10514 +$InputTCPServerRun 10514 + +$InputTCPServerBindRuleset remote10515 +$InputTCPServerRun 10515 + +$InputTCPServerBindRuleset remote10516 +$InputTCPServerRun 10516 +</pre> + +<p>Note that the "mail.*" rule inside the "remote10516"e; ruleset does +not affect processing inside any other rule set, including the default rule set. + + +<h2>Performance</h2> +<p>No rule processing can be faster than not processing a rule at all. As such, it is useful +for a high performance system to identify disjunct actions and try to split these off to +different rule sets. In the example section, we had a case where three different tcp listeners +need to write to three different files. This is a perfect example of where multiple rule sets +are easier to use and offer more performance. The performance is better simply because there +is no need to check the reception service - instead messages are automatically pushed to the +right rule set and can be processed by very simple rules (maybe even with +"*.*"-filters, the fastest ones available). + +<p>In the long term, multiple rule sets will probably lay the foundation for even better +optimizations. So it is not a bad idea to get aquainted with them. + +<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/omoracle.html b/doc/omoracle.html index 40f6360f..cfcf277f 100644 --- a/doc/omoracle.html +++ b/doc/omoracle.html @@ -67,6 +67,128 @@ it is suggested to post questions to the need to define the properties on the template in the correct order you want them passed to the statement! </pre> +<p>Some additional documentation contributed by Ronny Egner: +<pre> +REQUIREMENTS: +-------------- + +- Oracle Instantclient 10g (NOT 11g) Base + Devel + (if you´re on 64-bit linux you should choose the 64-bit libs!) +- JDK 1.6 (not neccessary for oracle plugin but "make" didd not finsished successfully without it) + +- "oracle-instantclient-config" script + (seems to shipped with instantclient 10g Release 1 but i was unable to find it for 10g Release 2 so here it is) + + +====================== /usr/local/bin/oracle-instantclient-config ===================== +#!/bin/sh +# +# Oracle InstantClient SDK config file +# Jean-Christophe Duberga - Bordeaux 2 University +# + +# just adapt it to your environment +incdirs="-I/usr/include/oracle/10.2.0.4/client64" +libdirs="-L/usr/lib/oracle/10.2.0.4/client64/lib" + +usage="\ +Usage: oracle-instantclient-config [--prefix[=DIR]] [--exec-prefix[=DIR]] [--version] [--cflags] [--libs] [--static-libs]" + +if test $# -eq 0; then + echo "${usage}" 1>&2 + exit 1 +fi + +while test $# -gt 0; do + case "$1" in + -*=*) optarg=`echo "$1" | sed 's/[-_a-zA-Z0-9]*=//'` ;; + *) optarg= ;; + esac + + case $1 in + --prefix=*) + prefix=$optarg + if test $exec_prefix_set = no ; then + exec_prefix=$optarg + fi + ;; + --prefix) + echo $prefix + ;; + --exec-prefix=*) + exec_prefix=$optarg + exec_prefix_set=yes + ;; + --exec-prefix) + echo ${exec_prefix} + ;; + --version) + echo ${version} + ;; + --cflags) + echo ${incdirs} + ;; + --libs) + echo $libdirs -lclntsh -lnnz10 -locci -lociei -locijdbc10 + ;; + --static-libs) + echo "No static libs" 1>&2 + exit 1 + ;; + *) + echo "${usage}" 1>&2 + exit 1 + ;; + esac + shift +done + +=============== END ============== + + + + +COMPILING RSYSLOGD +------------------- + + +./configure --enable-oracle + + + + +RUNNING +------- + +- make sure rsyslogd is able to locate the oracle libs (either via LD_LIBRARY_PATH or /etc/ld.so.conf) +- set TNS_ADMIN to point to your tnsnames.ora +- create a tnsnames.ora and test you are able to connect to the database + +- create user in oracle as shown in the following example: + create user syslog identified by syslog default tablespace users quota unlimited on users; + grant create session to syslog; + create role syslog_role; + grant syslog_role to syslog; + grant create table to syslog_role; + grant create sequence to syslog_role; + +- create tables as needed + +- configure rsyslog as shown in the following example + $ModLoad omoracle + + $OmoracleDBUser syslog + $OmoracleDBPassword syslog + $OmoracleDB syslog + $OmoracleBatchSize 1 + $OmoracleBatchItemSize 4096 + + $OmoracleStatementTemplate OmoracleStatement + $template OmoracleStatement,"insert into foo(hostname,message) values (:host,:message)" + $template TestStmt,"%hostname%%msg%" + *.* :omoracle:;TestStmt + (you guess it: username = password = database = "syslog".... see $rsyslogd_source/plugins/omoracle/omoracle.c for me info) +</pre> <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the diff --git a/doc/professional_support.html b/doc/professional_support.html deleted file mode 100644 index 7724ede8..00000000 --- a/doc/professional_support.html +++ /dev/null @@ -1,112 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>Professional Support for Rsyslog</title> - -</head> -<body> -<h1>Professional Services for Rsyslog</h1> -<p>Professional services are being offered by <a href="http://www.adiscon.com">Adiscon</a>, the company -that sponsors rsyslog development. For details, please contact <a href="mailto:info%40adiscon.com">Adiscon Sales</a>. </p> - -<h3>EMail Support Service</h3> -Price: 99.00 EURO <br> -Duration: 180 days -<br> -Support level: 8x5 -<p>Purchase rsyslog support directly from the source. This -contract provides priority email support. It is a great option if you -need to provide proof of software support in your organization. This -contract provides</p> -<ul> -<li>unlimited email support tickets during validity -</li> -<li><span style="font-weight: bold;">fixes for</span> -current and <span style="font-weight: bold;">past rsyslog -releases</span> -</li> -<li>advise on how to implement rsyslog in the best possible -way. -</li> -</ul> -<p>Under this contract, fixes for old rsyslog releases will be -provided / created, provided that it is possible to do that with the -code base in question. Phone support is not included.</p> -<h3>Custom-Written Config File</h3> -Price: 29.00 EURO -<br> -Duration: N/A -<br> -Support level: 8x5 -<p>Creating rsyslog config files is easy - but if you would like -to have that extra feature and have no time to do it, this service is -for you. Important: BEFORE you purchase this service, contact us and -inquire (via <a href="mailto:info%40adiscon.com">info@adiscon.com</a>) -whether or not your desired result can be achieved via rsyslog. Once -this is clear, order the service and we will ship a custom-made -configuration file within 5 working days (at latest, most often much -faster). For security reasons, we will not put passwords into the -configuration file, but will place easy to read comments in the places -where you need to put them in. The agreement is governed under German -law. You may also purchase this service if you would like to have your -own configuration file reviewed, e.g. for auditing purposes.</p> -<h3>Local Installation Support</h3> -<p>If you intend to install rsyslog on your system but would like -to do so with minimal effort and according to your specification, you -can ask us to perform the installation for you. You get a perfect -installation, exactly like you needed, but without a need to -touch the system. This is a perfect choice for the busy administrator! -<p>In order to perform this work, we just need ssh access to your -system and the proper permissions. -<p>We charge a low one-time fee for this service. For details, please -contact <a href="mailto:info@adiscon.com">info@adiscon.com</a>. -<h3>Local Installation Maintenance</h3> -<p>If you used our services to set up the system, why not keep it -running perfectly with maintenance support? Under this contract, we -assure you run a recent build that does not interfere with your -environment and we even carry out change requests you may have. So this -is a hassle-free, everything cared about solution. -<p>Again, all we need to have is ssh access and the proper permissions -to your machine. Of course, work will only be carried out when you -expect us to do so. You are always in control of what happens. This -is a perfect outsourcing solution for those who would like to run -a great logging system but can not afford the time to keep it -in perfect shape! -<p>We charge a low monthly fee for this service. For details, please -contact <a href="mailto:info@adiscon.com">info@adiscon.com</a>. -<h3>Custom Development</h3> -<p>Do you need an exotic feature that otherwise would not be implemented? -Do you need something really quick, quicker than it is available via -the regular development schedule? Then, you may consider funding -development for a specific functionality. We are always looking for -interesting projects. If you hire us to to do the job, you can be sure -to get the best possible and probably quickest solution, because we are -obviously at the heart of the source code. No need to get aquainted to -anything, no risk of misunderstanding program concepts. Benefit from -our vast syslog experience.</p> -<p>Please note that custom development is not limited to rsyslog. We offer -a number of logging solutions and can also work as part of your time -for specific requirements. The opportunities are endless, just ask. We -will work with you on your requirements and provide a quote on the -estimated cost. Just write to <a href="mailto:sales@adiscon.com">sales@adiscon.com</a> for details.</p><h3>Consulting Services</h3> -<p>Do you have demanding logging requirements? Why not talk to a -real logging professional? Instead of trying to find the solution -like a needle in the haystack, talk to the team that brought rsyslog, -phpLogCon, the Windows MonitorWare products and other logging -solutions. We sweat logging for over 15 years now and can help quickly. -Depending on your needs, consulting can be carried out via email, the -phone or on your premises (for larger or local projects). Everything is -possible, it just depends on your needs. Consulting services are -available in English and German. Just mail <a href="mailto:sales@adiscon.com">sales@adiscon.com</a> what you are interested in and we will work with you on a proposal that fits your needs. -</p><p></p><p>All agreements are -governed under German law. -</p> - -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/property_replacer.html b/doc/property_replacer.html index 7b604ea0..4d242a34 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -335,6 +335,19 @@ Especially useful for PIX.</td> <td>format as RFC 3164 date</td> </tr> <tr> +<tr> +<td valign="top"><b>date-rfc3164-buggyday</b></td> +<td>similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands +that a space is written for single-digit days. With this option, a zero is +written instead. This format seems to be used by syslog-ng and the +date-rfc3164-buggyday option can be used in migration scenarios where otherwise +lots of scripts would need to be adjusted. It is recommended <i>not</i> to use this +option when forwarding to remote hosts - they may treat the date as invalid +(especially when parsing strictly according to RFC 3164).</td> +<br><i>This feature was introduced in rsyslog 4.6.2 and v4 versions above and +5.5.3 and all versions above.</i> +</tr> +<tr> <td><b>date-rfc3339</b></td> <td>format as RFC 3339 date</td> </tr> diff --git a/doc/queues.html b/doc/queues.html index 4a9509a0..45ce1bd1 100644 --- a/doc/queues.html +++ b/doc/queues.html @@ -115,7 +115,11 @@ isolation. This is currently selected by specifying different <i>$WorkDirectory< config directives before the queue creation statement.</p> <p>To create a disk queue, use the "<i>$<object>QueueType Disk</i>" config directive. Checkpoint intervals can be specified via "<i>$<object>QueueCheckpointInterval</i>", -with 0 meaning no checkpoints. </p> +with 0 meaning no checkpoints. Note that disk-based queues can be made very reliable +by issuing a (f)sync after each write operation. Starting with version 4.3.2, this can +be requested via "<i><object>QueueSyncQueueFiles on/off</i> with the +default being off. Activating this option has a performance penalty, so it should +not be turned on without reason.</p> <h2>In-Memory Queues</h2> <p>In-memory queue mode is what most people have on their mind when they think about computing queues. Here, the enqueued data elements are held in memory. diff --git a/doc/rsconf1_markmessageperiod.html b/doc/rsconf1_markmessageperiod.html index 2c833339..a6486ba1 100644 --- a/doc/rsconf1_markmessageperiod.html +++ b/doc/rsconf1_markmessageperiod.html @@ -7,7 +7,7 @@ <h2>$MarkMessagePeriod</h2> <p><b>Type:</b> specific to immark input module</p> -<p><b>Default:</b> 1800 (20 minutes)</p> +<p><b>Default:</b> 1200 (20 minutes)</p> <p><b>Description:</b></p> <p>This specifies when mark messages are to be written to output modules. The time specified is in seconds. Specifying 0 is possible and disables mark diff --git a/doc/rsyslog_conf_actions.html b/doc/rsyslog_conf_actions.html index 2ef3f4b0..6020dd88 100644 --- a/doc/rsyslog_conf_actions.html +++ b/doc/rsyslog_conf_actions.html @@ -37,8 +37,15 @@ compared to the otherwise-equal config directives below:</p> *.=crit /var/log/critmsgs</b></code></p> <p> </p> <h3>Regular File</h3> -<p>Typically messages are logged to real files. The file has to -be specified with full pathname, beginning with a slash "/''.<br> +<p>Typically messages are logged to real files. The file usually is +specified by full pathname, beginning with a slash "/". +Starting with version 4.6.2 and 5.4.1 (previous v5 version do NOT support this) +relative file names can also be specified. To do so, these must begin with a +dot. For example, use "./file-in-current-dir.log" to specify a file in the +current directory. Please note that rsyslogd usually changes its working +directory to the root, so relative file names must be tested with care (they +were introduced primarily as a debugging vehicle, but may have useful other applications +as well).<br> <br> <br> You may prefix each entry with the minus "-'' sign to omit syncing the @@ -98,18 +105,14 @@ done, same with /dev/console.</p> messages to a remote host running rsyslogd(8) and to receive messages from remote hosts. Using this feature you're able to control all syslog messages on one host, if all other machines will log remotely to that. -This tears down<br> -administration needs.<br> -<br> -<b>Please note that this version of rsyslogd by default does NOT -forward messages it has received from the network to another host. -Specify the "-h" option to enable this.</b></p> +This tears down administration needs.</p> <p>To forward messages to another host, prepend the hostname with the at sign ("@"). A single at sign means that messages will be forwarded via UDP protocol (the standard for syslog). If you prepend two at signs ("@@"), the messages will be transmitted via TCP. Please note that plain TCP based syslog is not officially standardized, but -most major syslogds support it (e.g. syslog-ng or WinSyslog). The +most major syslogds support it (e.g. syslog-ng or +<a href="http://www.winsyslog.com/">WinSyslog</a>). The forwarding action indicator (at-sign) can be followed by one or more options. If they are given, they must be immediately (without a space) following the final at sign and be enclosed in parenthesis. The diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html index 1d30d8ae..63c29817 100644 --- a/doc/rsyslog_conf_filter.html +++ b/doc/rsyslog_conf_filter.html @@ -48,8 +48,8 @@ in rsyslog and offer the best performance for this job.</p> facility and a priority, separated by a period (".''). Both parts are case insensitive and can also be specified as decimal numbers, but don't do that, you have been warned. Both facilities and priorities are -described in rsyslog(3). The names mentioned below correspond to the -similar LOG_-values in /usr/include/rsyslog.h.<br> +described in syslog(3). The names mentioned below correspond to the +similar LOG_-values in /usr/include/syslog.h.<br> <br> The facility is one of the following keywords: auth, authpriv, cron, daemon, kern, lpr, mail, mark, news, security (same as auth), syslog, @@ -231,7 +231,7 @@ A few quick samples:<br> <br> <code> *.* /var/log/file1 # the traditional way<br> -if $msg contains 'error' /var/log/errlog # the expression-based way<br> +if $msg contains 'error' then /var/log/errlog # the expression-based way<br> </code> <br> Right now, you need to specify numerical values if you would like to diff --git a/doc/rsyslog_conf_global.html b/doc/rsyslog_conf_global.html index 7dda046f..ce46bac2 100644 --- a/doc/rsyslog_conf_global.html +++ b/doc/rsyslog_conf_global.html @@ -99,6 +99,13 @@ netstream drivers. For all others, it will be ignored. <li>$ActionSendStreamDriverPermittedPeer <ID>, accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific) -<span style="font-weight: bold;"> directive may go away</span>!</li> +<li><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send +action to close and re-open the connection to the remote host every nbr of messages sent. +Zero, the default, means that no such processing is done. This directive is useful for +use with load-balancers. Note that there is some performance overhead associated with it, +so it is advisable to not too often "rebind" the connection (what +"too often" actually means depends on your configuration, a rule of thumb is +that it should be not be much more often than once per second).</li> <li><b>$ActionSendUDPRebindInterval</b> nbr</a>- [available since 4.3.2] - instructs the UDP send action to rebind the send socket every nbr of messages sent. Zero, the default, means that no rebind is done. This directive is useful for use with load-balancers.</li> @@ -111,6 +118,10 @@ that no rebind is done. This directive is useful for use with load-balancers.</l <li>$DefaultNetstreamDriver <drivername>, the default <a href="netstream.html">network stream driver</a> to use. Defaults to ptcp.$DefaultNetstreamDriverCAFile </path/to/cafile.pem></li> <li>$DefaultNetstreamDriverCertFile </path/to/certfile.pem></li> <li>$DefaultNetstreamDriverKeyFile </path/to/keyfile.pem></li> +<li><b>$DefaultRuleset</b> <i>name</i> - changes the default ruleset for unbound inputs to +the provided <i>name</i> (the default default ruleset is named +"RSYSLOG_DefaultRuleset"). It is advised to also read +our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li> <li><b>$CreateDirs</b> [<b>on</b>/off] - create directories on an as-needed basis</li> <li><a href="rsconf1_dircreatemode.html">$DirCreateMode</a></li> <li><a href="rsconf1_dirgroup.html">$DirGroup</a></li> @@ -128,11 +139,15 @@ that no rebind is done. This directive is useful for use with load-balancers.</l <li><a href="rsconf1_gssforwardservicename.html">$GssForwardServiceName</a></li> <li><a href="rsconf1_gsslistenservicename.html">$GssListenServiceName</a></li> <li><a href="rsconf1_gssmode.html">$GssMode</a></li> -<li>$HUPisRestart [<b>on</b>/off] - if set to on, a HUP is a full daemon restart. This means any queued messages are discarded (depending +<li>$HUPisRestart [on/<b>off</b>] - if set to on, a HUP is a full daemon restart. This means any queued messages are discarded (depending on queue configuration, of course) all modules are unloaded and reloaded. This mode keeps compatible with sysklogd, but is -not recommended for use with rsyslog. To do a full restart, simply stop and start the daemon. The default is "on" for -compatibility reasons. If it is set to "off", a HUP will only close open files. This is a much quicker action and usually -the only one that is needed e.g. for log rotation. <b>It is recommended to set the setting to "off".</b></li> +not recommended for use with rsyslog. To do a full restart, simply stop and start the daemon. The default (since 4.5.1) is "off". +If it is set to "off", a HUP will only close open files. This is a much quicker action and usually +the only one that is needed e.g. for log rotation. <b>Restart-type HUPs (value "on") are depricated</b> +and will go away in rsyslog v5. So it is a good idea to change anything that needs it, now. +Usually that should not be a big issue, as the restart-type HUP can easily be replaced by +something along the lines of "/etc/init.d/rsyslog restart". +</li> <li><a href="rsconf1_includeconfig.html">$IncludeConfig</a></li><li>MainMsgQueueCheckpointInterval <number></li> <li>$MainMsgQueueDequeueSlowdown <number> [number is timeout in <i> micro</i>seconds (1000000us is 1sec!), @@ -192,6 +207,28 @@ supported in order to be compliant to the upcoming new syslog RFC series. <li><a href="rsconf1_maxopenfiles.html">$MaxOpenFiles</a></li> <li><a href="rsconf1_moddir.html">$ModDir</a></li> <li><a href="rsconf1_modload.html">$ModLoad</a></li> +<li><b>$OMFileAsyncWriting</b> [on/<b>off</b>], if turned on, the files will be written +in asynchronous mode via a separate thread. In that case, double buffers will be used so +that one buffer can be filled while the other buffer is being written. Note that in order +to enable $OMFileFlushInterval, $OMFileAsyncWriting must be set to "on". Otherwise, the flush +interval will be ignored. Also note that when $OMFileFlushOnTXEnd is "on" but +$OMFileAsyncWriting is off, output will only be written when the buffer is full. This may take +several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced +in that case by sending rsyslogd a HUP signal. +<li><b>$OMFileZipLevel</b> 0..9 [default 0] - if greater 0, turns on gzip compression +of the output file. The higher the number, the better the compression, but also the +more CPU is required for zipping.</li> +<li><b>$OMFileIOBufferSize</b> <size_nbr>, default 4k, size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)</li> +<li><b>$OMFileFlushOnTXEnd</b> <[<b>on</b>/off]>, default on. Omfile has the +capability to +write output using a buffered writer. Disk writes are only done when the buffer is +full. So if an error happens during that write, data is potentially lost. In cases where +this is unacceptable, set $OMFileFlushOnTXEnd to on. Then, data is written at the end +of each transaction (for pre-v5 this means after <b>each</b> log message) and the usual +error recovery thus can handle write errors without data loss. Note that this option +severely reduces the effect of zip compression and should be switched to off +for that use case. Note that the default -on- is primarily an aid to preserve +the traditional syslogd behaviour.</li> <li><b>$RepeatedMsgContainsOriginalMsg</b> [on/<b>off</b>] - "last message repeated n times" messages, if generated, have a different format that contains the message that is being repeated. Note that only the first "n" characters are included, with n to be at least 80 characters, most @@ -200,6 +237,12 @@ line is that n is large enough to get a good idea which message was repeated but large enough for the whole message. (Introduced with 4.1.5). Once set, it affects all following actions.</li> <li><a href="rsconf1_repeatedmsgreduction.html">$RepeatedMsgReduction</a></li> <li><a href="rsconf1_resetconfigvariables.html">$ResetConfigVariables</a></li> +<li><b>$Ruleset</b> <i>name</i> - starts a new ruleset or switches back to one already defined. +All following actions belong to that new rule set. +the <i>name</i> does not yet exist, it is created. To swith back to rsyslog's +default ruleset, specify "RSYSLOG_DefaultRuleset") as the name. +All following actions belong to that new rule set. It is advised to also read +our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li> <li><b>$OptimizeForUniprocessor</b> [on/<b>off</b>] - turns on optimizatons which lead to better performance on uniprocessors. If you run on multicore-machiens, turning this off lessens CPU load. The default may change as uniprocessor systems become less common. [available since 4.1.0]</li> diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index df9abeea..675b8bb3 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -2,11 +2,42 @@ <html><head><title>Modules - rsyslog.conf</title></head> <body> <p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h2>Modules</h2> -<p>Rsyslog has a modular design. Consequently, there is a growing +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Modules</h1> +<p>Rsyslog has a modular design. This enables functionality to be +dynamically loaded from modules, which may also be written by any +third party. Rsyslog itself offers all non-core functionality as +modules. Consequently, there is a growing number of modules. Here is the entry point to their documentation and what they do (list is currently not complete)</p> +<p>Please note that each module provides configuration +directives, which are NOT necessarily being listed below. Also +remember, that a modules configuration directive (and functionality) is +only available if it has been loaded (using $ModLoad).</p> +<p>It is relatively easy to write a rsyslog module. <b>If none of the provided +modules solve your need, you may consider writing one or have one written +for you by +<a href="http://www.rsyslog.com/professional-services">Adiscon's professional services for rsyslog</a> +</b>(this often is a very cost-effective and efficient way of getting what you need). + +<h2>Input Modules</h2> +<p>Input modules are used to gather messages from various sources. They interface +to message generators. +<ul> +<li><a href="imfile.html">imfile</a> - input module for text files</li> +<li><a href="imrelp.html">imrelp</a> - RELP input module</li> +<li>imudp - udp syslog message input</li> +<li><a href="imtcp.html">imtcp</a> - input plugin for plain tcp syslog</li> +<li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li> +<li>immark - support for mark messages</li> +<li><a href="imklog.html">imklog</a> - kernel logging</li> +<li><a href="imuxsock.html">imuxsock</a> - unix sockets, including the system log socket</li> +<li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li> +</ul> + +<h2>Output Modules</h2> +<p>Output modules process messages. With them, message formats can be transformed +and messages be transmitted to various different targets. <ul> <li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> <li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li> @@ -20,26 +51,14 @@ SQLLite, Ingres, Oracle, mSQL)</li> <li><a href="ommail.html">ommail</a> - permits rsyslog to alert folks by mail if something important happens</li> <li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li> -<li><a href="imfile.html">imfile</a> -- input module for text files</li> -<li><a href="imrelp.html">imrelp</a> - RELP -input module</li> -<li>imudp - udp syslog message input</li> -<li><a href="imtcp.html">imtcp</a> - input -plugin for plain tcp syslog</li> -<li><a href="imgssapi.html">imgssapi</a> - -input plugin for plain tcp and GSS-enabled syslog</li> -<li>immark - support for mark messages</li> -<li><a href="imklog.html">imklog</a> - kernel logging</li> -<li><a href="imuxsock.html">imuxsock</a> - -unix sockets, including the system log socket</li> -<li><a href="im3195.html">im3195</a> - -accepts syslog messages via RFC 3195</li> </ul> -<p>Please note that each module provides configuration -directives, which are NOT necessarily being listed below. Also -remember, that a modules configuration directive (and functionality) is -only available if it has been loaded (using $ModLoad).</p> + +<h2>Library Modules</h2> +<p>Library modules provide dynamically loadable functionality for parts of rsyslog, +most often for other loadable modules. They can not be user-configured and are loaded +automatically by some components. They are just mentioned so that error messages that +point to library moduls can be understood. No module list is provided. + <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html index 6c68b801..baa4ce29 100644 --- a/doc/rsyslog_conf_templates.html +++ b/doc/rsyslog_conf_templates.html @@ -87,8 +87,7 @@ option. Otherwise you will become vulnerable to SQL injection. <br> To escape:<br> % = \%<br> \ = \\ --> '\' is used to escape (as in C)<br> -$template TraditionalFormat,%timegenerated% %HOSTNAME% -%syslogtag%%msg%\n"<br> +$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> <br> Properties can be accessed by the <a href="property_replacer.html">property replacer</a> (see there for details).</p> diff --git a/doc/rsyslog_pgsql.html b/doc/rsyslog_pgsql.html new file mode 100644 index 00000000..dcb9dc3a --- /dev/null +++ b/doc/rsyslog_pgsql.html @@ -0,0 +1,336 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> +<HTML> +<HEAD> + <META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=utf-8"> + <TITLE></TITLE> + <META NAME="GENERATOR" CONTENT="OpenOffice.org 3.1 (Unix)"> + <META NAME="AUTHOR" CONTENT="Marc Schiffbauer"> + <META NAME="CREATED" CONTENT="20100129;15054500"> + <META NAME="CHANGEDBY" CONTENT="Marc Schiffbauer"> + <META NAME="CHANGED" CONTENT="20100129;16035000"> + <META NAME="Info 1" CONTENT=""> + <META NAME="Info 2" CONTENT=""> + <META NAME="Info 3" CONTENT=""> + <META NAME="Info 4" CONTENT=""> + <STYLE TYPE="text/css"> + <!-- + @page { size: 8.27in 11.69in; margin: 0.79in } + P { margin-bottom: 0.08in } + P.western { font-family: "Arial", sans-serif } + H1 { margin-bottom: 0.08in } + H1.western { font-family: "Times New Roman", serif } + H1.cjk { font-family: "DejaVu Sans" } + H1.ctl { font-family: "DejaVu Sans" } + H2 { margin-bottom: 0.08in } + H2.western { font-family: "Times New Roman", serif } + BLOCKQUOTE.western { font-family: "Arial", sans-serif } + H3 { margin-bottom: 0.08in } + H3.western { font-family: "Times New Roman", serif } + A:link { so-language: zxx } + --> + </STYLE> +</HEAD> +<BODY LANG="de-DE" DIR="LTR"> +<H1 CLASS="western"><SPAN LANG="en-US">Writing </SPAN>syslog messages +to MySQL, PostgreSQL or any other supported Database</H1> +<P CLASS="western"><FONT SIZE=2><I>Written by </I></FONT><A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php"><FONT SIZE=2><I>Rainer +Gerhards</I></FONT></A><FONT SIZE=2><I> with some additions by Marc +Schiffbauer (2008-02-28)</I></FONT></P> +<H2 CLASS="western">Abstract</H2> +<P CLASS="western"><SPAN LANG="en-US"><I><B>In this paper, I describe +how to write </B></I></SPAN><A HREF="http://www.monitorware.com/en/topics/syslog/">syslog</A><SPAN LANG="en-US"><I><B> +messages to a </B></I></SPAN><A HREF="http://www.mysql.com/">MySQL</A><SPAN LANG="en-US"><I><B> +or </B></I></SPAN><A HREF="http://www.postgresql.org/">PostgreSQL</A><SPAN LANG="en-US"><I><B> +database.</B></I></SPAN><SPAN LANG="en-US"><I> Having syslog messages +in a database is often handy, especially when you intend to set up a +front-end for viewing them. This paper describes an approach with +</I></SPAN><A HREF="http://www.rsyslog.com/">rsyslogd</A><SPAN LANG="en-US"><I>, +an alternative enhanced syslog daemon natively supporting MySQL and +PostgreSQL. I describe the components needed to be installed and how +to configure them. Please note that as of this writing, rsyslog +supports a variety of databases. While this guide is still MySQL- and +PostgreSQL-focused, you can probably use it together with other ones +too. You just need to modify a few settings.</I></SPAN></P> +<H2 CLASS="western">Background</H2> +<P LANG="en-US" CLASS="western">In many cases, syslog data is simply +written to text files. This approach has some advantages, most +notably it is very fast and efficient. However, data stored in text +files is not readily accessible for real-time viewing and analysis. +To do that, the messages need to be in a database. There are various +ways to store syslog messages in a database. For example, some have +the syslogd write text files which are later feed via a separate +script into the database. Others have written scripts taking the data +(via a pipe) from a non-database-aware syslogd and store them as they +appear. Some others use database-aware syslogds and make them write +the data directly to the database. In this paper, I use that "direct +write" approach. I think it is superior, because the syslogd +itself knows the status of the database connection and thus can +handle it intelligently (well ... hopefully ;)). I use rsyslogd to +acomplish this, simply because I have initiated the rsyslog project +with database-awareness as one goal.</P> +<P CLASS="western"><SPAN LANG="en-US"><B>One word of caution:</B></SPAN><SPAN LANG="en-US"> +while message storage in the database provides an excellent +foundation for interactive analysis, it comes at a cost. Database i/o +is considerably slower than text file i/o. As such, directly writing +to the database makes sense only if your message volume is low enough +to allow a) the syslogd, b) the network, and c) the database server +to catch up with it. Some time ago, I have written a paper on +</SPAN><A HREF="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">optimizing +syslog server performance</A><SPAN LANG="en-US">. While this paper +talks about Window-based solutions, the ideas in it are generic +enough to apply here, too. So it might be worth reading if you +anticipate medium high to high traffic. If you anticipate really high +traffic (or very large traffic spikes), you should seriously consider +forgetting about direct database writes - in my opinion, such a +situation needs either a very specialized system or a different +approach (the text-file-to-database approach might work better for +you in this case). </SPAN> +</P> +<H2 CLASS="western">Overall System Setup</H2> +<P CLASS="western"><SPAN LANG="en-US">In this paper, I concentrate on +the server side. If you are thinking about interactive syslog message +review, you probably want to centralize syslog. In such a scenario, +you have multiple machines (the so-called clients) send their data to +a central machine (called server in this context). While I expect +such a setup to be typical when you are interested in storing +messages in the database, I do not describe how to set it up. This is +beyond the scope of this paper. If you search a little, you will +probably find many good descriptions on </SPAN><SPAN LANG="en-US">how +to centralize syslog. If you do that, it might be a good idea to do +it securely, so you might also be interested in my paper on +</SPAN><A HREF="http://www.rsyslog.com/doc-rsyslog_stunnel.html">ssl-encrypting +syslog message transfer</A><SPAN LANG="en-US">.</SPAN></P> +<P LANG="en-US" CLASS="western">No matter how the messages arrive at +the server, their processing is always the same. So you can use this +paper in combination with any description for centralized syslog +reporting.</P> +<P CLASS="western"><SPAN LANG="en-US">As I already said, I use +rsyslogd on the server. It has intrinsic support for talking to the +supported databases. For obvious reasons, we also need an instance of +MySQL or PostgreSQL running. To keep us focused, the setup of the +database itself is also beyond the scope of this paper. I assume that +you have successfully installed the database and also have a +front-end at hand to work with it (for example, </SPAN><A HREF="http://www.phpmyadmin.net/">phpMyAdmin</A><SPAN LANG="en-US"> +or </SPAN><A HREF="http://phppgadmin.sourceforge.net/">phpPgAdmin</A><SPAN LANG="en-US">. +Please make sure that this is installed, actually working and you +have a basic understanding of how to handle it.</SPAN></P> +<H2 CLASS="western">Setting up the system</H2> +<P CLASS="western"><SPAN LANG="en-US">You need to download and +install rsyslogd first. Obtain it from the </SPAN><A HREF="http://www.rsyslog.com/">rsyslog +site</A><SPAN LANG="en-US">. Make sure that you disable stock +syslogd, otherwise you will experience some difficulties. On some +distributions (Fedora 8 and above, for example), rsyslog may +already by the default syslogd, in which case you obviously do not +need to do anything specific. For many others, there are prebuild +packages available. If you use either, please make sure that you have +the required database plugins for your database available. It usually +is a separate package and typically </SPAN><SPAN LANG="en-US"><B>not</B></SPAN><SPAN LANG="en-US"> +installed by default.</SPAN></P> +<P CLASS="western"><SPAN LANG="en-US">It is important to understand +how rsyslogd talks to the database. In rsyslogd, there is the concept +of "templates". Basically, a template is a string that +includes some replacement characters, which are called "properties" +in rsyslog. Properties are accessed via the "</SPAN><A HREF="http://www.rsyslog.com/doc-property_replacer.html">Property +Replacer</A><SPAN LANG="en-US">". Simply said, you access +properties by including their name between percent signs inside the +template. For example, if the syslog message is "Test", the +template "%msg%" would be expanded to "Test". +Rsyslogd supports sending template text as a SQL statement to the +database. As such, the template must be a valid SQL statement. There +is no limit in what the statement might be, but there are some +obvious and not so obvious choices. For example, a template "drop +table xxx" is possible, but does not make an awful lot of sense. +In practice, you will always use an "insert" statement +inside the template.</SPAN></P> +<P LANG="en-US" CLASS="western">An example: if you would just like to +store the msg part of the full syslog message, you have probably +created a table "syslog" with a single column "message". +In such a case, a good template would be "insert into +syslog(message) values ('%msg%')". With the example above, that +would be expanded to "insert into syslog(message) +values('Test')". This expanded string is then sent to the +database. It's that easy, no special magic. The only thing you must +ensure is that your template expands to a proper SQL statement and +that this statement matches your database design.</P> +<P CLASS="western"><SPAN LANG="en-US">Does that mean you need to +create database schema yourself and also must fully understand +rsyslogd's properties? No, that's not needed. Because we anticipated +that folks are probably more interested in getting things going +instead of designing them from scratch. So we have provided a default +schema as well as build-in support for it. This schema also offers an +additional benefit: rsyslog is part of </SPAN><A HREF="http://www.adiscon.com/en/">Adiscon</A><SPAN LANG="en-US">'s +</SPAN><A HREF="http://www.monitorware.com/en/">MonitorWare product +line</A><SPAN LANG="en-US"> (which includes open source and closed +source members). All of these tools share the same default schema and +know how to operate on it. For this reason, the default schema is +also called the "MonitorWare Schema". If you use it, you +can simply add </SPAN><A HREF="http://www.phplogcon.org/">phpLogCon, +a GPLed syslog web interface</A><SPAN LANG="en-US">, to your system +and have instant interactive access to your database. So there are +some benefits in using the provided schema.</SPAN></P> +<P LANG="en-US" CLASS="western">The schema definition is contained in +the file "createDB.sql". It comes with the rsyslog package +and one can be found for each supported database type (in the plugins +directory). Review it to check that the database name is acceptable +for you. Be sure to leave the table and field names unmodified, +because otherwise you need to customize rsyslogd's default sql +template, which we do not do in this paper. Then, run the script with +your favorite SQL client. Double-check that the table was +successfully created.</P> +<P LANG="en-US" CLASS="western">It is important to note that the +correct database encoding must be used so that the database will +accept strings independend of the string encoding. This is an +important part because it can not be guarantied that all syslog +messages will have a defined character encoding. This is especially +true if the rsyslog-Server will collect messages from different +clients and different products. +</P> +<P LANG="en-US" CLASS="western">For example PostgreSQL may refuse to +accept messages if you would set the database encoding to “UTF8†+while a client is sending invalid byte sequences for that encoding. +</P> +<P LANG="en-US" CLASS="western">Database support in rsyslog is +integrated via loadable plugin modules. To use the database +functionality, the database plugin must be enabled in the config file +BEFORE the first database table action is used. This is done by +placing the</P> +<BLOCKQUOTE CLASS="western"><CODE>$ModLoad ommysql</CODE></BLOCKQUOTE> +<P CLASS="western">directive at the begining of /etc/rsyslog.conf for +MySQL and</P> +<BLOCKQUOTE CLASS="western"><CODE>$ModLoad ompgsql</CODE></BLOCKQUOTE> +<P CLASS="western"><CODE><FONT FACE="Arial, sans-serif">for +PostgreSQL.</FONT></CODE></P> +<P LANG="en-US" CLASS="western"><FONT FACE="Arial, sans-serif">For +other databases, use their plugin name (e.g. omoracle).</FONT></P> +<P CLASS="western">Next, we need to tell rsyslogd to write data to +the database. As we use the default schema, we do NOT need to define +a template for this. We can use the hardcoded one (rsyslogd handles +the proper template linking). So all we need to do e.g. for MySQL is +add a simple selector line to /etc/rsyslog.conf:</P> +<BLOCKQUOTE CLASS="western"><CODE>*.* +:ommysql:database-server,database-name,database-userid,database-password</CODE></BLOCKQUOTE> +<P CLASS="western">Again, other databases have other selector names, +e.g. ":ompgsql:" instead of ":ommysql:". See the +output plugin's documentation for details.</P> +<P LANG="en-US" CLASS="western">In many cases, the database will run +on the local machine. In this case, you can simply use "127.0.0.1" +for <I>database-server</I>. This can be especially advisable, if you +do not need to expose the database to any process outside of the +local machine. In this case, you can simply bind it to 127.0.0.1, +which provides a quite secure setup. Of course, rsyslog also supports +remote database instances. In that case, use the remote server name +(e.g. mydb.example.com) or IP-address. The <I>database-name</I> by +default is "Syslog". If you have modified the default, use +your name here. <I>Database-userid</I> and <I>-password</I> are the +credentials used to connect to the database. As they are stored in +clear text in rsyslog.conf, that user should have only the least +possible privileges. It is sufficient to grant it INSERT privileges +to the systemevents table, only. As a side note, it is strongly +advisable to make the rsyslog.conf file readable by root only - if +you make it world-readable, everybody could obtain the password (and +eventually other vital information from it). In our example, let's +assume you have created a database user named "syslogwriter" +with a password of "topsecret" (just to say it bluntly: +such a password is NOT a good idea...). If your database is on the +local machine, your rsyslog.conf line might look like in this sample:</P> +<BLOCKQUOTE CLASS="western"><CODE>*.* +:ommysql:127.0.0.1,Syslog,syslogwriter,topsecret</CODE></BLOCKQUOTE> +<P CLASS="western">Save rsyslog.conf, restart rsyslogd - and you +should see syslog messages being stored in the "systemevents" +table!</P> +<P LANG="en-US" CLASS="western">The example line stores every message +to the database. Especially if you have a high traffic volume, you +will probably limit the amount of messages being logged. This is easy +to accomplish: the "write database" action is just a +regular selector line. As such, you can apply normal selector-line +filtering. If, for example, you are only interested in messages from +the mail subsystem, you can use the following selector line:</P> +<BLOCKQUOTE CLASS="western"><CODE>mail.* :ommysql:127.0.0.1,syslog,syslogwriter,topsecret</CODE></BLOCKQUOTE> +<P CLASS="western">Review the <A HREF="http://www.rsyslog.com/doc-rsyslog_conf.html">rsyslog.conf</A> +documentation for details on selector lines and their filtering.</P> +<P CLASS="western"><SPAN LANG="en-US"><B>You have now completed +everything necessary to store syslog messages to the a database.</B></SPAN><SPAN LANG="en-US"> +If you would like to try out a front-end, you might want to look at +</SPAN><A HREF="http://www.phplogcon.org/">phpLogCon</A><SPAN LANG="en-US">, +which displays syslog data in a browser. As of this writing, +phpLogCon is not yet a powerful tool, but it's open source, so it +might be a starting point for your own solution.</SPAN></P> +<H2 CLASS="western">On Reliability...</H2> +<P LANG="en-US" CLASS="western">Rsyslogd writes syslog messages +directly to the database. This implies that the database must be +available at the time of message arrival. If the database is offline, +no space is left or something else goes wrong - rsyslogd can not +write the database record. If rsyslogd is unable to store a message, +it performs one retry. This is helpful if the database server was +restarted. In this case, the previous connection was broken but a +reconnect immediately succeeds. However, if the database is down for +an extended period of time, an immediate retry does not help.</P> +<P CLASS="western"><SPAN LANG="en-US">Message loss in this scenario +can easily be prevented with rsyslog. All you need to do is run the +database writer in queued mode. This is now described in a generic +way and I do not intend to duplicate it here. So please be sure to +read "</SPAN><A HREF="http://www.rsyslog.com/doc-rsyslog_high_database_rate.html">Handling +a massive syslog database insert rate with Rsyslog</A><SPAN LANG="en-US">", +which describes the scenario and also includes configuration +examples.</SPAN></P> +<H2 CLASS="western">Conclusion</H2> +<P LANG="en-US" CLASS="western">With minimal effort, you can use +rsyslogd to write syslog messages to a database. You can even make it +absolutely fail-safe and protect it against database server downtime. +Once the messages are arrived there, you can interactively review and +analyze them. In practice, the messages are also stored in text files +for longer-term archival and the databases are cleared out after some +time (to avoid becoming too slow). If you expect an extremely high +syslog message volume, storing it in real-time to the database may +outperform your database server. In such cases, either filter out +some messages or used queued mode (which in general is recommended +with databases).</P> +<P LANG="en-US" CLASS="western">The method outlined in this paper +provides an easy to setup and maintain solution for most use cases.</P> +<H3 CLASS="western">Feedback Requested</H3> +<P CLASS="western">I would appreciate feedback on this paper. If you +have additional ideas, comments or find bugs, please <A HREF="mailto:rgerhards@adiscon.com">let +me know</A>.</P> +<H2 CLASS="western">References and Additional Material</H2> +<UL> + <LI><P CLASS="western" STYLE="margin-bottom: 0in"><A HREF="http://www.rsyslog.com/">www.rsyslog.com</A> + - the rsyslog site + </P> + <LI><P CLASS="western"><A HREF="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">Paper + on Syslog Server Optimization</A> + </P> +</UL> +<H2 CLASS="western">Revision History</H2> +<UL> + <LI><P CLASS="western" STYLE="margin-bottom: 0in">2005-08-02 * + <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer + Gerhards</A> * initial version created + </P> + <LI><P CLASS="western" STYLE="margin-bottom: 0in">2005-08-03 * + <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer + Gerhards</A> * added references to demo site + </P> + <LI><P CLASS="western" STYLE="margin-bottom: 0in">2007-06-13 * + <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer + Gerhards</A> * removed demo site - was torn down because too + expensive for usage count + </P> + <LI><P CLASS="western" STYLE="margin-bottom: 0in">2008-02-21 * + <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer + Gerhards</A> * updated reliability section, can now be done with + on-demand disk queues</P> + <LI><P CLASS="western">2008-02-28 * <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer + Gerhards</A> * added info on other databases, updated syntax to more + recent one + </P> + <LI><P CLASS="western">2010-01-29 * Marc Schiffbauer * added some + PostgreSQL stuff, made wording more database generic, fixed some + typos</P> +</UL> +<H2 CLASS="western">Copyright</H2> +<P CLASS="western">Copyright (c) 2005-2010 <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer +Gerhards</A>, Marc Schiffbauer and <A HREF="http://www.adiscon.com/en/">Adiscon</A>.</P> +<P CLASS="western"><BR><BR> +</P> +</BODY> +</HTML>
\ No newline at end of file diff --git a/doc/rsyslog_php_syslog_ng.html b/doc/rsyslog_php_syslog_ng.html index bf48a1eb..ed4d72fc 100644 --- a/doc/rsyslog_php_syslog_ng.html +++ b/doc/rsyslog_php_syslog_ng.html @@ -7,8 +7,10 @@ <P><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> (2005-08-04)</i></small></P> -<p><b>Note: it has been reported that this guide is somewhat outdated. Please -use with care. </b></p> +<p>Note: it has been reported that this guide is somewhat outdated. Please +use with care. Also, please note that <b>rsyslog's "native" web frontend is +<a href="http://www.phplogcon.org">phpLogCon</a></b>, which provides best integration +and a lot of extra functionality.</p> <h2>Abstract</h2> <p><i><b>In this paper, I describe how to use <a href="http://www.vermeer.org/projects/php-syslog-ng">php-syslog-ng</a> with @@ -116,11 +118,11 @@ those unfamiliar with syslog-ng, this configuration is probably easier to set up then switching to syslog-ng. For existing rsyslogd users, php-syslog-ng might be a nice add-on to their logging infrastructure.</P> <P>Please note that the <a href="http://www.monitorware.com/en/">MonitorWare family</a> (to which rsyslog belongs) also -offers a web-interface: <a href="http://www.phplogcon.org/">phpLogCon</a>. At the time of this writing, phpLogCon's code -is by far not as clean as I would like it to be. Also the user-interface is -definitely not as intutive as pp-syslog-ng. From a functionality point of view, -however, I think it already is a bit ahead. So you might -consider using it. I have set up a <a href="http://demo.rsyslog.com/">demo server</a>., +offers a web-interface: <a href="http://www.phplogcon.org/">phpLogCon</a>. +From my point of view, obviously, <b>phpLogCon is the more natural choice for a web interface +to be used together with rsyslog</b>. It also offers superb functionality and provides, +for example,native display of Windows event log entries. +I have set up a <a href="http://demo.phplogcon.org/">demo server</a>., You can have a peek at it without installing anything.</P> <h2>Feedback Requested</h2> diff --git a/doc/src/rsyslog_pgsql.odt b/doc/src/rsyslog_pgsql.odt Binary files differnew file mode 100644 index 00000000..5034c5fb --- /dev/null +++ b/doc/src/rsyslog_pgsql.odt diff --git a/doc/tls_cert_machine.html b/doc/tls_cert_machine.html index 5ecde0d1..095e15c2 100644 --- a/doc/tls_cert_machine.html +++ b/doc/tls_cert_machine.html @@ -75,7 +75,15 @@ Locality name: <font color="red">Somewhere</font> State or province name: <font color="red">CA</font> Common name: <font color="red">machine.example.net</font> UID: -Enter a challenge password: +Enter a dnsName of the subject of the certificate: +Enter the IP address of the subject of the certificate: +Enter the e-mail of the subject of the certificate: +Enter a challange password: +Does the certificate belong to an authority? (y/N): <font color="red">n</font> +Will the certificate be used for signing (DHE and RSA-EXPORT ciphersuites)? (y/N): +Will the certificate be used for encryption (RSA ciphersuites)? (y/N): +Is this a TLS web client certificate? (y/N): <font color="red">y</font> +Is this also a TLS web server certificate? (y/N): <font color="red">y</font> [root@rgf9dev sample]# <font color="red">certtool --generate-certificate --load-request request.pem --outfile cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem</font> Generating a signed certificate... Enter the certificate's serial number (decimal): @@ -86,10 +94,12 @@ The certificate will expire in (days): 1000 Extensions. +Do you want to honour the extensions from the request? (y/N): Does the certificate belong to an authority? (Y/N): <font color="red">n</font> Is this a TLS web client certificate? (Y/N): <font color="red">y</font> Is this also a TLS web server certificate? (Y/N): <font color="red">y</font> Enter the dnsName of the subject of the certificate: <font color="red">machine.example.net</font> <i>{This is the name of the machine that will use the certificate}</i> +Enter the IP address of the subject of certificate: Will the certificate be used for signing (DHE and RSA-EXPORT ciphersuites)? (Y/N): Will the certificate be used for encryption (RSA ciphersuites)? (Y/N): X.509 Certificate Information: diff --git a/doc/v4compatibility.html b/doc/v4compatibility.html new file mode 100644 index 00000000..5d877af1 --- /dev/null +++ b/doc/v4compatibility.html @@ -0,0 +1,77 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Compatibility notes for rsyslog v4</title> +</head> +<body> +<h1>Compatibility Notes for rsyslog v4</h1> +<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> +(2009-07-15)</i></small></p> +<p>The changes introduced in rsyslog v4 are numerous, but not very intrusive. +This document describes things to keep in mind when moving from v3 to v4. It +does not list enhancements nor does it talk about compatibility concerns introduced +by v3 (for this, see the <a href="v3compatibility.html">rsyslog v3 compatibility notes</a>). +<h2>HUP processing</h2> +<p>With v3 and below, rsyslog used the traditional HUP behaviour. That meant that +all output files are closed and the configuration file is re-read and the new configuration +applied. +<p>With a program as simple and static as sysklogd, this was not much of an issue. The +most important config settings (like udp reception) of a traditional syslogd can not be +modified via the configuration file. So a config file reload only meant setting up a new set of filters. It also didn't account as problem that while doing so messages may be lost - without +any threading and queuing model, a traditional syslogd will potentially always loose +messages, so it is irrelevant if this happens, too, during the short config re-read +phase. +<p>In rsyslog, things are quite different: the program is more or less a framework into +which loadable modules are loaded as needed for a particular configuration. The software +that will acutally be running is taylored via the config file. Thus, a re-read of +the config file requires a full, very heavy restart, because the software acutally +running with the new config can be totally different from what ran with the old config. +<p>Consequently, the traditional HUP is a very heavy operation and may even cause some +data loss because queues must be shut down, listeners stopped and so on. Some of these +operations (depending on their configuration) involve intentional message loss. The operation +also takes up a lot of system resources and needs quite some time (maybe seconds) to be +completed. During this restart period, the syslog subsytem is not fully available. +<p>From the software developer's point of view, the full restart done by a HUP is rather complex, +especially if user-timeout limits set on action completion are taken into consideration (for +those in the know: at the extreme ends this means we need to cancel threads as a last resort, +but than we need to make sure that such cancellation does not happen at points where it +would be fatal for a restart). A regular restart, where the process is actually terminated, is +much less complex, because the operating system does a full cleanup after process termination, +so rsyslogd does not need to take care for exotic cleanup cases and leave that to the OS. +In the end result, restart-type HUPs clutter the code, increase complexity (read: add bugs) +and cost performance. +<p>On the contrary, a HUP is typically needed for log rotation, and the real desire is +to close files. This is a non-disruptive and very lightweigth operation. +<p>Many people have said that they are used to HUP the syslogd to apply configuration +changes. This is true, but it is questionable if that really justifies all the cost that +comes with it. After all, it is the difference between typing +<pre> +$ kill -HUP `cat /var/run/rsyslogd.pid` +</pre> +versus +<pre> +$ /etc/init.d/rsyslog restart +</pre> +Semantically, both is mostly the same thing. The only difference is that with the restart +command rsyslogd can spit config error message to stderr, so that the user is able to see +any problems and fix them. With a HUP, we do not have access to stderr and thus can log +error messages only to their configured destinations; exprience tells that most users +will never find them there. What, by the way, is another strong argument against +restarting rsyslogd by HUPing it. +<p>So a restart via HUP is not strictly necessary +and most other deamons require that a restart command is typed in if a restart is required. +<p>Rsyslog will follow this paradigm in the next versions, resulting in many benefits. In v4, +we provide some support for the old-style semantics. We introduced a setting $HUPisRestart +which may be set to "on" (tradional, heavy operationg) +or "off" (new, lightweight "file close only" operation). +The initial versions had the default set to traditional behavior, but starting with 4.5.1 +we are now using the new behavior as the default. +<p>Most importantly, <b>this may break some scripts</b>, but my sincere belief is that +there are very few scripts that automatically <b>change</b> rsyslog's config and then do a +HUP to reload it. Anyhow, if you have some of these, it may be a good idea to change +them now instead of turning restart-type HUPs on. Other than that, one mainly needs +to change the habit of how to restart rsyslog after a configuration change. +<p><b>Please note that restart-type HUP is depricated and will go away in rsyslog v5.</b> +So it is a good idea to become ready for the new version now and also enjoy some of the +benefits of the "real restart", like the better error-reporting capability. +<p>Note that code complexity reduction (and thus performance improvement) needs the restart-type +HUP code to be removed, so these changes can (and will) only happen in version 5. +</body></html> |