diff options
Diffstat (limited to 'doc/rsyslog_conf_modules.html')
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 192 |
1 files changed, 192 insertions, 0 deletions
diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html new file mode 100644 index 00000000..c8c1c5d7 --- /dev/null +++ b/doc/rsyslog_conf_modules.html @@ -0,0 +1,192 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head><title>Modules - rsyslog.conf</title></head> +<body> +<p>This is a part of the rsyslog.conf documentation.</p> +<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> +<h1>Modules</h1> +<p>Rsyslog has a modular design. This enables functionality to be +dynamically loaded from modules, which may also be written by any +third party. Rsyslog itself offers all non-core functionality as +modules. Consequently, there is a growing +number of modules. Here is the entry point to their documentation and +what they do (list is currently not complete)</p> +<p>Please note that each module provides configuration +directives, which are NOT necessarily being listed below. Also +remember, that a modules configuration directive (and functionality) is +only available if it has been loaded (using $ModLoad).</p> +<p>It is relatively easy to write a rsyslog module. <b>If none of the provided +modules solve your need, you may consider writing one or have one written +for you by +<a href="http://www.rsyslog.com/professional-services">Adiscon's professional services for rsyslog</a> +</b>(this often is a very cost-effective and efficient way of getting what you need). +<p>There exist different classes of loadable modules: +<ul> +<li><a href="rsyslog_conf_modules.html#im">Input Modules</a> +<li><a href="rsyslog_conf_modules.html#om">Output Modules</a> +<li><a href="rsyslog_conf_modules.html#pm">Parser Modules</a> +<li><a href="rsyslog_conf_modules.html#mm">Message Modification Modules</a> +<li><a href="rsyslog_conf_modules.html#sm">String Generator Modules</a> +<li><a href="rsyslog_conf_modules.html#lm">Library Modules</a> +</ul> + +<a name"im"></a><h2>Input Modules</h2> +<p>Input modules are used to gather messages from various sources. They interface +to message generators. +<ul> +<li><a href="imfile.html">imfile</a> - input module for text files</li> +<li><a href="imrelp.html">imrelp</a> - RELP input module</li> +<li><a href="imudp.html">imudp</a> - udp syslog message input</li> +<li><a href="imtcp.html">imtcp</a> - input plugin for tcp syslog</li> +<li><a href="imptcp.html">imptcp</a> - input plugin for plain tcp syslog (no TLS but faster)</li> +<li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li> +<li>immark - support for mark messages</li> +<li><a href="imklog.html">imklog</a> - kernel logging</li> +<li><a href="imuxsock.html">imuxsock</a> - unix sockets, including the system log socket</li> +<li><a href="imsolaris.html">imsolaris</a> - input for the Sun Solaris system log source</li> +<li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li> +<li><a href="impstats.html">impstats</a> - provides periodic statistics of rsyslog internal counters</li> +<li><a href="imjournal.html">imjournal</a> - Linux journal inuput module</li> +</ul> + +<a name"om"></a><h2>Output Modules</h2> +<p>Output modules process messages. With them, message formats can be transformed +and messages be transmitted to various different targets. +<ul> +<li><a href="omfile.html">omfile</a> - file output module</li> +<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li> +<li><a href="omjournal.html">omjournal</a> - Linux journal output module</li> +<li><a href="ompipe.html">ompipe</a> - named pipe output module</li> +<li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li> +<li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> +<li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li> +<li><a href="omrelp.html">omrelp</a> - RELP output module</li> +<li><a href="omruleset.html">omruleset</a> - forward message to another ruleset</li> +<li>omgssapi - output module for GSS-enabled syslog</li> +<li><a href="ommysql.html">ommysql</a> - output module for MySQL</li> +<li>ompgsql - output module for PostgreSQL</li> +<li><a href="omlibdbi.html">omlibdbi</a> - +generic database output module (Firebird/Interbase, MS SQL, Sybase, +SQLLite, Ingres, Oracle, mSQL)</li> +<li><a href="ommail.html">ommail</a> - +permits rsyslog to alert folks by mail if something important happens</li> +<li><a href="omprog.html">omprog</a> - permits sending messages to a program for custom processing</li> +<li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li> +<li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li> +<li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li> +<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li> +<li><a href="ommongodb.html">ommongodb</a> - output module for MongoDB</li> +<li><a href="omelasticsearch.html">omelasticsearch</a> - output module for ElasticSearch</li> +</ul> + +<a name="pm"></a><h2>Parser Modules</h2> +<p>Parser modules are used to parse message content, once the message has been +received. They can be used to process custom message formats or invalidly formatted +messages. For details, please see the <a href="messageparser.html">rsyslog +message parser documentation</a>. +<p>The current modules are currently provided as part of rsyslog: +<ul> +<li>pmrfc5424[builtin] - rsyslog.rfc5424 - +parses RFC5424-formatted messages (the new syslog standard) +<li>pmrfc3164[builtin] - rsyslog.rfc3164 - +the traditional/legacy syslog parser +<li>pmrfc3164sd - rsyslog.rfc3164sd - +a contributed module supporting RFC5424 structured data inside +RFC3164 messages (not supported by the rsyslog team) +<li><a href="pmlastmsg.html">pmlastmsg</a> - rsyslog.lastmsg - +a parser module that handles the typically malformed "last messages +repated n times" messages emitted by some syslogds. +</ul> + +<a name="mm"></a><h2>Message Modification Modules</h2> +<p>Message modification modules are used to change the content of messages being processed. +They can be implemented using either the output module or the parser module interface. +From the rsyslog core's point of view, they actually are output or parser modules, it is their +implementation that makes them special. +<p>Currently, there exists only a limited set of such modules, but new ones could be written with +the methods the engine provides. They could be used, for example, to +add dynamically computed content to message (fields). +<p>Message modification modules are usually written for one specific task and thus +usually are not generic enough to be reused. However, existing module's code is +probably an excellent starting base for writing a new module. Currently, the following +modules exist inside the source tree: +<ul> +<li><a href="mmanon.html">mmanon</a> - used to anonymize log messages. +<li><a href="mmcount.html">mmcount</a> - message modification plugin which counts messages +<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages. +Note that this actually is a <b>generic</b> module. +<li><a href="mmjsonparse.html">mmjsonparse</a> - used to interpret CEE/lumberjack +enabled structured log messages. +<li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside +the tag to correct the original sender system and priority of messages. Implemented via +the output module interface. +</ul> + +<a name="lm"></a><h2>String Generator Modules</h2> +<p>String generator modules are used, as the name implies, to generate strings based +on the message content. They are currently tightly coupled with the template system. +Their primary use is to speed up template processing by providing a native C +interface to template generation. These modules exist since 5.5.6. To get an idea +of the potential speedup, the default file format, when generated by a string generator, +provides a roughly 5% speedup. For more complex strings, especially those that include +multiple regular expressions, the speedup may be considerably higher. +<p>String generator modules are written to a quite simple interface. However, a word of +caution is due: they access the rsyslog message object via a low-level interface. +That interface is not guaranteed yet to stay stable. So it may be necessary to +modify string generator modules if the interface changes. Obviously, we will not do that +without good reason, but it may happen. +<p>Rsyslog comes with a set of core, build-in string generators, which are used +to provide those default templates that we consider to be time-critical: +<ul> +<li>smfile - the default rsyslog file format +<li>smfwd - the default rsyslog (network) forwarding format +<li>smtradfile - the traditional syslog file format +<li>smfwd - the traditional syslog (network) forwarding format +</ul> +<p>Note that when you replace these defaults be some custom strings, you will +loose some performance (around 5%). For typical systems, this is not really relevant. +But for a high-performance systems, it may be very relevant. To solve that issue, create +a new string generator module for your custom format, starting out from one of the +default generators provided. If you can not do this yourself, you may want to +contact <a href="mailto:info%40adiscon.com">Adiscon</a> as we offer custom development +of string generators at a very low price. +<p>Note that string generator modules can be dynamically loaded. However, the default +ones provided are so important that they are build right into the executable. But this +does not need to be done that way (and it is straightforward to do it dynamic). + + +<a name="lm"></a><h2>Library Modules</h2> +<p>Library modules provide dynamically loadable functionality for parts of rsyslog, +most often for other loadable modules. They can not be user-configured and are loaded +automatically by some components. They are just mentioned so that error messages that +point to library moduls can be understood. No module list is provided. + +<h2>Where are the modules integrated into the Message Flow?</h2> +<p>Depending on their module type, modules may access and/or modify messages at +various stages during rsyslog's processing. Note that only the "core type" (e.g. input, +output) but not any type derived from it (message modification module) specifies when +a module is called. +<p>The simplified workflow is as follows: +<p align="center"> +<img src="module_workflow.png" alt"rsyslog: loadable modules and message flow"> +<p>As can be seen, messages are received by input modules, then passed to one or many +parser modules, which generate the in-memory representation of the message and may +also modify the message itself. The, the internal representation is passed to +output modules, which may output a message and (with the interfaces newly introduced +in v5) may also modify messageo object content. +<p>String generator modules are not included inside this picture, because they are +not a required part of the workflow. If used, they operate "in front of" the +output modules, because they are called during template generation. +<p>Note that the actual flow is much more complex and depends a lot on queue and +filter settings. This graphic above is a high-level message flow diagram. + +<p>[<a href="manual.html">manual index</a>] +[<a href="rsyslog_conf.html">rsyslog.conf</a>] +[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> +</body> +</html> + |