diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/history.html | 22 | ||||
| -rw-r--r-- | doc/manual.html | 2 | ||||
| -rw-r--r-- | doc/omelasticsearch.html | 177 | ||||
| -rw-r--r-- | doc/omfile.html | 2 | ||||
| -rw-r--r-- | doc/omfwd.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_basic_structure.html | 2 | ||||
| -rw-r--r-- | doc/rsyslog_conf_modules.html | 1 |
7 files changed, 202 insertions, 6 deletions
diff --git a/doc/history.html b/doc/history.html index a06aaf5d..57b64004 100644 --- a/doc/history.html +++ b/doc/history.html @@ -118,7 +118,25 @@ use case. On February, 28th rsyslog 3.12.0 was released, the first version to contain expression support. This also meant that rsyslog from that date on supported all syslog-ng major features, but had a number of major features exlusive to it. With 3.12.0, I consider -rsyslog fully superior to syslog-ng (except for platform support).</p><p>Be sure to visit Rainer's <a href="http://rgerhards.blogspot.com/">syslog blog</a> +rsyslog fully superior to syslog-ng (except for platform support).</p> + +<p>Following the Fedora Developer's conference in Brno <b>2012</b>, rsyslog +got very serious on implementing <b>structured logging</b> in +project Lumberjack (CEE) style. Project Lumberjack was a much broader +effort and brought closer collaboration with the syslog-ng folks, which +helped to maintain and improve interoperability. In the +<b>late winter/spring/summer 2012</b> timeframe numerous engine enhancements +were made and plugins written (among them the first "official" interfaces +to the Linux audit subsystem). At the end of the year, this culminated in the +rsyslog 7, which not only implemented Lumberjack but also was the first one +to support full condition nesting in rsyslog.conf (and a ton of other features as +well). + +<p>In <b>spring 2013</b> major new security features were engineered, +namely anonymization support, as well as log file signing and +encryption capabilities. + +<p>Be sure to visit Rainer's <a href="http://rgerhards.blogspot.com/">syslog blog</a> to get some more insight into the development and futures of rsyslog and syslog in general. Don't be shy to post to either the blog or the <a href="http://www.rsyslog.com/PNphpBB2.phtml">rsyslog forums</a>.</p> @@ -126,4 +144,4 @@ Don't be shy to post to either the blog or the <ul> <li><a href="http://www.rsyslog.com/Topic4.phtml">the rsyslog change log</a></li> </ul> -</body></html>
\ No newline at end of file +</body></html> diff --git a/doc/manual.html b/doc/manual.html index a1e39c9d..5a74c8c4 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ professional services</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 7.3.10 (devel branch) of rsyslog.</b> +<p><b>This documentation is for version 7.3.14 (beta branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might diff --git a/doc/omelasticsearch.html b/doc/omelasticsearch.html new file mode 100644 index 00000000..618b7065 --- /dev/null +++ b/doc/omelasticsearch.html @@ -0,0 +1,177 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta content="en" http-equiv="Content-Language" /> + <title>Elasticsearch Output Module</title> + </head> + <body> + <p> + <a href="rsyslog_conf_modules.html">back</a></p> + <h1> + Elasticsearch Output Module</h1> + <p> + <b>Module Name: omelasticsearch</b></p> + <p> + <b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> + <p> + <b>Available since: </b>6.4.0+</p> + <p> + <b>Description</b>:</p> + <p> + This module provides native support for logging to <a href="http://www.elasticsearch.org/">Elasticsearch</a>.</p> + <p> + <b>Action Parameters</b>:</p> + <ul> + <li> + <b>server</b><br /> + Host name or IP address of the Elasticsearch server. Defaults to "localhost"</li> + <li> + <b>serverport</b><br /> + HTTP port to connect to Elasticsearch. Defaults to 9200</li> + <li> + <b>searchIndex</b><br /> + <a href="http://www.elasticsearch.org/guide/appendix/glossary.html#index">Elasticsearch index</a> to send your logs to. Defaults to "system"</li> + <li> + <b>dynSearchIndex </b><on/<b>off</b>><br /> + Whether the string provided for <strong>searchIndex</strong> should be taken as a <a href="http://www.rsyslog.com/doc/rsyslog_conf_templates.html">template</a>. Defaults to "off", which means the index name will be taken literally. Otherwise, it will look for a template with that name, and the resulting string will be the index name. For example, let's assume you define a template named "date-days" containing "%timereported:1:10:date-rfc3339%". Then, with dynSearchIndex="on", if you say searchIndex="date-days", each log will be sent to and index named after the first 10 characters of the timestamp, like "2013-03-22".</li> + <li> + <b>searchType</b><br /> + <a href="http://www.elasticsearch.org/guide/appendix/glossary.html#type">Elasticsearch type</a> to send your index to. Defaults to "events"</li> + <li> + <b>dynSearchType</b> <on/<strong>off</strong>><br /> + Like <strong>dynSearchIndex</strong>, it allows you to specify a <a href="http://www.rsyslog.com/doc/rsyslog_conf_templates.html">template</a> for <strong>searchType</strong>, instead of a static string.</li> + <li> + <strong>asyncrepl </strong><on/<strong>off</strong>><br /> + By default, an indexing operation returns after all <a href="http://www.elasticsearch.org/guide/appendix/glossary.html#replica_shard">replica shards</a> have indexed the document. With asyncrepl="on" it will return after it was indexed on the <a href="http://www.elasticsearch.org/guide/appendix/glossary.html#primary_shard">primary shard</a> only - thus trading some consistency for speed.</li> + <li> + <strong>timeout</strong><br /> + How long Elasticsearch will wait for a primary shard to be available for indexing your log before sending back an error. Defaults to "1m".</li> + <li> + <strong>template</strong><br /> + This is the JSON document that will be indexed in Elasticsearch. The resulting string needs to be a valid JSON, otherwise Elasticsearch will return an error. Defaults to:</li> + </ul> + <pre> +$template JSONDefault, "{\"message\":\"%msg:::json%\",\"fromhost\":\"%HOSTNAME:::json%\",\"facility\":\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-text%\",\"timereported\":\"%timereported:::date-rfc3339%\",\"timegenerated\":\"%timegenerated:::date-rfc3339%\"}" +</pre> + <p> + Which will produce this sort of documents (pretty-printed here for readability):</p> + <ul> + </ul> + <pre> +{ + "message": " this is a test message", + "fromhost": "test-host", + "facility": "user", + "priority": "info", + "timereported": "2013-03-12T18:05:01.344864+02:00", + "timegenerated": "2013-03-12T18:05:01.344864+02:00" +}</pre> + <ul> + <li> + <strong>bulkmode </strong><on/<strong>off</strong>><br /> + The default "off" setting means logs are shipped one by one. Each in its own HTTP request, using the <a href="http://www.elasticsearch.org/guide/reference/api/index_.html">Index API</a>. Set it to "on" and it will use Elasticsearch's <a href="http://www.elasticsearch.org/guide/reference/api/bulk.html">Bulk API</a> to send multiple logs in the same request. The maximum number of logs sent in a single bulk request depends on your queue settings - usually limited by the <a href="http://www.rsyslog.com/doc/node35.html">dequeue batch size</a>. More information about queues can be found <a href="http://www.rsyslog.com/doc/node32.html">here</a>.</li> + <li> + <strong>parent</strong><br /> + Specifying a string here will index your logs with that string the parent ID of those logs. Please note that you need to define the <a href="http://www.elasticsearch.org/guide/reference/mapping/parent-field.html">parent field</a> in your <a href="http://www.elasticsearch.org/guide/reference/mapping/">mapping</a> for that to work. By default, logs are indexed without a parent.</li> + <li> + <strong>dynParent </strong><on/<strong>off</strong>><br /> + Using the same parent for all the logs sent in the same action is quite unlikely. So you'd probably want to turn this "on" and specify a <a href="http://www.rsyslog.com/doc/rsyslog_conf_templates.html">template</a> that will provide meaningful parent IDs for your logs.</li> + <li> + <strong>uid</strong><br /> + If you have basic HTTP authentication deployed (eg: through the <a href="https://github.com/Asquera/elasticsearch-http-basic">elasticsearch-basic plugin</a>), you can specify your user-name here.</li> + <li> + <strong>pwd</strong><br /> + Password for basic authentication.</li> + </ul> + <p> + <b>Samples:</b></p> + <p> + The following sample does the following:</p> + <ul> + <li> + loads the omelasticsearch module</li> + <li> + outputs all logs to Elasticsearch using the default settings</li> + </ul> + <pre> +module(load="omelasticsearch") +*.* action(type="omelasticsearch")</pre> + <p> + The following sample does the following:</p> + <ul> + <li> + loads the omelasticsearch module</li> + <li> + defines a template that will make the JSON contain the following properties (more info about what properties you can use <a href="http://www.rsyslog.com/doc/property_replacer.html">here</a>): + <ul> + <li> + RFC-3339 timestamp when the event was generated</li> + <li> + the message part of the event</li> + <li> + hostname of the system that generated the message</li> + <li> + severity of the event, as a string</li> + <li> + facility, as a string</li> + <li> + the tag of the event</li> + </ul> + </li> + <li> + outputs to Elasticsearch with the following settings + <ul> + <li> + host name of the server is myserver.local</li> + <li> + port is 9200</li> + <li> + JSON docs will look as defined in the template above</li> + <li> + index will be "test-index"</li> + <li> + type will be "test-type"</li> + <li> + activate bulk mode. For that to work effectively, we use an in-memory queue that can hold up to 5000 events. The maximum bulk size will be 300</li> + <li> + retry indefinitely if the HTTP request failed (eg: if the target server is down)</li> + </ul> + </li> + </ul> + <pre> +module(load="omelasticsearch") +template(name="testTemplate" + type="list" + option.json="on") { + constant(value="{") + constant(value="\"timestamp\":\"") property(name="timereported" dateFormat="rfc3339") + constant(value="\",\"message\":\"") property(name="msg") + constant(value="\",\"host\":\"") property(name="hostname") + constant(value="\",\"severity\":\"") property(name="syslogseverity-text") + constant(value="\",\"facility\":\"") property(name="syslogfacility-text") + constant(value="\",\"syslogtag\":\"") property(name="syslogtag") + constant(value="\"}") + } +*.* action(type="omelasticsearch" + server="myserver.local" + serverport="9200" + template="testTemplate" + searchIndex="test-index" + searchType="test-type" + bulkmode="on" + queue.type="linkedlist" + queue.size="5000" + queue.dequeuebatchsize="300" + action.resumeretrycount="-1")</pre> + <p> + </p> + <pre> +</pre> + <p> + [<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> + <p> + <font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br /> + Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the ASL 2.0.</font></p> + </body> +</html> + diff --git a/doc/omfile.html b/doc/omfile.html index 06b738bb..3966ab13 100644 --- a/doc/omfile.html +++ b/doc/omfile.html @@ -28,7 +28,7 @@ <li><strong>ZipLevel </strong>0..9 [default 0]<br> if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> - <li><b>VeryReliableZip</b> [<b>on</b>/off] (v7.3.0+) - if ZipLevel is greater 0, + <li><b>VeryRobustZip</b> [<b>on</b>/off] (v7.3.0+) - if ZipLevel is greater 0, then this setting controls if extra headers are written to make the resulting file extra hardened against malfunction. If set to off, data appended to previously unclean closed files may not be accessible without extra tools. diff --git a/doc/omfwd.html b/doc/omfwd.html index fb9145e1..53f9e527 100644 --- a/doc/omfwd.html +++ b/doc/omfwd.html @@ -59,7 +59,7 @@ <p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> <p><b>Sample:</b></p> <p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> -<textarea rows="5" cols="60">Module (path="builtin:omfwd") +<textarea rows="5" cols="60">Module (load="builtin:omfwd") *.* action(type="omfwd" Target="192.168.2.11" Port="10514" diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html index 00a700d4..f5d4891a 100644 --- a/doc/rsyslog_conf_basic_structure.html +++ b/doc/rsyslog_conf_basic_structure.html @@ -80,7 +80,7 @@ A ruleset can be "bound" (assigned) to a specific input. In the analogy, this me a message comes in via that input, the "program" (ruleset) bound to it will be executed (but not any other!). <p>There is detail documentation available for -<a href="multi_ruleset">rsyslog rulesets</a>. +<a href="multi_ruleset.html">rsyslog rulesets</a>. <p>For quick reference, rulesets are defined as follows: <pre> ruleset(name="rulesetname") { diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index 554b20f4..2668bf2d 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -74,6 +74,7 @@ permits rsyslog to alert folks by mail if something important happens</li> <li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li> <li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li> <li><a href="ommongodb.html">ommongodb</a> - output module for MongoDB</li> +<li><a href="omelasticsearch.html">omelasticsearch</a> - output module for ElasticSearch</li> </ul> <a name="pm"></a><h2>Parser Modules</h2> |