7 files changed, 169 insertions, 27 deletions
diff --git a/doc/Makefile.am b/doc/Makefile.am
index a7417e44..9a7ed162 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -122,6 +122,9 @@ html_files = \
 	v3compatibility.html \
 	v4compatibility.html \
 	v5compatibility.html \
+	v6compatibility.html \
+	v7compatibility.html \
+	v8compatibility.html \
 	im3195.html \
 	netstream.html \
 	ns_gtls.html \
@@ -158,8 +161,6 @@ html_files = \
 	rsconf1_omfileforcechown.html \
 	rsyslog_queue_pointers.jpeg \
 	rsyslog_queue_pointers2.jpeg \
-	v6compatibility.html \
-	v7compatibility.html \
 	rsyslog_conf_basic_structure.html \
 	rsyslog_conf_sysklogd_compatibility.html \
 	imkmsg.html \
diff --git a/doc/manual.html b/doc/manual.html
index 7e24ea7d..017ece24 100644
--- a/doc/manual.html
+++ b/doc/manual.html
@@ -19,7 +19,7 @@ professional services</a> available directly from the source!</p>
 <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a>
 to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the
 project goals.</p>
-<p><b>This documentation is for version 7.5.7 (devel branch) of rsyslog.</b>
+<p><b>This documentation is for version 8.1.2 (devel branch) of rsyslog.</b>
 Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b>
 to obtain current version information and project status.
 </p><p><b>If you like rsyslog, you might
@@ -27,16 +27,18 @@ want to lend us a helping hand. </b>It doesn't require a lot of
 time - even a single mouse click helps. Learn <a href="how2help.html">how to help the rsyslog project</a>.
 Due to popular demand, there is now a <a href="rsyslog_ng_comparison.html">side-by-side comparison
 between rsyslog and syslog-ng</a>.</p>
-<p>If you are upgrading from rsyslog v2 or stock sysklogd,
-<a href="v3compatibility.html">be sure to read the rsyslog v3 compatibility notes</a>,
-and if you are upgrading from v3, read the
-<a href="v4compatibility.html">rsyslog v4 compatibility notes</a>,
-if you upgrade from v4, read the
-<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and
-if you upgrade from v5, read the
-<a href="v6compatibility.html">rsyslog v6 compatibility notes</a>.
+If you upgrade from v7, read the
+<a href="v8compatibility.html">rsyslog v8 compatibility notes</a>.
 if you upgrade from v6, read the
 <a href="v7compatibility.html">rsyslog v7 compatibility notes</a>.
+if you upgrade from v5, read the
+<a href="v6compatibility.html">rsyslog v6 compatibility notes</a>.
+if you upgrade from v4, read the
+<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>.
+if you upgrade from v3, read the
+<a href="v4compatibility.html">rsyslog v4 compatibility notes</a>.
+<p>If you are upgrading from rsyslog v2 or stock sysklogd,
+<a href="v3compatibility.html">be sure to read the rsyslog v3 compatibility notes</a>,
 <p>Rsyslog will work even
 if you do not read the doc, but doing so will definitely improve your experience.</p>
 <p><b>Follow the links below for the</b></p>
@@ -44,7 +46,7 @@ if you do not read the doc, but doing so will definitely improve your experience
 <li><a href="troubleshoot.html">troubleshooting rsyslog problems</a></li>
 <li><a href="rsyslog_conf.html">configuration file format (rsyslog.conf)</a></li>
 <li><a href="http://www.rsyslog.com/tool-regex">a regular expression checker/generator tool for rsyslog</a></li>
-<li> <a href="property_replacer.html">property replacer, an important core component</a></li>
+<li><a href="property_replacer.html">property replacer, an important core component</a></li>
 <li><a href="bugs.html">rsyslog bug list</a></li>
 <li><a href="messageparser.html">understanding rsyslog message parsers</a></li>
 <li><a href="generic_design.html">backgrounder on generic syslog application design</a></li>
diff --git a/doc/mmnormalize.html b/doc/mmnormalize.html
index fc6ec6d2..f09d0993 100644
--- a/doc/mmnormalize.html
+++ b/doc/mmnormalize.html
@@ -38,6 +38,11 @@ can be found in the <a href="http://www.liblognorm.com/files/manual/index.html">
 <li><b>useRawMsg</b> [boolean]<br>
 Specifies if the raw message should be used for normalization (on) or just the
 MSG part of the message (off). Default is "off".
+<li><b>path</b> [word], defaults to "$!"<br>
+Specifies the JSON path under which parsed elements should be placed. By default,
+all parsed properties are merged into root of message properties. You can place them
+under a subtree, instead. You can place them in local variables, also, by setting
+path="$.".
 </ul>
 <p><b>Legacy Configuration Directives</b>:</p>
 <ul>
diff --git a/doc/multi_ruleset.html b/doc/multi_ruleset.html
index 14a761c5..1be2c81e 100644
--- a/doc/multi_ruleset.html
+++ b/doc/multi_ruleset.html
@@ -89,6 +89,15 @@ modify the ruleset to which the next input is bound but rather provides a system
 default rule set for those inputs that did not explicitly bind to one. As such, the directive
 can not be used as a work-around to bind inputs to non-default rulesets that do not support
 ruleset binding.
+
+<h2>Rulesets and Main Queues</h2>
+By default, rulesets do not have their own queue. It must be activated via
+the $RulesetCreateMainQueue directive, or if using rainerscript format, by
+specifying queue parameters on the ruleset directive, e.g.
+ruleset(name="whatever" queue.type="fixedArray" queue. ...)
+See <a href="http://www.rsyslog.com/doc/queue_parameters.html">http://www.rsyslog.com/doc/queue_parameters.html</a>
+for more details. 
+
 <h2>Examples</h2>
 <h3>Split local and remote logging</h3>
 <p>Let's say you have a pretty standard system that logs its local messages to the usual
diff --git a/doc/rsyslog_conf_actions.html b/doc/rsyslog_conf_actions.html
index adca540b..260d1f2b 100644
--- a/doc/rsyslog_conf_actions.html
+++ b/doc/rsyslog_conf_actions.html
@@ -31,8 +31,14 @@ implemented via <a href="rsyslog_conf_modules.html#om">output modules</a>.
 	<br>used for statistics gathering and documentation
 	<li><b>type</b> string
 	<br>Mandatory parameter for every action. The name of the module that should be used. </li>
-	<li><b>action.writeAllMarkMessages</b> on/off
-	<br>Normally, mark messages are written to actions only if the action was not recently executed (by default, recently means within the past 20 minutes). If this setting is switched to "on", mark messages are always sent to actions, no matter how recently they have been executed. In this mode, mark messages can be used as a kind of heartbeat.</li>
+	<li><b>action.writeAllMarkMessages</b> <i>on</i>/off
+	<br>This setting tells if mark messages are always written ("on", the default) or only
+	if the action was not recently executed ("off"). By default, recently means within the
+	past 20 minutes. If this setting is "on", mark messages are always sent to actions, no
+	matter how recently they have been executed. In this mode, mark messages can be used as
+	a kind of heartbeat. This mode also enables faster processing inside the rule engine. So
+	it should be set to "off" only when there is a good reason to do so.
+	</li>
 	<li><b>action.execOnlyEveryNthTime</b> integer
 	<br>If configured, the next action will only be executed every n-th time. For example, if configured to 3, the first two messages that go into the action will be dropped, the 3rd will actually cause the action to execute, the 4th and 5th will be dropped, the 6th executed under the action, ... and so on.</li>
 	<li><b>action.execOnlyEveryNthTimeout</b> integer
diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html
index 8dc3ed56..99557215 100644
--- a/doc/rsyslog_conf_modules.html
+++ b/doc/rsyslog_conf_modules.html
@@ -53,28 +53,28 @@ to message generators.
 and messages be transmitted to various different targets.
 <ul>
 <li><a href="omfile.html">omfile</a> - file output module</li>
-<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li>
+<li><a href="omfwd.html">omfwd</a> (does NOT yet work in v8) - syslog forwarding output module</li>
 <li><a href="omjournal.html">omjournal</a> - Linux journal output module</li>
 <li><a href="ompipe.html">ompipe</a> - named pipe output module</li>
 <li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li>
-<li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li>
+<li><a href="omsnmp.html">omsnmp</a> (does NOT yet work in v8) - SNMP trap output module</li>
 <li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li>
-<li><a href="omrelp.html">omrelp</a> - RELP output module</li>
+<li><a href="omrelp.html">omrelp</a> (does NOT yet work in v8) - RELP output module</li>
 <li><a href="omruleset.html">omruleset</a> - forward message to another ruleset</li>
-<li>omgssapi - output module for GSS-enabled syslog</li>
+<li>omgssapi (does NOT yet work in v8) - output module for GSS-enabled syslog</li>
 <li><a href="ommysql.html">ommysql</a> - output module for MySQL</li>
-<li>ompgsql - output module for PostgreSQL</li>
-<li><a href="omlibdbi.html">omlibdbi</a> -
+<li>ompgsql (does NOT yet work in v8) - output module for PostgreSQL</li>
+<li><a href="omlibdbi.html">omlibdbi</a> (does NOT yet work in v8) -
 generic database output module (Firebird/Interbase, MS SQL, Sybase,
 SQLLite, Ingres, Oracle, mSQL)</li>
-<li><a href="ommail.html">ommail</a> -
+<li><a href="ommail.html">ommail</a> (does NOT yet work in v8) -
 permits rsyslog to alert folks by mail if something important happens</li>
-<li><a href="omprog.html">omprog</a> - permits sending messages to a program for custom processing</li>
-<li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li>
-<li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li>
-<li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li>
-<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li>
-<li><a href="ommongodb.html">ommongodb</a> - output module for MongoDB</li>
+<li><a href="omprog.html">omprog</a> (does NOT yet work in v8) - permits sending messages to a program for custom processing</li>
+<li><a href="omoracle.html">omoracle</a> (orphaned) - output module for Oracle (native OCI interface)</li>
+<li><a href="omudpspoof.html">omudpspoof</a> (does NOT yet work in v8) - output module sending UDP syslog messages with a spoofed address</li>
+<li><a href="omuxsock.html">omuxsock</a> (does NOT yet work in v8) - output module Unix domain sockets</li>
+<li><a href="omhdfs.html">omhdfs</a> (does NOT yet work in v8) - output module for Hadoop's HDFS file system</li>
+<li><a href="ommongodb.html">ommongodb</a> (does NOT yet work in v8) - output module for MongoDB</li>
 <li><a href="omelasticsearch.html">omelasticsearch</a> - output module for ElasticSearch</li>
 </ul>
 
diff --git a/doc/v8compatibility.html b/doc/v8compatibility.html
new file mode 100644
index 00000000..549a7fef
--- /dev/null
+++ b/doc/v8compatibility.html
@@ -0,0 +1,119 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html><head><title>Compatibility notes for rsyslog v8</title>
+</head>
+<body>
+<h1>Compatibility Notes for rsyslog v8</h1>
+This document describes things to keep in mind when moving from v7 to v8. It 
+does not list enhancements nor does it talk about compatibility concerns introduced
+by earlier versions (for this, see their respective compatibility documents). Its focus
+is primarily on what you need to know if you used v7 and want to use v8 without hassle.
+<p>Version 8 offers a completely rewritten core rsyslog engine. This resulted in
+a number of changes that are visible to users and (plugin) developers.
+Most importantly, pre-v8 plugins <b>do not longer work</b> and need to
+be updated to support the new calling interfaces. If you developed a plugin,
+be sure to review the developer section below.
+</p>
+<h2>Mark Messages</h2>
+<p>In previous versions, mark messages were by default only processed if an
+action was not executed for some time. The default has now changed, and mark
+messages are now always processed. Note that this enables faster processing
+inside rsyslog. To change to previous behaviour, you need to add
+action.writeAllMarkMessages="off" to the actions in question.
+
+<h2>Modules WITHOUT v8 Support</h2>
+<p>The following modules will not properly build under rsyslog v8 and
+need to be updated:
+<ul>
+<li>plugins/omgssapi
+<li>plugins/omhdfs
+<li>plugins/omrabbitmq - not project supported
+<li>plugins/omzmq3 - not project supported
+<li>plugins/mmrfc5424addhmac - was a custom project, requires sponsoring
+    for conversion
+<li>plugins/omoracle - orphaned since a while -- use omlibdbi instead
+<li>plugins/mmsnmptrapd - waiting for demand
+<li>plugins/mmcount - scheduled to be updated soon
+</ul>
+
+<h2>Untested Modules</h2>
+<p>The following modules have been updated and successfully
+build, but no "real" test were conducted. Users of these modules
+should use extra care.
+<ul>
+<li>mmsequence
+<li>omprog
+<li>omsnmp
+<li>mmfields
+<li>mmpstrucdata
+<li>plugins/mmaudit
+<li>omlibdbi - will be tested soon
+<li>ommongodb - larger changes still outstanding
+<li>ompgsql - larger chages still outstanding
+<li>omuxsock
+</ul>
+<p>In addition to bug reports, success reports are also appreciated for
+these modules (this may save us testing).
+
+<h2>What Developers need to Know</h2>
+<h3>output plugin interface</h3>
+<p>To support the new core engine, the output interface has been considerably
+changed. It is suggested to review some of the project-provided plugins for
+full details. In this doc, we describe the most important changes from a high
+level perspective.
+<p><b>NOTE: the v8 output module interface is not yet stable.</b> It is highly
+likely that additional changes will be made within the next weeks. So if you
+convert a module now, be prepared for additional mandatory changes.
+
+<p><b>Multi-thread awareness required</b></p>
+<p>The new engine activates one <b>worker</b>instance of output actions on 
+each worker thread. This means an action has now three types of data:
+<ul>
+<li>global
+<li>action-instance - previously known pData, one for each action inside the config
+<li>worker-action-instance - one for each worker thread (called pWrkrData), note
+that this is specific to exactly one pData
+</ul>
+The plugin <b>must</b> now by multi-threading aware. It may be called by multiple
+threads concurrently, but it is guaranteed that each call is for a unique
+pWrkrData structure. This still permits to write plugins easily, but enables the
+engine to work with much higher performance. Note that plugin developers
+should assume it is the norm that multiple concurrent worker action instances
+are active a the some time.
+
+<p><b>New required entry points</b></p>
+<p>In order to support the new threading model, new entry points are required.
+Most importantly, only the plugin knows which data must be present in pData and
+pWrkrData, so it must created and destroy these data structures on request of
+the engine. Note that pWrkrData may be destroyed at any time and new ones
+re-created later. Depending on workload structure and configuration, this can happen
+frequently.
+<p>New entry points are:
+<ul>
+<li>createWrkrInstance
+<li>freeWrkrInstance
+</ul>
+
+The calling interface for these entry points has changed. Basically,
+they now receive a pWrkrData object instead pData. It is assumed that
+createWrkrInstance populates pWrkrData-&gt;pData appropriately.
+<ul>
+<li>beginTransaction
+<li>doAction
+<li>endTransaction
+</ul>
+
+<p><b>RS_RET_SUSPENDED is no longer supported when creating an action instance</b>
+<p>This means a plugin must not try to establish any connections or the like
+before any of its processing entry points (like beginTransaction or doAction)
+is called. This was generally also the case von v7, but was not enforced in
+all cases. In v8, creating action creation fails if anything but RS_RET_OK
+is returned.
+
+<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
+
+<p><font size="2">This documentation is part of the
+<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
+Copyright &copy; 2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
+<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL 
+version 2 or higher.</font></p>
+</body></html>