From 4208db41a338c372d8afb8b81ece7c3320334250 Mon Sep 17 00:00:00 2001 From: Florian Riedl Date: Mon, 24 Sep 2012 08:33:03 +0200 Subject: Fiddeling a bit with the template doc --- doc/rsyslog_conf_templates.html | 171 ++++++++++++++++++++++++++-------------- 1 file changed, 111 insertions(+), 60 deletions(-) diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html index 4360c691..affb692c 100644 --- a/doc/rsyslog_conf_templates.html +++ b/doc/rsyslog_conf_templates.html @@ -68,14 +68,24 @@ you do not have any capability to specify constant text, and as such cannot incl line breaks. As a consequence, using this template type for text outputs is usually only useful for debugging or very special cases (e.g. where the text is interpreted by a JSON parser later on). - +
Config example: +
template(name="tpl1" type="list" option.json="off") { +
constant(value="Syslog: ") +
property(name="msg" outname="mymessage") +
constant(value=" on ") +
property(name="timereported" dateFormat="rfc3339" caseConversion="lower") +
constant(value="\n") +
} +

subtree

Available since rsyslog 7.1.4

In this case, the template is generated based on a complete (CEE) subtree. This type of template is most useful for outputs that know how to process hierarchical structure, like ommongodb. With that type, the parameter - +
Config example: +

template(name="tpl2" type="subtree" subtree="$!") +

string

This closely resembles the legacy template statement. It has a mandatory parameter string, which holds the template string to be @@ -85,7 +95,9 @@ content when the final string to be passed to a plugin is generated. String-base templates are a great way to specify textual content, especially if no complex manipulation to properties is necessary. Full details on how to specify template text can be found below. - +
Config example: +

template(name="tpl3" type="list" string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n") +

plugin

In this case, the template is generated by a plugin (which is then called @@ -96,6 +108,101 @@ that "last bit" can make a difference). Refer to the plugin's documentation for further details. For this type, the paramter plugin must be specified and must contain the name of the plugin as it identifies itself. Note that the plugin must be loaded prior to being used inside a template. +
Config example: +
template(name="tpl4" type="plugin" plugin="mystrgen") +
+ +

options

+The <options> part is optional. It carries options +influencing the template as whole and is part of the template parameters. +See details below. Be sure NOT to mistake template options with property +options - the latter ones are processed by the property replacer and +apply to a SINGLE property, only (and not the whole template).
+
+Template options are case-insensitive. Currently defined are:

+

option.sql - format the string suitable for a SQL +statement in MySQL format. This will replace single quotes ("'") and +the backslash character by their backslash-escaped counterpart ("\'" +and "\\") inside each field. Please note that in MySQL configuration, +the NO_BACKSLASH_ESCAPES +mode must be turned off for this format to work (this is the default).

+

option.stdsql - format the string suitable for a +SQL statement that is to be sent to a standards-compliant sql server. +This will replace single quotes ("'") by two single quotes ("''") +inside each field. You must use stdsql together with MySQL if in MySQL +configuration the +NO_BACKSLASH_ESCAPES is +turned on.

+

option.json - format the string suitable for a +json statement. +This will replace single quotes ("'") by two single quotes ("''") +inside each field.

+

At no time, multiple template option should be used. This can cause +unpredictable behaviour and is against all logic.

+

Either the sql or stdsql  +option must be specified when a template is used +for writing to a database, otherwise injection might occur. Please note +that due to the unfortunate fact that several vendors have violated the +sql standard and introduced their own escape methods, it is impossible +to have a single option doing all the work.  So you yourself +must make sure you are using the right format. If you choose +the wrong one, you are still vulnerable to sql injection.
+
+Please note that the database writer *checks* that the sql option is +present in the template. If it is not present, the write database +action is disabled. This is to guard you against accidental forgetting +it and then becoming vulnerable to SQL injection. The sql option can +also be useful with files - especially if you want to import them into +a database on another machine for performance reasons. However, do NOT +use it if you do not have a real need for it - among others, it takes +some toll on the processing time. Not much, but on a really busy system +you might notice it ;)

+

The default template for the write to database action has the +sql option set. As we currently support only MySQL and the sql option +matches the default MySQL configuration, this is a good choice. +However, if you have turned on +NO_BACKSLASH_ESCAPES in +your MySQL config, you need to supply a template with the stdsql +option. Otherwise you will become vulnerable to SQL injection.
+
+To escape:
+% = \%
+\ = \\ --> '\' is used to escape (as in C)
+template (name="TraditionalFormat" type="string" string="%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"
+
+ +

Examples

+Here are some real life examples. They will show how to rebuild legacy templates +with current list style. +
+
FileFormat: $template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" +
template(name="FileFormat" type="list" option.json="off") { +
property(name="timestamp" dateFormat="rfc3339") +
constant(value=" ") +
property(name="hostname") +
constant(value=" ") +
property(name="syslogtag") +
constant(value=" ") +
property(name="msg" spifno1stsp="on" ) +
property(name="msg" droplastlf="on" ) +
constant(value="\n") +
} +
+ +
ForwardFormat: $template ForwardFormat,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" +
template(name="ForwardFormat" type="list" option.json="off") {, +
constant(value="<") +
property(name="PRI") +
constant(value="<") +
property(name="timestamp" dateFormat="rfc3339") +
constant(value=" ") +
property(name="hostname") +
constant(value=" ") +
property(name="syslogtag" position.from="1" position.to="32") +
constant(value=" ") +
property(name="msg" spifno1stsp="on" ) +
} +

legacy format

In pre v6-versions of rsyslog, you need to use the $template @@ -226,62 +333,7 @@ contents of the syslog message. Properties are accessed via the example, it can pick a substring or do date-specific formatting. More on this is below, on some lines of the property replacer.

-The <options> part is optional. It carries options -influencing the template as whole. See details below. Be sure NOT to -mistake template options with property options - the latter ones are -processed by the property replacer and apply to a SINGLE property, only -(and not the whole template).
-
-Template options are case-insensitive. Currently defined are:

-

option.sql - format the string suitable for a SQL -statement in MySQL format. This will replace single quotes ("'") and -the backslash character by their backslash-escaped counterpart ("\'" -and "\\") inside each field. Please note that in MySQL configuration, -the NO_BACKSLASH_ESCAPES -mode must be turned off for this format to work (this is the default).

-

option.stdsql - format the string suitable for a -SQL statement that is to be sent to a standards-compliant sql server. -This will replace single quotes ("'") by two single quotes ("''") -inside each field. You must use stdsql together with MySQL if in MySQL -configuration the -NO_BACKSLASH_ESCAPES is -turned on.

-

option.json - format the string suitable for a -json statement. -This will replace single quotes ("'") by two single quotes ("''") -inside each field.

-

At no time, multiple template option should be used. This can cause -unpredictable behaviour and is against all logic.

-

Either the sql or stdsql  -option must be specified when a template is used -for writing to a database, otherwise injection might occur. Please note -that due to the unfortunate fact that several vendors have violated the -sql standard and introduced their own escape methods, it is impossible -to have a single option doing all the work.  So you yourself -must make sure you are using the right format. If you choose -the wrong one, you are still vulnerable to sql injection.
-
-Please note that the database writer *checks* that the sql option is -present in the template. If it is not present, the write database -action is disabled. This is to guard you against accidental forgetting -it and then becoming vulnerable to SQL injection. The sql option can -also be useful with files - especially if you want to import them into -a database on another machine for performance reasons. However, do NOT -use it if you do not have a real need for it - among others, it takes -some toll on the processing time. Not much, but on a really busy system -you might notice it ;)

-

The default template for the write to database action has the -sql option set. As we currently support only MySQL and the sql option -matches the default MySQL configuration, this is a good choice. -However, if you have turned on -NO_BACKSLASH_ESCAPES in -your MySQL config, you need to supply a template with the stdsql -option. Otherwise you will become vulnerable to SQL injection.
-
-To escape:
-% = \%
-\ = \\ --> '\' is used to escape (as in C)
-$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"
+
Properties can be accessed by the property replacer (see there for details).

@@ -312,7 +364,6 @@ helps to keep a good overview over the different parts of the template. Though, writing it in a single line will work, it is much harder to debug if anything goes wrong with the template.

-

Please note that templates can also be used to generate selector lines with dynamic file names. For example, if you would like to split syslog messages from different -- cgit v1.2.3