From 31bf5b8bae41b8a65697ae07413fd36631c0751c Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Tue, 22 Oct 2013 10:04:40 +0200 Subject: remove global variable support The original idea was not well thought out and global variables, as implemented, worked far different from what anybody would expect. As such, we consider the current approach as an experiment that did not work out and opt to removing it, clearing the way for a better future solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013. --- ChangeLog | 6 ++++++ doc/rainerscript.html | 38 ++++++++------------------------------ grammar/lexer.l | 4 ++-- 3 files changed, 16 insertions(+), 32 deletions(-) diff --git a/ChangeLog b/ChangeLog index 90b2f9a2..8fc92cd5 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,11 @@ --------------------------------------------------------------------------- Version 7.5.6 [devel] 2013-10-?? +- removed global variable support + The original idea was not well thought out and global variables, as + implemented, worked far different from what anybody would expect. As + such, we consider the current approach as an experiment that did not + work out and opt to removing it, clearing the way for a better future + solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013. - imudp: support for binding to ruleset added - bugfix: running imupd on multiple threads lead to segfault if recvmmsg is available diff --git a/doc/rainerscript.html b/doc/rainerscript.html index aaaced40..0a780ac4 100644 --- a/doc/rainerscript.html +++ b/doc/rainerscript.html @@ -37,44 +37,22 @@ script interpreter when there is need to do so.

Variable (Property) types

All rsyslog properties (see the property replacer page for a list) can be used in RainerScript. In addition, it also -supports local and global variables. Local variables are local to the current message, but are +supports local variables. Local variables are local to the current message, but are NOT message properties (e.g. the "$!" all JSON property does not contain -them). Global variables have a truely global scope and are NOT bound to -a specifc message. Thus they can be used to persist values across -multiple messages (for things like counters). Please note that rsyslog -ensures proper synchronization for global variables (which also means -they are slower than the others). HOWEVER, in a highly multithreaded -configuration operations like
-set $/var = $/var + 1;
-are not atomic, so some updates to the counter variable may be missing. The -classical sample for this is in a two-thread environment: Variable $/var is set -to 1 at the start. Now the following happens in the following order: -

    -
  1. Thread A reads 1 from $/var and adds 1, result is 2, but not yet stored -
  2. Thread B reads 1 from $/var and adds 1, result is 2, but not yet stored -
  3. Thread A stores its result of 2 -
  4. Thread B stores its result of 2 -
-After this sequence, $/var contains the value two, which is probably not what was -expected. Rsyslog does not provide looking primitives for individual variables, -as this can lead to serious configuration problems if not used 100% correctly. -However, rsyslog provides (or will in the future provide) special function which -provide guaranteed atomic updates (in the sample, the end result would be three -no matter what the scheduling order is). +them).

Only message json (CEE/Lumberjack) properties can be modified by the "set" and "unset" statements, not any other message property. Obviously, -local and global variables are also modifieable. +local variables are also modifieable.

Message JSON property names start with "$!" where the bang character represents the root. -

Local variables names start with "$.", where the dot denotes the root. Similarly, -global variables start with "$/". -

Both JSON properties as well as global/local variables may contain an arbitrary +

Local variables names start with "$.", where the dot denotes the root. +

Both JSON properties as well as local variables may contain an arbitrary deep path before the final element. The bang character is always used as path -separator, no matter if it is a message property or a global/local variable. For example +separator, no matter if it is a message property or a local variable. For example "$!path1!path2!varname" is a three-level deep message property where as the very similar looking "$.path1!path2!varname" specifies a three-level -deep local variable. The similar global variables is named "$/path1!path2!varname". -The bang, slash, or dot character immediately following the +deep local variable. +The bang or dot character immediately following the dollar sign is used by rsyslog to separate the different types.

configuration objects

main_queue()

diff --git a/grammar/lexer.l b/grammar/lexer.l index d254d47e..f1b29755 100644 --- a/grammar/lexer.l +++ b/grammar/lexer.l @@ -9,7 +9,7 @@ * cases. So while we hope that cfsysline support can be dropped some time in * the future, we will probably keep these useful constructs. * - * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -129,7 +129,7 @@ int fileno(FILE *stream); 0[0-7]+ | /* octal number */ 0x[0-7a-f] | /* hex number, following rule is dec; strtoll handles all! */ ([1-9][0-9]*|0) { yylval.n = strtoll(yytext, NULL, 0); return NUMBER; } -\$[$!./]{0,1}[a-z][!a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } +\$[$!.]{0,1}[a-z][!a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } \'([^'\\]|\\['"\\$bntr]|\\x[0-9a-f][0-9a-f]|\\[0-7][0-7][0-7])*\' { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); -- cgit v1.2.3 From e367e2bca96f2c6164b3faaa063319b4fcbb190d Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Wed, 23 Oct 2013 16:00:11 +0200 Subject: re-enable global vars, but as an undocumented & unsupported feature This is done as we may be able to preserve the syntax. --- grammar/lexer.l | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/grammar/lexer.l b/grammar/lexer.l index f1b29755..36c23c7d 100644 --- a/grammar/lexer.l +++ b/grammar/lexer.l @@ -129,7 +129,7 @@ int fileno(FILE *stream); 0[0-7]+ | /* octal number */ 0x[0-7a-f] | /* hex number, following rule is dec; strtoll handles all! */ ([1-9][0-9]*|0) { yylval.n = strtoll(yytext, NULL, 0); return NUMBER; } -\$[$!.]{0,1}[a-z][!a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } +\$[$!./]{0,1}[a-z][!a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } \'([^'\\]|\\['"\\$bntr]|\\x[0-9a-f][0-9a-f]|\\[0-7][0-7][0-7])*\' { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); -- cgit v1.2.3 From d77a8b1b8877d2dd1a0248b07564d1f88be00deb Mon Sep 17 00:00:00 2001 From: Pavel Levshin Date: Thu, 24 Oct 2013 12:19:00 +0200 Subject: imuxsock: When syssock.use set to off, additional listeners cannot be created --- plugins/imuxsock/imuxsock.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index c6db8c5a..9d52fe04 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -1277,8 +1277,6 @@ BEGINactivateCnfPrePrivDrop int i; CODESTARTactivateCnfPrePrivDrop runModConf = pModConf; - if(runModConf->bOmitLocalLogging && nfd == 1) - ABORT_FINALIZE(RS_RET_OK); /* we first calculate the number of listeners so that we can * appropriately size the listener array. */ -- cgit v1.2.3 From 559b3000414ec23ea147a0f00db5c69bded0befb Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Thu, 24 Oct 2013 12:34:21 +0200 Subject: imuxsock: add some clarifiying comments --- plugins/imuxsock/imuxsock.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index 9d52fe04..01e07524 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -155,7 +155,8 @@ static int startIndexUxLocalSockets; /* process fd from that index on (used to * suppress local logging. rgerhards 2005-08-01 * read-only after startup */ -static int nfd = 1; /* number of Unix sockets open / read-only after startup */ +static int nfd = 1; /* number of active unix sockets (socket 0 is always reserved for the system + socket, even if it is not enabled. */ static int sd_fds = 0; /* number of systemd activated sockets */ /* config vars for legacy config system */ @@ -1278,7 +1279,8 @@ BEGINactivateCnfPrePrivDrop CODESTARTactivateCnfPrePrivDrop runModConf = pModConf; /* we first calculate the number of listeners so that we can - * appropriately size the listener array. + * appropriately size the listener array. Note that we will + * always allocate memory for the system log socket. */ nLstn = 0; for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { -- cgit v1.2.3 From 599572684b94f16eb8254f4c7c89ca735b270193 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Thu, 24 Oct 2013 12:52:15 +0200 Subject: doc: add imuxsock fix to ChangeLog --- ChangeLog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ChangeLog b/ChangeLog index 8fc92cd5..aaf6e7d7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -7,6 +7,8 @@ Version 7.5.6 [devel] 2013-10-?? work out and opt to removing it, clearing the way for a better future solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013. - imudp: support for binding to ruleset added +- bugfix: imuxsock did not suport addtl sockets if syssock was disabled + Thanks to Pavel Levshin for the fix - bugfix: running imupd on multiple threads lead to segfault if recvmmsg is available - bugfix: imudp when using recvmmsg could report wrong sender IP -- cgit v1.2.3 From 053576d9f1c194a071187f9b87641d7da73e5a72 Mon Sep 17 00:00:00 2001 From: Pavel Levshin Date: Thu, 24 Oct 2013 16:03:18 +0200 Subject: bugfix: Segmentation fault on incorrect variable assignment --- runtime/msg.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/runtime/msg.c b/runtime/msg.c index e159bba9..e30ff671 100644 --- a/runtime/msg.c +++ b/runtime/msg.c @@ -4030,6 +4030,12 @@ msgAddJSONObj(msg_t *pM, uchar *name, struct json_object *json, struct json_obje } leaf = jsonPathGetLeaf(name, ustrlen(name)); CHKiRet(jsonPathFindParent(*pjroot, name, leaf, &parent, 1)); + if (json_object_get_type(parent) != json_type_object) { + DBGPRINTF("msgAddJSON: not a container in json path," + "name is '%s'\n", name); + json_object_put(json); + ABORT_FINALIZE(RS_RET_INVLD_SETOP); + } leafnode = json_object_object_get(parent, (char*)leaf); if(leafnode == NULL) { json_object_object_add(parent, (char*)leaf, json); -- cgit v1.2.3 From 3a2a5442d93c8f994d8a274acb604c156a059e5e Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Thu, 24 Oct 2013 16:05:38 +0200 Subject: doc: update ChangeLog --- ChangeLog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ChangeLog b/ChangeLog index aaf6e7d7..eea8d0f9 100644 --- a/ChangeLog +++ b/ChangeLog @@ -7,6 +7,8 @@ Version 7.5.6 [devel] 2013-10-?? work out and opt to removing it, clearing the way for a better future solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013. - imudp: support for binding to ruleset added +- bugfix: segfault if variable was assigned to non-container subtree + Thanks to Pavel Levshin for the fix - bugfix: imuxsock did not suport addtl sockets if syssock was disabled Thanks to Pavel Levshin for the fix - bugfix: running imupd on multiple threads lead to segfault if recvmmsg -- cgit v1.2.3