From 89b086855359089ba771d6455cf497be0ae868a7 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Wed, 11 Mar 2009 20:40:35 +0100 Subject: added link to (new) online video tutorials to manual set --- doc/manual.html | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/manual.html b/doc/manual.html index 7aebacd8..e7db5768 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -86,6 +86,7 @@ wiki, a community resource which includes rsyslog discussion forum - use this for technical support +

syslog device configuration guide (off-site)

-- cgit v1.2.3 From e23e2e391ca1b0a5987b95f1a9f1ec8910faae8a Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Mon, 16 Mar 2009 10:20:41 +0100 Subject: added link to online tutorial on the web --- doc/rsyslog_conf.html | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/rsyslog_conf.html b/doc/rsyslog_conf.html index 01e693e4..9855a1b0 100644 --- a/doc/rsyslog_conf.html +++ b/doc/rsyslog_conf.html @@ -232,7 +232,11 @@ Lines starting with a hash mark ("#'') and empty lines are ignored. any format a user might want. They are also used for dynamic file name generation. Every output in rsyslog uses templates - this holds true -for files, user messages and so on. The database writer expects its +for files, user messages and so on. +Please note that there is an +online tutorial on rsyslog templates +available on the web. We recommend viewing it. +The database writer expects its template to be a proper SQL statement - so this is highly customizable too. You might ask how does all of this work when no templates at all are specified. Good question ;) The answer is simple, though. Templates -- cgit v1.2.3 From 16b960939ee44155a9ac6ac6999b6ab71f237034 Mon Sep 17 00:00:00 2001 From: varmojfekoj Date: Mon, 16 Mar 2009 15:40:03 +0100 Subject: fixed bugs in RainerScript - when converting a number and a string to a common type, both were actually converted to the other variable's type. - the value of rsCStrConvertToNumber() was miscalculated. Signed-off-by: Rainer Gerhards --- ChangeLog | 5 +++++ runtime/stringbuf.c | 2 +- runtime/var.c | 4 ++-- 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index 35bb29c8..f797f854 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,10 @@ --------------------------------------------------------------------------- Version 3.20.5 [v3-stable] (rgerhards), 2009-0?-?? +- fixed bugs in RainerScript: + o when converting a number and a string to a common type, both were + actually converted to the other variable's type. + o the value of rsCStrConvertToNumber() was miscalculated. + Thanks to varmojfekoj for the patch - fixed a bug in configure.ac which resulted in problems with environment detection - thanks to Michael Biebl for the patch - fixed a potential segfault problem in gssapi code diff --git a/runtime/stringbuf.c b/runtime/stringbuf.c index 93d1e1ef..63b42348 100644 --- a/runtime/stringbuf.c +++ b/runtime/stringbuf.c @@ -820,7 +820,7 @@ rsCStrConvertToNumber(cstr_t *pStr, number_t *pNumber) /* TODO: octal? hex? */ n = 0; while(i < pStr->iStrLen && isdigit(pStr->pBuf[i])) { - n = n * 10 + pStr->pBuf[i] * 10; + n = n * 10 + pStr->pBuf[i] - '0'; ++i; } diff --git a/runtime/var.c b/runtime/var.c index 7e51fc6d..f964874b 100644 --- a/runtime/var.c +++ b/runtime/var.c @@ -327,7 +327,7 @@ ConvForOperation(var_t *pThis, var_t *pOther) case VARTYPE_NUMBER: /* check if we can convert pThis to a number, if so use number format. */ iRet = ConvToNumber(pThis); - if(iRet != RS_RET_NOT_A_NUMBER) { + if(iRet == RS_RET_NOT_A_NUMBER) { CHKiRet(ConvToString(pOther)); } else { FINALIZE; /* OK or error */ @@ -345,7 +345,7 @@ ConvForOperation(var_t *pThis, var_t *pOther) break; case VARTYPE_STR: iRet = ConvToNumber(pOther); - if(iRet != RS_RET_NOT_A_NUMBER) { + if(iRet == RS_RET_NOT_A_NUMBER) { CHKiRet(ConvToString(pThis)); } else { FINALIZE; /* OK or error */ -- cgit v1.2.3 From 01adeab0cba21ad6193addf1a4e90689b507d092 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Thu, 2 Apr 2009 14:41:32 +0200 Subject: preparing for 3.20.5 release --- ChangeLog | 4 ++-- configure.ac | 2 +- doc/manual.html | 2 +- runtime/wtp.c | 8 ++++++++ 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/ChangeLog b/ChangeLog index a94eb24a..a4eb6151 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,9 +1,9 @@ +--------------------------------------------------------------------------- +Version 3.20.5 [v3-stable] (rgerhards), 2009-04-02 - bugfix: potential abort with DA queue after high watermark is reached There exists a race condition that can lead to a segfault. Thanks go to vbernetr, who performed the analysis and provided patch, which I only tweaked a very little bit. ---------------------------------------------------------------------------- -Version 3.20.5 [v3-stable] (rgerhards), 2009-0?-?? - fixed bugs in RainerScript: o when converting a number and a string to a common type, both were actually converted to the other variable's type. diff --git a/configure.ac b/configure.ac index 6a79d9a3..e636070c 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[3.20.4],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[3.20.5],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE AC_CONFIG_SRCDIR([ChangeLog]) AC_CONFIG_HEADERS([config.h]) diff --git a/doc/manual.html b/doc/manual.html index e7db5768..c9fd9b90 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -16,7 +16,7 @@ relay chains while at the same time being very easy to setup for the novice user. And as we know what enterprise users really need, there is also professional rsyslog support available directly from the source!

This documentation is for version 3.20.4 (v3-stable branch) of rsyslog. +

This documentation is for version 3.20.5 (v3-stable branch) of rsyslog. Visit the rsyslog status page to obtain current version information and project status.

If you like rsyslog, you might diff --git a/runtime/wtp.c b/runtime/wtp.c index fcefa1d8..3e3ff09a 100644 --- a/runtime/wtp.c +++ b/runtime/wtp.c @@ -196,6 +196,14 @@ wtpProcessThrdChanges(wtp_t *pThis) FINALIZE; } + /* Note: there is a left-over potential race condition below: + * pThis->bThrdStateChanged may be re-set by another thread while + * we work on it and thus the loop may terminate too early. However, + * there are no really bad effects from that so I perfer - for this + * version - to live with the problem as is. Not a good idea to + * introduce that large change into the stable branch without very + * good reason. -- rgerhards, 2009-04-02 + */ do { /* reset the change marker */ pThis->bThrdStateChanged = 0; -- cgit v1.2.3 From 0550512b28049c391be424ad518ed0debb77089e Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Fri, 3 Apr 2009 15:06:15 +0200 Subject: preparing for 3.21.11 --- ChangeLog | 2 +- configure.ac | 2 +- doc/manual.html | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index 72d056c6..e22fee6c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,5 @@ --------------------------------------------------------------------------- -Version 3.21.11 [BETA] (rgerhards), 2009-0?-?? +Version 3.21.11 [BETA] (rgerhards), 2009-04-03 - build system improvements contributed by Michael Biebl - thx! - all patches from 3.20.5 incorporated (see it's ChangeLog entry) --------------------------------------------------------------------------- diff --git a/configure.ac b/configure.ac index 8538a8f9..ea0505fe 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[3.21.10],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[3.21.11],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE AC_CONFIG_SRCDIR([ChangeLog]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/doc/manual.html b/doc/manual.html index abaa1ab5..2e87a668 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -16,7 +16,7 @@ relay chains while at the same time being very easy to setup for the novice user. And as we know what enterprise users really need, there is also professional rsyslog support available directly from the source!

-
This documentation is for version 3.21.10 (beta branch) of rsyslog. +

This documentation is for version 3.21.11 (beta branch) of rsyslog. Visit the rsyslog status page to obtain current version information and project status.

If you like rsyslog, you might -- cgit v1.2.3 From 9103817f9fe811b49938036a1f9ff23672a9ec44 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Fri, 3 Apr 2009 15:48:55 +0200 Subject: added (some) developer documentation for output plugin interface --- ChangeLog | 1 + doc/dev_oplugins.html | 178 ++++++++++++++++++++++++++++++++++++++++++++++++++ doc/manual.html | 6 +- doc/status.html | 12 ++-- 4 files changed, 190 insertions(+), 7 deletions(-) create mode 100644 doc/dev_oplugins.html diff --git a/ChangeLog b/ChangeLog index 7f79271f..8d4191d3 100644 --- a/ChangeLog +++ b/ChangeLog @@ -39,6 +39,7 @@ Version 4.1.6 [DEVEL] (rgerhards), 2009-03-?? - added a new way how output plugins may be passed parameters. This is more effcient for some outputs. They new can receive fields not only as a single string but rather in an array where each string is seperated. +- added (some) developer documentation for output plugin interface --------------------------------------------------------------------------- Version 4.1.5 [DEVEL] (rgerhards), 2009-03-11 - bugfix: parser did not correctly parse fields in UDP-received messages diff --git a/doc/dev_oplugins.html b/doc/dev_oplugins.html new file mode 100644 index 00000000..5bfc974c --- /dev/null +++ b/doc/dev_oplugins.html @@ -0,0 +1,178 @@ + + +writing rsyslog output plugins (developer's guide) + + +

Writing Rsyslog Output Plugins

+
This page is the begin of some developer documentation for writing output +plugins. Doing so is quite easy (and that was a design goal), but there currently +is only sparse documentation on the process available. I was tempted NOT to +write this guide here because I know I will most probably not be able to +write a complete guide. +
However, I finally concluded that it may be better to have same information +and pointers than to have nothing. +
Getting Started and Samples
+
The best to get started with rsyslog plugin development is by looking at +existing plugins. All that start with "om" are output modules. That +means they are primarily thought of being message sinks. In theory, however, output +plugins may aggergate other functionality, too. Nobody has taken this route so far +so if you would like to do that, it is highly suggested to post your plan on the +rsyslog mailing list, first (so that we can offer advise). +
The rsyslog distribution tarball contains two plugins that are extremely well +targeted for getting started: +
+
omtemplate +
omstdout +
+Plugin omtemplate was specifically created to provide a copy template for new output +plugins. It is bare of real functionality but has ample comments. Even if you decide +to start from another plugin (or even from scratch), be sure to read omtemplate source +and comments first. The omstdout is primarily a testing aide, but offers support for +the two different parameter-passing conventions plugins can use (plus the way to +differentiate between the two). It also is not bare of functionaly, only mostly +bare of it ;). But you can actually execute it and play with it. +
In any case, you should also read the comments in ./runtime/module-template.h. +Output plugins are build based on a large set of code-generating macros. These +macros handle most of the plumbing needed by the interface. As long as no +special callback to rsyslog is needed (it typically is not), an output plugin does +not really need to be aware that it is executed by rsyslog. As a plug-in programmer, +you can (in most cases) "code as usual". However, all macros and entry points need to be +provided and thus reading the code comments in the files mentioned is highly suggested. +
In short, the best idea is to start with a template. Let's assume you start by +copying omtemplate. Then, the basic steps you need to do are: +
+
cp ./plugins/omtemplate ./plugins/your-plugin +
mv cd ./plugins/your-plugin +
vi Makefile.am, adjust to your-plugin +
mv omtemplate.c your-plugin.c +
cd ../.. +
vi Makefile.am configure.ac +
serach for omtemplate, copy and modify (follow comments) +
+
Basically, this is all you need to do ... Well, except, of course, coding +your plugin ;). For testing, you need rsyslog's debugging support. Some useful +information is given in "troubleshooting rsyslog +from the doc set. +
Special Topics
+
Threading
+
Rsyslog uses massive parallel processing and multithreading. However, a plugin's entry +points are guaranteed to be never called concurrently for the same action. +That means your plugin must be able to be called concurrently by two or more +threads, but you can be sure that for the same instance no concurrent calls +happen. This is guaranteed by the interface specification and the rsyslog core +guards against multiple concurrent calls. An instance, in simple words, is one +that shares a single instanceData structure. +
So as long as you do not mess around with global data, you do not need +to think about multithreading (and can apply a purely sequential programming +methodology). +
Please note that duringt the configuraton parsing stage of execution, access to +global variables for the configuration system is safe. In that stage, the core will +only call sequentially into the plugin. +
Getting Message Data
+
The doAction() entry point of your plugin is provided with messages to be processed. +It will only be activated after filtering and all other conditions, so you do not need +to apply any other conditional but can simply process the message. +
Note that you do NOT receive the full internal representation of the message +object. There are various (including historical) reasons for this and, among +others, this is a design decision based on security. +
Your plugin will only receive what the end user has configured in a $template +statement. However, starting with 4.1.6, there are two ways of receiving the +template content. The default mode, and in most cases sufficient and optimal, +is to receive a single string with the expanded template. As I said, this is usually +optimal, think about writing things to files, emailing content or forwarding it. +
The important philosophy is that a plugin should never reformat any +of such strings - that would either remove the user's ability to fully control +message formats or it would lead to duplicating code that is already present in the +core. If you need some formatting that is not yet present in the core, suggest it +to the rsyslog project, best done by sending a patch ;), and we will try hard to +get it into the core (so far, we could accept all such suggestions - no promise, though). +
If a single string seems not suitable for your application, the plugin can also +request access to the template components. The typical use case seems to be databases, where +you would like to access properties via specific fields. With that mode, you receive a +char ** array, where each array element points to one field from the template (from +left to right). Fields start at arrray index 0 and a NULL pointer means you have +reached the end of the array (the typical Unix "poor man's linked list in an array" +design). Note, however, that each of the individual components is a string. It is +not a date stamp, number or whatever, but a string. This is because rsyslog processes +strings (from a high-level design look at it) and so this is the natural data type. +Feel free to convert to whatever you need, but keep in mind that malformed packets +may have lead to field contents you'd never expected... +
If you like to use the array-based parameter passing method, think that it +is only available in rsyslog 4.1.6 and above. If you can accept that your plugin +will not be working with previous versions, you do not need to handle pre 4.1.6 cases. +However, it would be "nice" if you shut down yourself in these cases - otherwise the +older rsyslog core engine will pass you a string where you expect the array of pointers, +what most probably results in a segfault. To check whether or not the core supports the +functionality, you can use this code sequence: +
++BEGINmodInit() + rsRetVal localRet; + rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts); + unsigned long opts; + int bArrayPassingSupported; /* does core support template passing as an array? */ +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + /* check if the rsyslog core supports parameter passing code */ + bArrayPassingSupported = 0; + localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", &pomsrGetSupportedTplOpts); + if(localRet == RS_RET_OK) { + /* found entry point, so let's see if core supports array passing */ + CHKiRet((*pomsrGetSupportedTplOpts)(&opts)); + if(opts & OMSR_TPL_AS_ARRAY) + bArrayPassingSupported = 1; + } else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) { + ABORT_FINALIZE(localRet); /* Something else went wrong, what is not acceptable */ + } + DBGPRINTF("omstdout: array-passing is %ssupported by rsyslog core.\n", bArrayPassingSupported ? "" : "not "); + + if(!bArrayPassingSupported) { + DBGPRINTF("rsyslog core too old, shutting down this plug-in\n"); + ABORT_FINALIZE(RS_RET_ERR); + } + + +
+
The code first checks if the core supports the OMSRgetSupportedTplOpts() API (which is +also not present in all versions!) and, if so, queries the core if the OMSR_TPL_AS_ARRAY mode +is supported. If either does not exits, the core is too old for this functionality. The sample +snippet above then shuts down, but a plugin may instead just do things different. In +omstdout, you can see how a plugin may deal with the situation. +
In any case, it is recommended that at least a graceful shutdown is made and the +array-passing capability not blindly be used. In such cases, we can not guard the +plugin from segfaulting and if the plugin (as currently always) is run within +rsyslog's process space, that results in a segfault for rsyslog. So do not do this. +
Batching of Messages
+
With the current plugin interface, each message is passed via a separate call to the plugin. +This is annoying and costs performance in some uses cases (primarily for database outputs). +However, that's the way it (currently) is, no easy way around it. There are some ideas +to implement batching capabilities inside the rsyslog core, but without that the only +resort is to do it inside your plugin yourself. You are not prohibited from doing so. +There are some consequences, though: most importantly, the rsyslog core is no longer +intersted in messages that it passed to a plugin. As such, it will not try to make sure +the message is not lost before it was ultimately processed (because rsyslog, due to +doAction() returning successfully, thinks the message *was* ultimately processed). +
When the rsyslog core receives batching capabilities, this will be implemented in +a way that is fully compatible to the existing plugin interface. While we have not yet +thought about the implementation, that will probably mean that some new interfaces +or options be used to turn on batching capabilities. +
Licensing
+
From the rsyslog point of view, plugins constitute separate projects. As such, +we think plugins are not required to be compatible with GPLv3. However, this is +no legal advise. If you intend to release something under a non-GPLV3 compatible license +it is probably best to consult with your lawyer. +
Most importantly, and this is definite, the rsyslog team does not expect +or require you to contribute your plugin to the rsyslog project (but of course +we are happy if you do). +
Copyright
+
Copyright (c) 2009 Rainer Gerhards +and Adiscon.
+
Permission is granted to copy, distribute and/or modify this document under +the terms of the GNU Free Documentation License, Version 1.2 or any later +version published by the Free Software Foundation; with no Invariant Sections, +no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be +viewed at +http://www.gnu.org/copyleft/fdl.html.
+ + diff --git a/doc/manual.html b/doc/manual.html index 99a90594..0b03a255 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -64,7 +64,11 @@ the syslog priority (severity and facility) to the log file syslog sender over NAT (online only)
an overview and howto of rsyslog gssapi support

debug support in rsyslog
-
the rsyslog message queue object (developer's view)
+
Developer Documentation +
+
writing rsyslog output plugins
+
the rsyslog message queue object (developer's view)
+

Our rsyslog history page is for you if you would like to learn a little more diff --git a/doc/status.html b/doc/status.html index 59fd0809..dae94884 100644 --- a/doc/status.html +++ b/doc/status.html @@ -2,19 +2,19 @@ rsyslog status page
rsyslog status page
-
This page reflects the status as of 2009-02-02.
+
This page reflects the status as of 2009-04-03.

Current Releases

development: 4.1.5 [2009-03-11] - change log - download -
beta: 3.21.10 [2009-02-02] - -change log - -download
+
beta: 3.21.11 [2009-04-03] - +change log - +download
-
v3 stable: 3.20.3 [2009-01-19] - change log - -download +
v3 stable: 3.20.3 [2009-04-02] - change log - +download
v2 stable: 2.0.6 [2008-08-07] - change log - download -- cgit v1.2.3 From ce6b7e86cdd63ba1540d20aa22403d2b13d2e59f Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Fri, 3 Apr 2009 17:54:09 +0200 Subject: improved test suite and added test for new output module interface The testbench has now a generic driver that can run a whole class of test suites just by providing a config file and test cases. This does not cover all testing needs, but a lot. We have now added one test for the new array-passing output plugin interface. --- plugins/omstdout/omstdout.c | 2 + tests/Makefile.am | 28 ++-- tests/omod-if-array.sh | 1 + tests/parsertest.c | 269 --------------------------------- tests/parsertest.sh | 1 + tests/testruns/1.parse1 | 3 - tests/testruns/parser.conf | 9 -- tests/testruns/rfc3164.parse1 | 4 - tests/testruns/rfc5424-1.parse1 | 3 - tests/testruns/rfc5424-2.parse1 | 4 - tests/testruns/rfc5424-3.parse1 | 4 - tests/testruns/rfc5424-4.parse1 | 4 - tests/testsuites/1.omod-if-array | 2 + tests/testsuites/1.parse1 | 3 + tests/testsuites/omod-if-array.conf | 14 ++ tests/testsuites/parse1.conf | 9 ++ tests/testsuites/rfc3164.parse1 | 4 + tests/testsuites/rfc5424-1.parse1 | 3 + tests/testsuites/rfc5424-2.parse1 | 4 + tests/testsuites/rfc5424-3.parse1 | 4 + tests/testsuites/rfc5424-4.parse1 | 4 + tests/udptester.c | 291 ++++++++++++++++++++++++++++++++++++ 22 files changed, 358 insertions(+), 312 deletions(-) create mode 100755 tests/omod-if-array.sh delete mode 100644 tests/parsertest.c create mode 100755 tests/parsertest.sh delete mode 100644 tests/testruns/1.parse1 delete mode 100644 tests/testruns/parser.conf delete mode 100644 tests/testruns/rfc3164.parse1 delete mode 100644 tests/testruns/rfc5424-1.parse1 delete mode 100644 tests/testruns/rfc5424-2.parse1 delete mode 100644 tests/testruns/rfc5424-3.parse1 delete mode 100644 tests/testruns/rfc5424-4.parse1 create mode 100644 tests/testsuites/1.omod-if-array create mode 100644 tests/testsuites/1.parse1 create mode 100644 tests/testsuites/omod-if-array.conf create mode 100644 tests/testsuites/parse1.conf create mode 100644 tests/testsuites/rfc3164.parse1 create mode 100644 tests/testsuites/rfc5424-1.parse1 create mode 100644 tests/testsuites/rfc5424-2.parse1 create mode 100644 tests/testsuites/rfc5424-3.parse1 create mode 100644 tests/testsuites/rfc5424-4.parse1 create mode 100644 tests/udptester.c diff --git a/plugins/omstdout/omstdout.c b/plugins/omstdout/omstdout.c index e491005c..7c63b5c4 100644 --- a/plugins/omstdout/omstdout.c +++ b/plugins/omstdout/omstdout.c @@ -107,6 +107,8 @@ CODESTARTdoAction iParam = 0; iBuf = 0; while(szParams[iParam] != NULL) { + if(iParam > 0) + szBuf[iBuf++] = ','; /* all but first need a delimiter */ iParamVal = 0; while(szParams[iParam][iParamVal] != '\0' && iBuf < sizeof(szBuf)) { szBuf[iBuf++] = szParams[iParam][iParamVal++]; diff --git a/tests/Makefile.am b/tests/Makefile.am index 09d1a0b6..ab1c5a62 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,6 +1,6 @@ -TESTRUNS = rt_init rscript parsertest -check_PROGRAMS = $(TESTRUNS) ourtail -TESTS = $(TESTRUNS) cfg.sh +TESTRUNS = rt_init rscript +check_PROGRAMS = $(TESTRUNS) ourtail udptester +TESTS = $(TESTRUNS) cfg.sh parsertest.sh omod-if-array.sh TESTS_ENVIRONMENT = RSYSLOG_MODDIR='$(abs_top_builddir)'/runtime/.libs/ DISTCLEANFILES=rsyslog.pid test_files = testbench.h runtime-dummy.c @@ -17,19 +17,23 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ DevNull.cfgtest \ err1.rstest \ NoExistFile.cfgtest \ - testruns/parser.conf \ - testruns/1.parse1 \ - testruns/rfc3164.parse1 \ - testruns/rfc5424-1.parse1 \ - testruns/rfc5424-2.parse1 \ - testruns/rfc5424-3.parse1 \ - testruns/rfc5424-4.parse1 \ + testsuites/parse1.conf \ + testsuites/1.parse1 \ + testsuites/rfc3164.parse1 \ + testsuites/rfc5424-1.parse1 \ + testsuites/rfc5424-2.parse1 \ + testsuites/rfc5424-3.parse1 \ + testsuites/rfc5424-4.parse1 \ + testsuites/omod-if-array.conf \ + testsuites/1.omod-if-array \ + parsertest.sh \ + omod-if-array.sh \ cfg.sh ourtail_SOURCES = ourtail.c -parsertest_SOURCES = parsertest.c getline.c -parsertest_LDADD = $(SOL_LIBS) +udptester_SOURCES = udptester.c getline.c +udptester_LDADD = $(SOL_LIBS) rt_init_SOURCES = rt-init.c $(test_files) rt_init_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) diff --git a/tests/omod-if-array.sh b/tests/omod-if-array.sh new file mode 100755 index 00000000..cac08928 --- /dev/null +++ b/tests/omod-if-array.sh @@ -0,0 +1 @@ +./udptester omod-if-array diff --git a/tests/parsertest.c b/tests/parsertest.c deleted file mode 100644 index 6c2221e8..00000000 --- a/tests/parsertest.c +++ /dev/null @@ -1,269 +0,0 @@ -/* Runs a test suite on the rsyslog parser (and later potentially - * other things). - * - * Please note that this - * program works together with the config file AND easily extensible - * test case files (*.parse1) to run a number of checks. All test - * cases are executed, even if there is a failure early in the - * process. When finished, the numberof failed tests will be given. - * - * Part of the testbench for rsyslog. - * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of rsyslog. - * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see . - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - */ -#include "config.h" -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define EXIT_FAILURE 1 - - -void readLine(int fd, char *ln) -{ - char c; - int lenRead; - lenRead = read(fd, &c, 1); - while(lenRead == 1 && c != '\n') { - *ln++ = c; - lenRead = read(fd, &c, 1); - } - *ln = '\0'; -} - - -/* send a message via UDP - * returns 0 if ok, something else otherwise. - */ -int -udpSend(char *buf, int lenBuf) -{ - struct sockaddr_in si_other; - int s, slen=sizeof(si_other); - - if((s=socket(AF_INET, SOCK_DGRAM, 0))==-1) { - perror("socket()"); - return(1); - } - - memset((char *) &si_other, 0, sizeof(si_other)); - si_other.sin_family = AF_INET; - si_other.sin_port = htons(12514); - if(inet_aton("127.0.0.1", &si_other.sin_addr)==0) { - fprintf(stderr, "inet_aton() failed\n"); - return(1); - } - - if(sendto(s, buf, lenBuf, 0, (struct sockaddr*) &si_other, slen)==-1) { - perror("sendto"); - fprintf(stderr, "sendto() failed\n"); - return(1); - } - - close(s); - return 0; -} - -/* open pipe to test candidate - so far, this is - * always rsyslogd and with a fixed config. Later, we may - * change this. Returns 0 if ok, something else otherwise. - * rgerhards, 2009-03-31 - */ -int openPipe(pid_t *pid, int *pfd) -{ - int pipefd[2]; - pid_t cpid; - char *newargv[] = {"../tools/rsyslogd", "dummy", "-c4", "-u2", "-n", "-irsyslog.pid", - "-M../runtime//.libs", NULL }; - char confFile[1024]; - char *newenviron[] = { NULL }; - - - sprintf(confFile, "-f%s/testruns/parser.conf", getenv("srcdir")); - newargv[1] = confFile; - - if (pipe(pipefd) == -1) { - perror("pipe"); - exit(EXIT_FAILURE); - } - - cpid = fork(); - if (cpid == -1) { - perror("fork"); - exit(EXIT_FAILURE); - } - - if(cpid == 0) { /* Child reads from pipe */ - fclose(stdout); - dup(pipefd[1]); - close(pipefd[1]); - close(pipefd[0]); - fclose(stdin); - execve("../tools/rsyslogd", newargv, newenviron); - } else { - close(pipefd[1]); - *pid = cpid; - *pfd = pipefd[0]; - } - - return(0); -} - - -/* Process a specific test case. File name is provided. - * Needs to return 0 if all is OK, something else otherwise. - */ -int -processTestFile(int fd, char *pszFileName) -{ - FILE *fp; - char *testdata = NULL; - char *expected = NULL; - int ret = 0; - size_t lenLn; - char buf[4096]; - - if((fp = fopen((char*)pszFileName, "r")) == NULL) { - perror((char*)pszFileName); - return(2); - } - - /* skip comments at start of file */ - - getline(&testdata, &lenLn, fp); - while(!feof(fp)) { - if(*testdata == '#') - getline(&testdata, &lenLn, fp); - else - break; /* first non-comment */ - } - - - testdata[strlen(testdata)-1] = '\0'; /* remove \n */ - /* now we have the test data to send */ - if(udpSend(testdata, strlen(testdata)) != 0) - return(2); - - /* next line is expected output - * we do not care about EOF here, this will lead to a failure and thus - * draw enough attention. -- rgerhards, 2009-03-31 - */ - getline(&expected, &lenLn, fp); - expected[strlen(expected)-1] = '\0'; /* remove \n */ - - /* pull response from server and then check if it meets our expectation */ - readLine(fd, buf); - if(strcmp(expected, buf)) { - printf("\nExpected Response:\n'%s'\nActual Response:\n'%s'\n", - expected, buf); - ret = 1; - } - - free(testdata); - free(expected); - fclose(fp); - return(ret); -} - - -/* carry out all tests. Tests are specified via a file name - * wildcard. Each of the files is read and the test carried - * out. - * Returns the number of tests that failed. Zero means all - * success. - */ -int -doTests(int fd, char *files) -{ - int iFailed = 0; - int iTests = 0; - int ret; - char *testFile; - glob_t testFiles; - size_t i = 0; - struct stat fileInfo; - - glob(files, GLOB_MARK, NULL, &testFiles); - - for(i = 0; i < testFiles.gl_pathc; i++) { - testFile = testFiles.gl_pathv[i]; - - if(stat((char*) testFile, &fileInfo) != 0) - continue; /* continue with the next file if we can't stat() the file */ - - ++iTests; - /* all regular files are run through the test logic. Symlinks don't work. */ - if(S_ISREG(fileInfo.st_mode)) { /* config file */ - printf("processing test case '%s' ... ", testFile); - ret = processTestFile(fd, testFile); - if(ret == 0) { - printf("successfully completed\n"); - } else { - printf("failed!\n"); - ++iFailed; - } - } - } - globfree(&testFiles); - - if(iTests == 0) { - printf("Error: no test cases found, no tests executed.\n"); - iFailed = 1; - } else { - printf("Number of tests run: %d, number of failures: %d\n", iTests, iFailed); - } - - return(iFailed); -} - - -/* */ -int main(int argc, char *argv[]) -{ - int fd; - pid_t pid; - int ret = 0; - char buf[4096]; - char testcases[4096]; - - printf("running rsyslog parser tests ($srcdir=%s)\n", getenv("srcdir")); - - openPipe(&pid, &fd); - readLine(fd, buf); - - /* generate filename */ - sprintf(testcases, "%s/testruns/*.parse1", getenv("srcdir")); - if(doTests(fd, testcases) != 0) - ret = 1; - - /* cleanup */ - kill(pid, SIGTERM); - printf("End of parser tests.\n"); - exit(ret); -} diff --git a/tests/parsertest.sh b/tests/parsertest.sh new file mode 100755 index 00000000..e7985bb0 --- /dev/null +++ b/tests/parsertest.sh @@ -0,0 +1 @@ +./udptester parse1 diff --git a/tests/testruns/1.parse1 b/tests/testruns/1.parse1 deleted file mode 100644 index 5ae655e6..00000000 --- a/tests/testruns/1.parse1 +++ /dev/null @@ -1,3 +0,0 @@ -<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 -167,local4,debug,Mar 6 16:57:54,172.20.245.8,%PIX-7-710005,%PIX-7-710005:, UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 -#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testruns/parser.conf b/tests/testruns/parser.conf deleted file mode 100644 index 0fb7d16d..00000000 --- a/tests/testruns/parser.conf +++ /dev/null @@ -1,9 +0,0 @@ -$ModLoad ../plugins/omstdout/.libs/omstdout -$ModLoad ../plugins/imudp/.libs/imudp -$UDPServerRun 12514 - -$ErrorMessagesToStderr off - -# use a special format that we can easily parse in expect -$template expect,"%PRI%,%syslogfacility-text%,%syslogseverity-text%,%timestamp%,%hostname%,%programname%,%syslogtag%,%msg%\n" -*.* :omstdout:;expect diff --git a/tests/testruns/rfc3164.parse1 b/tests/testruns/rfc3164.parse1 deleted file mode 100644 index e7a5fa18..00000000 --- a/tests/testruns/rfc3164.parse1 +++ /dev/null @@ -1,4 +0,0 @@ -<34>Oct 11 22:14:15 mymachine su: 'su root' failed for lonvick on /dev/pts/8 -34,auth,crit,Oct 11 22:14:15,mymachine,su,su:, 'su root' failed for lonvick on /dev/pts/8 -#Example from RFC3164, section 5.4 -#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testruns/rfc5424-1.parse1 b/tests/testruns/rfc5424-1.parse1 deleted file mode 100644 index 23836c9f..00000000 --- a/tests/testruns/rfc5424-1.parse1 +++ /dev/null @@ -1,3 +0,0 @@ -#Example from RFC5424, section 6.5 / sample 1 -<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8 -34,auth,crit,Oct 11 22:14:15,mymachine.example.com,,su,- BOM'su root' failed for lonvick on /dev/pts/8 diff --git a/tests/testruns/rfc5424-2.parse1 b/tests/testruns/rfc5424-2.parse1 deleted file mode 100644 index a86fbc35..00000000 --- a/tests/testruns/rfc5424-2.parse1 +++ /dev/null @@ -1,4 +0,0 @@ -<165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - - %% It's time to make the do-nuts. -165,local4,notice,Aug 24 05:14:15,192.0.2.1,,myproc[8710],- %% It's time to make the do-nuts. -#Example from RFC5424, section 6.5 / sample 2 -#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testruns/rfc5424-3.parse1 b/tests/testruns/rfc5424-3.parse1 deleted file mode 100644 index 6ad4073d..00000000 --- a/tests/testruns/rfc5424-3.parse1 +++ /dev/null @@ -1,4 +0,0 @@ -<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"][examplePriority@32473 class="high"] -165,local4,notice,Oct 11 22:14:15,mymachine.example.com,,evntslog, -#Example from RFC5424, section 6.5 / sample 4 -#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testruns/rfc5424-4.parse1 b/tests/testruns/rfc5424-4.parse1 deleted file mode 100644 index ecf27e14..00000000 --- a/tests/testruns/rfc5424-4.parse1 +++ /dev/null @@ -1,4 +0,0 @@ -<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"] BOMAn application event log entry... -165,local4,notice,Oct 11 22:14:15,mymachine.example.com,,evntslog,BOMAn application event log entry... -#Example from RFC5424, section 6.5 / sample 3 -#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testsuites/1.omod-if-array b/tests/testsuites/1.omod-if-array new file mode 100644 index 00000000..c464b19c --- /dev/null +++ b/tests/testsuites/1.omod-if-array @@ -0,0 +1,2 @@ +<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 +167,Mar 6 16:57:54,172.20.245.8,%PIX-7-710005,%PIX-7-710005:, diff --git a/tests/testsuites/1.parse1 b/tests/testsuites/1.parse1 new file mode 100644 index 00000000..5ae655e6 --- /dev/null +++ b/tests/testsuites/1.parse1 @@ -0,0 +1,3 @@ +<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 +167,local4,debug,Mar 6 16:57:54,172.20.245.8,%PIX-7-710005,%PIX-7-710005:, UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 +#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testsuites/omod-if-array.conf b/tests/testsuites/omod-if-array.conf new file mode 100644 index 00000000..e6c05a52 --- /dev/null +++ b/tests/testsuites/omod-if-array.conf @@ -0,0 +1,14 @@ +# Test config for array-passing output module interface +# (stanard string passing is already tested via the other test inside +# the testbench, so we do not need to focus on that) +# rgerhards, 2009-04-03 +$ModLoad ../plugins/omstdout/.libs/omstdout +$ModLoad ../plugins/imudp/.libs/imudp +$UDPServerRun 12514 + +$ActionOMStdoutArrayInterface on +$ErrorMessagesToStderr off + +# do NOT remove \n, that would hang the test driver! +$template expect,"%PRI%%timestamp%%hostname%%programname%%syslogtag%\n" +*.* :omstdout:;expect diff --git a/tests/testsuites/parse1.conf b/tests/testsuites/parse1.conf new file mode 100644 index 00000000..0fb7d16d --- /dev/null +++ b/tests/testsuites/parse1.conf @@ -0,0 +1,9 @@ +$ModLoad ../plugins/omstdout/.libs/omstdout +$ModLoad ../plugins/imudp/.libs/imudp +$UDPServerRun 12514 + +$ErrorMessagesToStderr off + +# use a special format that we can easily parse in expect +$template expect,"%PRI%,%syslogfacility-text%,%syslogseverity-text%,%timestamp%,%hostname%,%programname%,%syslogtag%,%msg%\n" +*.* :omstdout:;expect diff --git a/tests/testsuites/rfc3164.parse1 b/tests/testsuites/rfc3164.parse1 new file mode 100644 index 00000000..e7a5fa18 --- /dev/null +++ b/tests/testsuites/rfc3164.parse1 @@ -0,0 +1,4 @@ +<34>Oct 11 22:14:15 mymachine su: 'su root' failed for lonvick on /dev/pts/8 +34,auth,crit,Oct 11 22:14:15,mymachine,su,su:, 'su root' failed for lonvick on /dev/pts/8 +#Example from RFC3164, section 5.4 +#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testsuites/rfc5424-1.parse1 b/tests/testsuites/rfc5424-1.parse1 new file mode 100644 index 00000000..23836c9f --- /dev/null +++ b/tests/testsuites/rfc5424-1.parse1 @@ -0,0 +1,3 @@ +#Example from RFC5424, section 6.5 / sample 1 +<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8 +34,auth,crit,Oct 11 22:14:15,mymachine.example.com,,su,- BOM'su root' failed for lonvick on /dev/pts/8 diff --git a/tests/testsuites/rfc5424-2.parse1 b/tests/testsuites/rfc5424-2.parse1 new file mode 100644 index 00000000..a86fbc35 --- /dev/null +++ b/tests/testsuites/rfc5424-2.parse1 @@ -0,0 +1,4 @@ +<165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - - %% It's time to make the do-nuts. +165,local4,notice,Aug 24 05:14:15,192.0.2.1,,myproc[8710],- %% It's time to make the do-nuts. +#Example from RFC5424, section 6.5 / sample 2 +#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testsuites/rfc5424-3.parse1 b/tests/testsuites/rfc5424-3.parse1 new file mode 100644 index 00000000..6ad4073d --- /dev/null +++ b/tests/testsuites/rfc5424-3.parse1 @@ -0,0 +1,4 @@ +<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"][examplePriority@32473 class="high"] +165,local4,notice,Oct 11 22:14:15,mymachine.example.com,,evntslog, +#Example from RFC5424, section 6.5 / sample 4 +#Only the first two lines are important, you may place anything behind them! diff --git a/tests/testsuites/rfc5424-4.parse1 b/tests/testsuites/rfc5424-4.parse1 new file mode 100644 index 00000000..ecf27e14 --- /dev/null +++ b/tests/testsuites/rfc5424-4.parse1 @@ -0,0 +1,4 @@ +<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"] BOMAn application event log entry... +165,local4,notice,Oct 11 22:14:15,mymachine.example.com,,evntslog,BOMAn application event log entry... +#Example from RFC5424, section 6.5 / sample 3 +#Only the first two lines are important, you may place anything behind them! diff --git a/tests/udptester.c b/tests/udptester.c new file mode 100644 index 00000000..ea642db6 --- /dev/null +++ b/tests/udptester.c @@ -0,0 +1,291 @@ +/* Runs a test suite on the rsyslog (and later potentially + * other things). + * + * The name of the test suite must be given as argv[1]. In this config, + * rsyslogd is loaded with config ./testsuites/.conf and then + * test cases ./testsuites/ *. are executed on it. This test driver is + * suitable for testing cases where a message sent (via UDP) results in + * exactly one response. It can not be used in cases where no response + * is expected (that would result in a hang of the test driver). + * Note: each test suite can contain many tests, but they all need to work + * with the same rsyslog configuration. + * + * Part of the testbench for rsyslog. + * + * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Rsyslog is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Rsyslog is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Rsyslog. If not, see . + * + * A copy of the GPL can be found in the file "COPYING" in this distribution. + */ +#include "config.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define EXIT_FAILURE 1 + +static char *srcdir; /* global $srcdir, set so that we can run outside of "make check" */ +static char *testSuite; /* name of current test suite */ + + +void readLine(int fd, char *ln) +{ + char c; + int lenRead; + lenRead = read(fd, &c, 1); + while(lenRead == 1 && c != '\n') { + *ln++ = c; + lenRead = read(fd, &c, 1); + } + *ln = '\0'; +} + + +/* send a message via UDP + * returns 0 if ok, something else otherwise. + */ +int +udpSend(char *buf, int lenBuf) +{ + struct sockaddr_in si_other; + int s, slen=sizeof(si_other); + + if((s=socket(AF_INET, SOCK_DGRAM, 0))==-1) { + perror("socket()"); + return(1); + } + + memset((char *) &si_other, 0, sizeof(si_other)); + si_other.sin_family = AF_INET; + si_other.sin_port = htons(12514); + if(inet_aton("127.0.0.1", &si_other.sin_addr)==0) { + fprintf(stderr, "inet_aton() failed\n"); + return(1); + } + + if(sendto(s, buf, lenBuf, 0, (struct sockaddr*) &si_other, slen)==-1) { + perror("sendto"); + fprintf(stderr, "sendto() failed\n"); + return(1); + } + + close(s); + return 0; +} + + +/* open pipe to test candidate - so far, this is + * always rsyslogd and with a fixed config. Later, we may + * change this. Returns 0 if ok, something else otherwise. + * rgerhards, 2009-03-31 + */ +int openPipe(char *configFile, pid_t *pid, int *pfd) +{ + int pipefd[2]; + pid_t cpid; + char *newargv[] = {"../tools/rsyslogd", "dummy", "-c4", "-u2", "-n", "-irsyslog.pid", + "-M../runtime//.libs", NULL }; + char confFile[1024]; + char *newenviron[] = { NULL }; + + + sprintf(confFile, "-f%s/testsuites/%s.conf", srcdir, configFile); + newargv[1] = confFile; + + if (pipe(pipefd) == -1) { + perror("pipe"); + exit(EXIT_FAILURE); + } + + cpid = fork(); + if (cpid == -1) { + perror("fork"); + exit(EXIT_FAILURE); + } + + if(cpid == 0) { /* Child reads from pipe */ + fclose(stdout); + dup(pipefd[1]); + close(pipefd[1]); + close(pipefd[0]); + fclose(stdin); + execve("../tools/rsyslogd", newargv, newenviron); + } else { + close(pipefd[1]); + *pid = cpid; + *pfd = pipefd[0]; + } + + return(0); +} + + +/* Process a specific test case. File name is provided. + * Needs to return 0 if all is OK, something else otherwise. + */ +int +processTestFile(int fd, char *pszFileName) +{ + FILE *fp; + char *testdata = NULL; + char *expected = NULL; + int ret = 0; + size_t lenLn; + char buf[4096]; + + if((fp = fopen((char*)pszFileName, "r")) == NULL) { + perror((char*)pszFileName); + return(2); + } + + /* skip comments at start of file */ + + getline(&testdata, &lenLn, fp); + while(!feof(fp)) { + if(*testdata == '#') + getline(&testdata, &lenLn, fp); + else + break; /* first non-comment */ + } + + + testdata[strlen(testdata)-1] = '\0'; /* remove \n */ + /* now we have the test data to send */ + if(udpSend(testdata, strlen(testdata)) != 0) + return(2); + + /* next line is expected output + * we do not care about EOF here, this will lead to a failure and thus + * draw enough attention. -- rgerhards, 2009-03-31 + */ + getline(&expected, &lenLn, fp); + expected[strlen(expected)-1] = '\0'; /* remove \n */ + + /* pull response from server and then check if it meets our expectation */ + readLine(fd, buf); + if(strcmp(expected, buf)) { + printf("\nExpected Response:\n'%s'\nActual Response:\n'%s'\n", + expected, buf); + ret = 1; + } + + free(testdata); + free(expected); + fclose(fp); + return(ret); +} + + +/* carry out all tests. Tests are specified via a file name + * wildcard. Each of the files is read and the test carried + * out. + * Returns the number of tests that failed. Zero means all + * success. + */ +int +doTests(int fd, char *files) +{ + int iFailed = 0; + int iTests = 0; + int ret; + char *testFile; + glob_t testFiles; + size_t i = 0; + struct stat fileInfo; + + glob(files, GLOB_MARK, NULL, &testFiles); + + for(i = 0; i < testFiles.gl_pathc; i++) { + testFile = testFiles.gl_pathv[i]; + + if(stat((char*) testFile, &fileInfo) != 0) + continue; /* continue with the next file if we can't stat() the file */ + + ++iTests; + /* all regular files are run through the test logic. Symlinks don't work. */ + if(S_ISREG(fileInfo.st_mode)) { /* config file */ + printf("processing test case '%s' ... ", testFile); + ret = processTestFile(fd, testFile); + if(ret == 0) { + printf("successfully completed\n"); + } else { + printf("failed!\n"); + ++iFailed; + } + } + } + globfree(&testFiles); + + if(iTests == 0) { + printf("Error: no test cases found, no tests executed.\n"); + iFailed = 1; + } else { + printf("Number of tests run: %d, number of failures: %d\n", iTests, iFailed); + } + + return(iFailed); +} + + +/* Run the test suite. This must be called with exactly one parameter, the + * name of the test suite. For details, see file header comment at the top + * of this file. + * rgerhards, 2009-04-03 + */ +int main(int argc, char *argv[]) +{ + int fd; + pid_t pid; + int ret = 0; + char buf[4096]; + char testcases[4096]; + + if(argc != 2) { + printf("Invalid call of udptester\n"); + printf("Usage: udptester testsuite-name\n"); + exit(1); + } + + testSuite = argv[1]; + + if((srcdir = getenv("srcdir")) == NULL) + srcdir = "."; + + printf("Start of udptester run ($srcdir=%s, testsuite=%s)\n", srcdir, testSuite); + + openPipe(argv[1], &pid, &fd); + readLine(fd, buf); + + /* generate filename */ + sprintf(testcases, "%s/testsuites/*.%s", srcdir, testSuite); + if(doTests(fd, testcases) != 0) + ret = 1; + + /* cleanup */ + kill(pid, SIGTERM); + printf("End of udptester run.\n"); + exit(ret); +} -- cgit v1.2.3 From 04b06af335bedcb651ccffd852863b7d17bf20dc Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Fri, 3 Apr 2009 18:20:52 +0200 Subject: improved parser test suite new tests added, now much better --- runtime/datetime.c | 2 +- tests/testsuites/2.parse1 | 3 +++ tests/testsuites/date1.parse1 | 3 +++ tests/testsuites/date2.parse1 | 3 +++ tests/testsuites/date3.parse1 | 3 +++ tests/testsuites/date4.parse1 | 3 +++ tests/testsuites/date5.parse1 | 3 +++ 7 files changed, 19 insertions(+), 1 deletion(-) create mode 100644 tests/testsuites/2.parse1 create mode 100644 tests/testsuites/date1.parse1 create mode 100644 tests/testsuites/date2.parse1 create mode 100644 tests/testsuites/date3.parse1 create mode 100644 tests/testsuites/date4.parse1 create mode 100644 tests/testsuites/date5.parse1 diff --git a/runtime/datetime.c b/runtime/datetime.c index 676f76d5..deb66eb4 100644 --- a/runtime/datetime.c +++ b/runtime/datetime.c @@ -479,7 +479,7 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, char** ppszTS) if(hour > 1970 && hour < 2100) { /* if so, we assume this actually is a year. This is a format found * e.g. in Cisco devices. - * (if you read this 2100+ trying to fix a bug, congratulate myself + * (if you read this 2100+ trying to fix a bug, congratulate me * to how long the code survived - me no longer ;)) -- rgerhards, 2008-11-18 */ year = hour; diff --git a/tests/testsuites/2.parse1 b/tests/testsuites/2.parse1 new file mode 100644 index 00000000..628e06df --- /dev/null +++ b/tests/testsuites/2.parse1 @@ -0,0 +1,3 @@ +<38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) +38,auth,info,Mar 27 19:06:53,source_server,sshd(pam_unix),sshd(pam_unix)[12750]:, session opened for user foo by (uid=0) +# yet another real-life sample where we had some issues with diff --git a/tests/testsuites/date1.parse1 b/tests/testsuites/date1.parse1 new file mode 100644 index 00000000..ffc7c373 --- /dev/null +++ b/tests/testsuites/date1.parse1 @@ -0,0 +1,3 @@ +<38> Mar 7 19:06:53 example tag: testmessage (only date actually tested) +38,auth,info,Mar 7 19:06:53,example,tag,tag:, testmessage (only date actually tested) +# one space in front of the date diff --git a/tests/testsuites/date2.parse1 b/tests/testsuites/date2.parse1 new file mode 100644 index 00000000..8d587d9d --- /dev/null +++ b/tests/testsuites/date2.parse1 @@ -0,0 +1,3 @@ +<38>Mar 7 19:06:53 example tag: testmessage (only date actually tested) +38,auth,info,Mar 7 19:06:53,example,tag,tag:, testmessage (only date actually tested) +# only one space between "Mar" and "7" diff --git a/tests/testsuites/date3.parse1 b/tests/testsuites/date3.parse1 new file mode 100644 index 00000000..940d261e --- /dev/null +++ b/tests/testsuites/date3.parse1 @@ -0,0 +1,3 @@ +<38>Mar 7 2008 19:06:53: example tag: testmessage (only date actually tested) +38,auth,info,Mar 7 19:06:53,example,tag,tag:, testmessage (only date actually tested) +# the year should not be there, nor the colon after the date, but we accept it... diff --git a/tests/testsuites/date4.parse1 b/tests/testsuites/date4.parse1 new file mode 100644 index 00000000..eee5fb09 --- /dev/null +++ b/tests/testsuites/date4.parse1 @@ -0,0 +1,3 @@ +<38>Mar 7 2008 19:06:53 example tag: testmessage (only date actually tested) +38,auth,info,Mar 7 19:06:53,example,tag,tag:, testmessage (only date actually tested) +# the year should not be there, but we accept it... diff --git a/tests/testsuites/date5.parse1 b/tests/testsuites/date5.parse1 new file mode 100644 index 00000000..be32e605 --- /dev/null +++ b/tests/testsuites/date5.parse1 @@ -0,0 +1,3 @@ +<38>Mar 7 19:06:53: example tag: testmessage (only date actually tested) +38,auth,info,Mar 7 19:06:53,example,tag,tag:, testmessage (only date actually tested) +# colon after timestamp is strictly not ok, but we accept it -- cgit v1.2.3 From 010060289a729dd930ac04b72237f0ca0db9ea1d Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Mon, 6 Apr 2009 10:56:27 +0200 Subject: made sure udptester terminates only after rsyslgod it spawned We noticed this race issue under Solaris (thanks to its different scheduler, I guess). In some cases, the previous instance of rsyslogd was not terminated, resulting in a failure on the next test. Now handled correctly. --- tests/udptester.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/udptester.c b/tests/udptester.c index ea642db6..a3c6658d 100644 --- a/tests/udptester.c +++ b/tests/udptester.c @@ -259,6 +259,7 @@ int main(int argc, char *argv[]) { int fd; pid_t pid; + int status; int ret = 0; char buf[4096]; char testcases[4096]; @@ -286,6 +287,7 @@ int main(int argc, char *argv[]) /* cleanup */ kill(pid, SIGTERM); + waitpid(pid, &status, 0); /* wait until instance terminates */ printf("End of udptester run.\n"); exit(ret); } -- cgit v1.2.3 From 8e3c5a9ca3732a41fbb8581b13c49acd699820da Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Mon, 6 Apr 2009 17:55:04 +0200 Subject: improved internal handling of RainerScript functions - building the necessary plumbing to support more functions with decent runtime performance. This is also necessary towards the long-term goal of loadable library modules. - added new RainerScript function "tolower" --- ChangeLog | 7 ++ runtime/expr.c | 4 +- runtime/rsyslog.h | 4 + runtime/vm.c | 252 +++++++++++++++++++++++++++++++++++++++++++++++++++--- runtime/vm.h | 5 +- runtime/vmop.c | 53 ++++++++++-- runtime/vmop.h | 11 ++- runtime/vmprg.c | 28 +++++- runtime/vmprg.h | 4 +- runtime/vmstk.h | 4 +- tests/3.rstest | 4 +- tools/syslogd.c | 1 + 12 files changed, 345 insertions(+), 32 deletions(-) diff --git a/ChangeLog b/ChangeLog index 8d4191d3..24898dc7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,10 @@ +--------------------------------------------------------------------------- +Version 4.1.7 [DEVEL] (rgerhards), 2009-03-?? +- improved internal handling of RainerScript functions, building the + necessary plumbing to support more functions with decent runtime + performance. This is also necessary towards the long-term goal + of loadable library modules. +- added new RainerScript function "tolower" - bugfix: potential abort with DA queue after high watermark is reached There exists a race condition that can lead to a segfault. Thanks go to vbernetr, who performed the analysis and provided patch, which diff --git a/runtime/expr.c b/runtime/expr.c index 38ed1c68..e449d1c7 100644 --- a/runtime/expr.c +++ b/runtime/expr.c @@ -142,7 +142,9 @@ terminal(expr_t *pThis, ctok_t *tok) * we have all relevant information) */ CHKiRet(ctok_token.UnlinkVar(pToken, &pVar)); - CHKiRet(vmprg.AddVarOperation(pThis->pVmprg, opcode_FUNC_CALL, pVar)); /* add to program */ + CHKiRet(var.ConvToString(pVar)); /* make sure we have a string */ + CHKiRet(vmprg.AddCallOperation(pThis->pVmprg, pVar->val.pStr)); /* add to program */ + CHKiRet(var.Destruct(&pVar)); break; case ctok_MSGVAR: dbgoprint((obj_t*) pThis, "MSGVAR\n"); diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index 032d8c04..cea457d8 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -84,6 +84,8 @@ typedef rsRetVal (*errLogFunc_t)(uchar*); /* this is a trick to store a function typedef struct permittedPeers_s permittedPeers_t; /* this should go away in the long term -- rgerhards, 2008-05-19 */ typedef struct permittedPeerWildcard_s permittedPeerWildcard_t; /* this should go away in the long term -- rgerhards, 2008-05-19 */ typedef struct tcpsrv_s tcpsrv_t; +typedef struct vmstk_s vmstk_t; +typedef rsRetVal (*prsf_t)(struct vmstk_s*, int); /* pointer to a RainerScript function */ /* some universal 64 bit define... */ typedef long long int64; @@ -258,6 +260,8 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_FUNC_MISSING_EXPR = -2111, /**< no expression after comma in function call (rainerscript) */ RS_RET_INVLD_NBR_ARGUMENTS = -2112, /**< invalid number of arguments for function call (rainerscript) */ RS_RET_INVLD_FUNC = -2113, /**< invalid function name for function call (rainerscript) */ + RS_RET_DUP_FUNC_NAME = -2114, /**< duplicate function name (rainerscript) */ + RS_RET_UNKNW_FUNC = -2115, /**< unkown function name (rainerscript) */ /* RainerScript error messages (range 1000.. 1999) */ RS_RET_SYSVAR_NOT_FOUND = 1001, /**< system variable could not be found (maybe misspelled) */ diff --git a/runtime/vm.c b/runtime/vm.c index a25476c2..41d3e483 100644 --- a/runtime/vm.c +++ b/runtime/vm.c @@ -26,6 +26,7 @@ #include "config.h" #include #include +#include #include "rsyslog.h" #include "obj.h" @@ -39,6 +40,142 @@ DEFobjCurrIf(vmstk) DEFobjCurrIf(var) DEFobjCurrIf(sysvar) +/* ------------------------------ function registry code and structures ------------------------------ */ + +/* we maintain a registry of known functions */ +/* currently, this is a singly-linked list, this shall become a binary + * tree when we add the real call interface. So far, entries are added + * at the root, only. + */ +typedef struct s_rsf_entry { + cstr_t *pName; /* function name */ + prsf_t rsf; /* pointer to function code */ + struct s_rsf_entry *pNext; /* Pointer to next element or NULL */ +} rsf_entry_t; +rsf_entry_t *funcRegRoot = NULL; + + +/* add a function to the function registry. + * The handed-over cstr_t* object must no longer be used by the caller. + * A duplicate function name is an error. + * rgerhards, 2009-04-06 + */ +static rsRetVal +rsfrAddFunction(uchar *szName, prsf_t rsf) +{ + rsf_entry_t *pEntry; + size_t lenName; + DEFiRet; + + assert(szName != NULL); + assert(rsf != NULL); + + /* first check if we have a duplicate name, with the current approach this means + * we need to go through the whole list. + */ + lenName = strlen((char*)szName); + for(pEntry = funcRegRoot ; pEntry != NULL ; pEntry = pEntry->pNext) + if(!rsCStrSzStrCmp(pEntry->pName, szName, lenName)) + ABORT_FINALIZE(RS_RET_DUP_FUNC_NAME); + + /* unique name, so add to head of list */ + CHKmalloc(pEntry = calloc(1, sizeof(rsf_entry_t))); + CHKiRet(rsCStrConstructFromszStr(&pEntry->pName, szName)); + pEntry->rsf = rsf; + pEntry->pNext = funcRegRoot; + funcRegRoot = pEntry; + +finalize_it: + RETiRet; +} + + +/* find a function inside the function registry + * The caller provides a cstr_t with the function name and receives + * a function pointer back. If no function is found, an RS_RET_UNKNW_FUNC + * error is returned. So if the function returns with RS_RET_OK, the caller + * can savely assume the function pointer is valid. + * rgerhards, 2009-04-06 + */ +static rsRetVal +findRSFunction(cstr_t *pcsName, prsf_t *prsf) +{ + rsf_entry_t *pEntry; + rsf_entry_t *pFound; + DEFiRet; + + assert(prsf != NULL); + + /* find function by list walkthrough. */ + pFound = NULL; + for(pEntry = funcRegRoot ; pEntry != NULL && pFound == NULL ; pEntry = pEntry->pNext) + if(!rsCStrCStrCmp(pEntry->pName, pcsName)) + pFound = pEntry; + + if(pFound == NULL) + ABORT_FINALIZE(RS_RET_UNKNW_FUNC); + + *prsf = pFound->rsf; + +finalize_it: + RETiRet; +} + + +/* find the name of a RainerScript function whom's function pointer + * is known. This function returns the cstr_t object, which MUST NOT + * be modified by the caller. + * rgerhards, 2009-04-06 + */ +static rsRetVal +findRSFunctionName(prsf_t rsf, cstr_t **ppcsName) +{ + rsf_entry_t *pEntry; + rsf_entry_t *pFound; + DEFiRet; + + assert(rsf != NULL); + assert(ppcsName != NULL); + + /* find function by list walkthrough. */ + pFound = NULL; + for(pEntry = funcRegRoot ; pEntry != NULL && pFound == NULL ; pEntry = pEntry->pNext) + if(pEntry->rsf == rsf) + pFound = pEntry; + + if(pFound == NULL) + ABORT_FINALIZE(RS_RET_UNKNW_FUNC); + + *ppcsName = pFound->pName; + +finalize_it: + RETiRet; +} + + +/* free the whole function registry + */ +static void +rsfrRemoveAll(void) +{ + rsf_entry_t *pEntry; + rsf_entry_t *pEntryDel; + + BEGINfunc + pEntry = funcRegRoot; + while(pEntry != NULL) { + pEntryDel = pEntry; + pEntry = pEntry->pNext; + rsCStrDestruct(&pEntryDel->pName); + free(pEntryDel); + } + funcRegRoot = NULL; + ENDfunc +} + + +/* ------------------------------ end function registry code and structures ------------------------------ */ + /* ------------------------------ instruction set implementation ------------------------------ * * The following functions implement the VM's instruction set. @@ -330,7 +467,6 @@ CODESTARTop(PUSHSYSVAR) finalize_it: ENDop(PUSHSYSVAR) - /* The function call operation is only very roughly implemented. While the plumbing * to reach this instruction is fine, the instruction itself currently supports only * functions with a single argument AND with a name that we know. @@ -340,20 +476,9 @@ ENDop(PUSHSYSVAR) */ BEGINop(FUNC_CALL) /* remember to set the instruction also in the ENDop macro! */ var_t *numOperands; - var_t *operand1; - int iStrlen; CODESTARTop(FUNC_CALL) vmstk.PopNumber(pThis->pStk, &numOperands); - if(numOperands->val.num != 1) - ABORT_FINALIZE(RS_RET_INVLD_NBR_ARGUMENTS); - vmstk.PopString(pThis->pStk, &operand1); /* guess there's just one ;) */ - if(!rsCStrSzStrCmp(pOp->operand.pVar->val.pStr, (uchar*) "strlen", 6)) { /* only one supported so far ;) */ -RUNLOG_VAR("%s", rsCStrGetSzStr(operand1->val.pStr)); - iStrlen = strlen((char*) rsCStrGetSzStr(operand1->val.pStr)); -RUNLOG_VAR("%d", iStrlen); - } else - ABORT_FINALIZE(RS_RET_INVLD_FUNC); - PUSHRESULTop(operand1, iStrlen); // TODO: dummy, FIXME + CHKiRet((*pOp->operand.rsf)(pThis->pStk, numOperands->val.num)); var.Destruct(&numOperands); /* no longer needed */ finalize_it: ENDop(FUNC_CALL) @@ -362,6 +487,89 @@ ENDop(FUNC_CALL) /* ------------------------------ end instruction set implementation ------------------------------ */ +/* ------------------------------ begin built-in function implementation ------------------------------ */ +/* note: this shall probably be moved to a separate module, but for the time being we do it directly + * in here. This is on our way to get from a dirty to a clean solution via baby steps that are + * a bit less dirty each time... + * + * The advantage of doing it here is that we do not yet need to think about how to handle the + * exit case, where we must not unload function modules which functions are still referenced. + * + * CALLING INTERFACE: + * The function must pop its parameters off the stack and pop its result onto + * the stack when it is finished. The number of parameters the function was + * called with is provided to it. If the argument count is less then what the function + * expected, it may handle the situation with defaults (or return an error). If the + * argument count is greater than expected, returnung an error is highly + * recommended (use RS_RET_INVLD_NBR_ARGUMENTS for these cases). + * + * All function names are prefixed with "rsf_" (RainerScript Function) to have + * a separate "name space". + * + * rgerhards, 2009-04-06 + */ + + +/* The strlen function, also probably a prototype of how all functions should be + * implemented. + * rgerhards, 2009-04-06 + */ +static rsRetVal +rsf_strlen(vmstk_t *pStk, int numOperands) +{ + DEFiRet; + var_t *operand1; + int iStrlen; + + if(numOperands != 1) + ABORT_FINALIZE(RS_RET_INVLD_NBR_ARGUMENTS); + + /* pop args and do operaton (trivial case here...) */ + vmstk.PopString(pStk, &operand1); + iStrlen = strlen((char*) rsCStrGetSzStr(operand1->val.pStr)); + + /* Store result and cleanup */ + var.SetNumber(operand1, iStrlen); + vmstk.Push(pStk, operand1); +finalize_it: + RETiRet; +} + + +/* The "tolower" function, which converts its sole argument to lower case. + * Quite honestly, currently this is primarily a test driver for me... + * rgerhards, 2009-04-06 + */ +static rsRetVal +rsf_tolower(vmstk_t *pStk, int numOperands) +{ + DEFiRet; + var_t *operand1; + uchar *pSrc; + cstr_t *pcstr; + int iStrlen; + + if(numOperands != 1) + ABORT_FINALIZE(RS_RET_INVLD_NBR_ARGUMENTS); + + /* pop args and do operaton */ + CHKiRet(rsCStrConstruct(&pcstr)); + vmstk.PopString(pStk, &operand1); + pSrc = rsCStrGetSzStr(operand1->val.pStr); + iStrlen = strlen((char*)pSrc); + while(iStrlen--) { + CHKiRet(rsCStrAppendChar(pcstr, tolower(*pSrc++))); + } + + /* Store result and cleanup */ + CHKiRet(rsCStrFinish(pcstr)); + var.SetString(operand1, pcstr); + vmstk.Push(pStk, operand1); +finalize_it: + RETiRet; +} + + /* Standard-Constructor */ BEGINobjConstruct(vm) /* be sure to specify the object type also in END macro! */ @@ -531,10 +739,23 @@ CODESTARTobjQueryInterface(vm) pIf->PopBoolFromStack = PopBoolFromStack; pIf->PopVarFromStack = PopVarFromStack; pIf->SetMsg = SetMsg; + pIf->FindRSFunction = findRSFunction; + pIf->FindRSFunctionName = findRSFunctionName; finalize_it: ENDobjQueryInterface(vm) +/* Exit the vm class. + * rgerhards, 2009-04-06 + */ +BEGINObjClassExit(vm, OBJ_IS_CORE_MODULE) /* class, version */ + rsfrRemoveAll(); + objRelease(sysvar, CORE_COMPONENT); + objRelease(var, CORE_COMPONENT); + objRelease(vmstk, CORE_COMPONENT); +ENDObjClassExit(vm) + + /* Initialize the vm class. Must be called as the very first method * before anything else is called inside this class. * rgerhards, 2008-02-19 @@ -548,6 +769,11 @@ BEGINObjClassInit(vm, 1, OBJ_IS_CORE_MODULE) /* class, version */ /* set our own handlers */ OBJSetMethodHandler(objMethod_DEBUGPRINT, vmDebugPrint); OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, vmConstructFinalize); + + /* register built-in functions // TODO: move to its own module */ + CHKiRet(rsfrAddFunction((uchar*)"strlen", rsf_strlen)); + CHKiRet(rsfrAddFunction((uchar*)"tolower", rsf_tolower)); + ENDObjClassInit(vm) /* vi:set ai: diff --git a/runtime/vm.h b/runtime/vm.h index d2458220..cb3c69d0 100644 --- a/runtime/vm.h +++ b/runtime/vm.h @@ -55,8 +55,11 @@ BEGINinterface(vm) /* name must also be changed in ENDinterface macro! */ rsRetVal (*PopBoolFromStack)(vm_t *pThis, var_t **ppVar); /* there are a few cases where we need this... */ rsRetVal (*PopVarFromStack)(vm_t *pThis, var_t **ppVar); /* there are a few cases where we need this... */ rsRetVal (*SetMsg)(vm_t *pThis, msg_t *pMsg); /* there are a few cases where we need this... */ + /* v2 (4.1.7) */ + rsRetVal (*FindRSFunction)(cstr_t *pcsName, prsf_t *prsf); /* 2009-06-04 */ + rsRetVal (*FindRSFunctionName)(prsf_t rsf, cstr_t **ppcsName); /* 2009-06-04 */ ENDinterface(vm) -#define vmCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define vmCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ /* prototypes */ diff --git a/runtime/vmop.c b/runtime/vmop.c index a343481e..3e001d27 100644 --- a/runtime/vmop.c +++ b/runtime/vmop.c @@ -32,10 +32,12 @@ #include "rsyslog.h" #include "obj.h" #include "vmop.h" +#include "vm.h" /* static data */ DEFobjStaticHelpers DEFobjCurrIf(var) +DEFobjCurrIf(vm) /* forward definitions */ @@ -61,8 +63,10 @@ rsRetVal vmopConstructFinalize(vmop_t __attribute__((unused)) *pThis) /* destructor for the vmop object */ BEGINobjDestruct(vmop) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(vmop) - if(pThis->operand.pVar != NULL) - var.Destruct(&pThis->operand.pVar); + if(pThis->opcode != opcode_FUNC_CALL) { + if(pThis->operand.pVar != NULL) + var.Destruct(&pThis->operand.pVar); + } ENDobjDestruct(vmop) @@ -72,13 +76,19 @@ BEGINobjDebugPrint(vmop) /* be sure to specify the object type also in END and C cstr_t *pStrVar; CODESTARTobjDebugPrint(vmop) vmopOpcode2Str(pThis, &pOpcodeName); - CHKiRet(rsCStrConstruct(&pStrVar)); - CHKiRet(rsCStrFinish(&pStrVar)); - if(pThis->operand.pVar != NULL) { - CHKiRet(var.Obj2Str(pThis->operand.pVar, pStrVar)); + if(pThis->opcode == opcode_FUNC_CALL) { + CHKiRet(vm.FindRSFunctionName(pThis->operand.rsf, &pStrVar)); + assert(pStrVar != NULL); + } else { + CHKiRet(rsCStrConstruct(&pStrVar)); + if(pThis->operand.pVar != NULL) { + CHKiRet(var.Obj2Str(pThis->operand.pVar, pStrVar)); + } } + CHKiRet(rsCStrFinish(&pStrVar)); dbgoprint((obj_t*) pThis, "%.12s\t%s\n", pOpcodeName, rsCStrGetSzStrNoNULL(pStrVar)); - rsCStrDestruct(&pStrVar); + if(pThis->opcode != opcode_FUNC_CALL) + rsCStrDestruct(&pStrVar); finalize_it: ENDobjDebugPrint(vmop) @@ -98,6 +108,7 @@ static rsRetVal Obj2Str(vmop_t *pThis, cstr_t *pstrPrg) { uchar *pOpcodeName; + cstr_t *pcsFuncName; uchar szBuf[2048]; size_t lenBuf; DEFiRet; @@ -107,8 +118,13 @@ Obj2Str(vmop_t *pThis, cstr_t *pstrPrg) vmopOpcode2Str(pThis, &pOpcodeName); lenBuf = snprintf((char*) szBuf, sizeof(szBuf), "%s\t", pOpcodeName); CHKiRet(rsCStrAppendStrWithLen(pstrPrg, szBuf, lenBuf)); - if(pThis->operand.pVar != NULL) - CHKiRet(var.Obj2Str(pThis->operand.pVar, pstrPrg)); + if(pThis->opcode == opcode_FUNC_CALL) { + CHKiRet(vm.FindRSFunctionName(pThis->operand.rsf, &pcsFuncName)); + CHKiRet(rsCStrAppendCStr(pstrPrg, pcsFuncName)); + } else { + if(pThis->operand.pVar != NULL) + CHKiRet(var.Obj2Str(pThis->operand.pVar, pstrPrg)); + } CHKiRet(rsCStrAppendChar(pstrPrg, '\n')); finalize_it: @@ -116,6 +132,23 @@ finalize_it: } +/* set function + * rgerhards, 2009-04-06 + */ +static rsRetVal +vmopSetFunc(vmop_t *pThis, cstr_t *pcsFuncName) +{ + prsf_t rsf; /* pointer to function */ + DEFiRet; + ISOBJ_TYPE_assert(pThis, vmop); + CHKiRet(vm.FindRSFunction(pcsFuncName, &rsf)); /* check if function exists and obtain pointer to it */ + assert(rsf != NULL); /* just double-check, would be very hard to find! */ + pThis->operand.rsf = rsf; +finalize_it: + RETiRet; +} + + /* set operand (variant case) * rgerhards, 2008-02-20 */ @@ -248,6 +281,7 @@ CODESTARTobjQueryInterface(vmop) pIf->ConstructFinalize = vmopConstructFinalize; pIf->Destruct = vmopDestruct; pIf->DebugPrint = vmopDebugPrint; + pIf->SetFunc = vmopSetFunc; pIf->SetOpcode = vmopSetOpcode; pIf->SetVar = vmopSetVar; pIf->Opcode2Str = vmopOpcode2Str; @@ -263,6 +297,7 @@ ENDobjQueryInterface(vmop) BEGINObjClassInit(vmop, 1, OBJ_IS_CORE_MODULE) /* class, version */ /* request objects we use */ CHKiRet(objUse(var, CORE_COMPONENT)); + CHKiRet(objUse(vm, CORE_COMPONENT)); OBJSetMethodHandler(objMethod_DEBUGPRINT, vmopDebugPrint); OBJSetMethodHandler(objMethod_CONSTRUCTION_FINALIZER, vmopConstructFinalize); diff --git a/runtime/vmop.h b/runtime/vmop.h index 938b08fd..67048c26 100644 --- a/runtime/vmop.h +++ b/runtime/vmop.h @@ -26,6 +26,7 @@ #define INCLUDED_VMOP_H #include "ctok_token.h" +#include "vmstk.h" #include "stringbuf.h" /* machine instructions types */ @@ -96,7 +97,8 @@ typedef struct vmop_s { BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ opcode_t opcode; union { - var_t *pVar; /* for function call, this is the name (string) of function to be called */ + var_t *pVar; + prsf_t rsf; /* pointer to function for "call" instruction */ } operand; struct vmop_s *pNext; /* next operation or NULL, if end of program (logically this belongs to vmprg) */ } vmop_t; @@ -112,8 +114,13 @@ BEGINinterface(vmop) /* name must also be changed in ENDinterface macro! */ rsRetVal (*SetVar)(vmop_t *pThis, var_t *pVar); rsRetVal (*Opcode2Str)(vmop_t *pThis, uchar **ppName); rsRetVal (*Obj2Str)(vmop_t *pThis, cstr_t *pstr); + /* v2 */ + rsRetVal (*SetFunc)(vmop_t *pThis, cstr_t *pcsFuncName); ENDinterface(vmop) -#define vmopCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define vmopCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ +/* interface changes, v1 -> v2 + * added SetFuct after existing function pointers -- rgerhards, 2009-04-06 + */ /* the remaining prototypes */ PROTOTYPEObj(vmop); diff --git a/runtime/vmprg.c b/runtime/vmprg.c index 75915025..07757b98 100644 --- a/runtime/vmprg.c +++ b/runtime/vmprg.c @@ -155,7 +155,6 @@ vmprgAddVarOperation(vmprg_t *pThis, opcode_t opcode, var_t *pVar) /* construct and fill vmop */ CHKiRet(vmop.Construct(&pOp)); CHKiRet(vmop.ConstructFinalize(pOp)); - CHKiRet(vmop.ConstructFinalize(pOp)); CHKiRet(vmop.SetOpcode(pOp, opcode)); if(pVar != NULL) CHKiRet(vmop.SetVar(pOp, pVar)); @@ -168,6 +167,32 @@ finalize_it: } +/* this is another shortcut for high-level callers. It is similar to vmprgAddVarOperation + * but adds a call operation. Among others, this include a check if the function + * is known. + */ +static rsRetVal +vmprgAddCallOperation(vmprg_t *pThis, cstr_t *pcsName) +{ + DEFiRet; + vmop_t *pOp; + + ISOBJ_TYPE_assert(pThis, vmprg); + + /* construct and fill vmop */ + CHKiRet(vmop.Construct(&pOp)); + CHKiRet(vmop.ConstructFinalize(pOp)); + CHKiRet(vmop.SetFunc(pOp, pcsName)); + CHKiRet(vmop.SetOpcode(pOp, opcode_FUNC_CALL)); + + /* and add it to the program */ + CHKiRet(vmprgAddOperation(pThis, pOp)); + +finalize_it: + RETiRet; +} + + /* queryInterface function * rgerhards, 2008-02-21 */ @@ -189,6 +214,7 @@ CODESTARTobjQueryInterface(vmprg) pIf->Obj2Str = Obj2Str; pIf->AddOperation = vmprgAddOperation; pIf->AddVarOperation = vmprgAddVarOperation; + pIf->AddCallOperation = vmprgAddCallOperation; finalize_it: ENDobjQueryInterface(vmprg) diff --git a/runtime/vmprg.h b/runtime/vmprg.h index c1042f7d..66f03913 100644 --- a/runtime/vmprg.h +++ b/runtime/vmprg.h @@ -57,8 +57,10 @@ BEGINinterface(vmprg) /* name must also be changed in ENDinterface macro! */ rsRetVal (*AddOperation)(vmprg_t *pThis, vmop_t *pOp); rsRetVal (*AddVarOperation)(vmprg_t *pThis, opcode_t opcode, var_t *pVar); rsRetVal (*Obj2Str)(vmprg_t *pThis, cstr_t *pstr); + /* v2 (4.1.7) */ + rsRetVal (*AddCallOperation)(vmprg_t *pThis, cstr_t *pVar); /* added 2009-04-06 */ ENDinterface(vmprg) -#define vmprgCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define vmprgCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ /* prototypes */ diff --git a/runtime/vmstk.h b/runtime/vmstk.h index 2d45ee4d..06657cf4 100644 --- a/runtime/vmstk.h +++ b/runtime/vmstk.h @@ -27,11 +27,11 @@ #define VMSTK_SIZE 256 /* the vmstk object */ -typedef struct vmstk_s { +struct vmstk_s { BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ var_t *vStk[VMSTK_SIZE];/* the actual stack */ int iStkPtr; /* stack pointer, points to next free location, grows from 0 --> topend */ -} vmstk_t; +}; /* interfaces */ diff --git a/tests/3.rstest b/tests/3.rstest index 93cb941a..e75d9754 100644 --- a/tests/3.rstest +++ b/tests/3.rstest @@ -7,10 +7,10 @@ out: 00000000: push_msgvar msg[cstr] 00000001: push_const abc[cstr] 00000002: push_const 1[nbr] -00000003: func_call strlen[cstr] +00000003: func_call strlen 00000004: strconcat 00000005: push_const 1[nbr] -00000006: func_call strlen[cstr] +00000006: func_call strlen 00000007: push_const 20[nbr] 00000008: push_const 30[nbr] 00000009: add diff --git a/tools/syslogd.c b/tools/syslogd.c index b23c12a7..c85c4371 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -3091,6 +3091,7 @@ GlobalClassExit(void) objRelease(net, LM_NET_FILENAME);/* TODO: the dependency on net shall go away! -- rgerhards, 2008-03-07 */ objRelease(conf, CORE_COMPONENT); objRelease(expr, CORE_COMPONENT); + vmClassExit(); /* this is hack, currently core_modules do not get this automatically called */ objRelease(vm, CORE_COMPONENT); objRelease(var, CORE_COMPONENT); objRelease(datetime, CORE_COMPONENT); -- cgit v1.2.3 From 7db9f96fe9ecb9f8e05d45cc888aa488d8aed85f Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Mon, 6 Apr 2009 18:07:52 +0200 Subject: testcase added (on user request) --- tests/testsuites/3.parse1 | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 tests/testsuites/3.parse1 diff --git a/tests/testsuites/3.parse1 b/tests/testsuites/3.parse1 new file mode 100644 index 00000000..a6b4e884 --- /dev/null +++ b/tests/testsuites/3.parse1 @@ -0,0 +1,3 @@ +<38>Apr 6 15:07:10 lxcvs07 sshd(pam_unix)[31738]: session closed for user cvsadmin +38,auth,info,Apr 6 15:07:10,lxcvs07,sshd(pam_unix),sshd(pam_unix)[31738]:, session closed for user cvsadmin +# yet another real-life sample where we had some issues with -- cgit v1.2.3 From d30388a499ed134b7f0b64180d03d5c7959f2fe4 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Tue, 7 Apr 2009 15:17:49 +0200 Subject: updating changelog for release --- ChangeLog | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index 694ee4b8..adf36932 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,13 +1,11 @@ --------------------------------------------------------------------------- -Version 4.1.6 [DEVEL] (rgerhards), 2009-03-?? +Version 4.1.6 [DEVEL] (rgerhards), 2009-04-07 - added new "csv" property replacer options to enable simple creation of CSV-formatted outputs (format from RFC4180 is used) - implemented function support in RainerScript. That means the engine parses and compile functions, as well as executes a few build-in ones. Dynamic loading and registration of functions is not yet supported - but we now have a good foundation to do that later on. - NOTE: nested function calls are not yet supported due to a design - issue with the function call VM instruction set design. - implemented the strlen() RainerScript function - added a template output module - added -T rsyslogd command line option, enables to specify a directory @@ -36,6 +34,10 @@ Version 4.1.6 [DEVEL] (rgerhards), 2009-03-?? more effcient for some outputs. They new can receive fields not only as a single string but rather in an array where each string is seperated. - added (some) developer documentation for output plugin interface +- bugfix: potential abort with DA queue after high watermark is reached + There exists a race condition that can lead to a segfault. Thanks + go to vbernetr, who performed the analysis and provided patch, which + I only tweaked a very little bit. --------------------------------------------------------------------------- Version 4.1.5 [DEVEL] (rgerhards), 2009-03-11 - bugfix: parser did not correctly parse fields in UDP-received messages -- cgit v1.2.3 From 13344f4928654fc33cc37406164a4a56d22fff93 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Luis=20Fernando=20Mu=C3=B1oz=20Mej=C3=ADas?= Date: Tue, 7 Apr 2009 11:15:52 +0200 Subject: Re-enable parsing host names from message. There was a subtle bug that made all messages fill their HOSTNAME from the source IP (which may be wrong in a long chain of relays) and not by reading the message. This fixes it. --- tools/syslogd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/syslogd.c b/tools/syslogd.c index b23c12a7..a40b34dd 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -1496,7 +1496,7 @@ int parseLegacySyslogMsg(msg_t *pMsg, int flags) * the fields. I think this logic shall work with any type of syslog message. */ bTAGCharDetected = 0; - if(pMsg->bParseHOSTNAME) { + if(flags & PARSE_HOSTNAME) { /* TODO: quick and dirty memory allocation */ /* the memory allocated is far too much in most cases. But on the plus side, * it is quite fast... - rgerhards, 2007-09-20 -- cgit v1.2.3 From 845c6f59b91e9988f856556cbb0e88e275e8e591 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards Date: Tue, 7 Apr 2009 15:23:12 +0200 Subject: final touches for 4.1.6 release --- ChangeLog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ChangeLog b/ChangeLog index adf36932..937ee22f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -38,6 +38,8 @@ Version 4.1.6 [DEVEL] (rgerhards), 2009-04-07 There exists a race condition that can lead to a segfault. Thanks go to vbernetr, who performed the analysis and provided patch, which I only tweaked a very little bit. +- bugfix: imtcp did incorrectly parse hostname/tag + Thanks to Luis Fernando Muñoz Mejías for the patch. --------------------------------------------------------------------------- Version 4.1.5 [DEVEL] (rgerhards), 2009-03-11 - bugfix: parser did not correctly parse fields in UDP-received messages -- cgit v1.2.3