diff options
48 files changed, 1600 insertions, 175 deletions
@@ -1,6 +1,56 @@ --------------------------------------------------------------------------- -Version 7.3.7 [devel] 2013-02-?? +Version 7.3.9 [devel] 2013-03-?? +- bugfix: imudp scheduling parameters did affect main thread, not imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409 +- omlibdbi: now supports transaction interface + if recent enough lbdbi is present +- imuxsock: add ability to NOT create/delete sockets during startup and + shutdown + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=259 +- imfile: errors persisting state file are now reported + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=292 +- imfile: now detects file change when rsyslog was inactive + Previosly, this case could not be detected, so if a file was overwritten + or rotated away while rsyslog was stopped, some data was missing. This + is now detected and the new file being forwarded right from the + beginning. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=228 +- updated systemd files to match current systemd source +- bugfix: build problem on platforms without GLOB_NOMAGIC +- bugfix: build problems on non-Linux platforms +- bugfix: stdout/stderr were not closed on forking + but were closed when running in the forground - this was just reversed + of what it should be. This is a regression of a recent change. +--------------------------------------------------------------------------- +Version 7.3.8 [devel] 2013-03-18 +- imrelp: now supports listening to IPv4/v6 only instead of always both + build now requires librelp 1.0.2 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=378 +- bugfix: mmanon did not build on some platforms (e.g. Ubuntu) +- bugfix: segfault in expression optimizer + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=423 +- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter + To use that functionality, legacy rsyslog.conf syntax had to be used. + Also, the doc was missing information on the "ParseTrusted" set of + config directives. +- bugfix: include files got included in the wrong order + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411 + This happens if an $IncludeConfig directive was done on multiple + files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf). + In that case, the order of include file processing is reversed, which + could lead to all sorts of problems. + Thanks to Nathan Stratton Treadway for his great analysis of the problem, + which made bug fixing really easy. +--------------------------------------------------------------------------- +Version 7.3.7 [devel] 2013-03-12 +- add support for anonymizing IPv4 addresses +- add support for writing to the Linux Journal (omjournal) +- imuxsock: add capability to ignore messages from ourselfes + This helps prevent message routing loops, and is vital to have + if omjournal is used together with traditional syslog. - field() function now supports a string as field delimiter +- added ability to configure debug system via rsyslog.conf +- bugfix: imuxsock segfault when system log socket was used - bugfix: mmjsonparse segfault if new-style config was used - bugfix: script == comparison did not work properly on JSON objects - bugfix: field() function did never return "***FIELD NOT FOUND***" @@ -151,8 +201,54 @@ Version 7.3.0 [devel] 2012-10-09 This was achieved by somewhat reducing the robustness of the zip archive. This is controlled by the new action parameter "VeryReliableZip". ---------------------------------------------------------------------------- -Version 7.2.6 [v7-stable] 2013-01-?? +Version 7.2.7 [v7-stable] 2013-03-?? +- rsyslogd startup information is now properly conveyed back to init + when privileges are beging dropped + Actually, we have moved termination of the parent in front of the + priv drop. So it shall work now in all cases. See code comments in + commit for more details. +- If forking, the parent now waits for a maximum of 60 seconds for + termination by the child +- improved debugging support in forked (auto-backgrounding) mode + The rsyslog debug log file is now continued to be written across the + fork. +- updated systemd files to match current systemd source +- bugfix: imudp scheduling parameters did affect main thread, not imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409 +- bugfix: imuxsock rate-limiting could not be configured via legacy conf + Rate-limiting for the system socket could not be configured via legacy + configuration directives. However, the new-style RainerScript config + options worked. + Thanks to Milan Bartos for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=390 +- bugfix: using group resolution could lead to endless loop + Thanks to Tomas Heinrich for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 +- bugfix: $mmnormalizeuseramsg paramter was specified with wrong type + Thank to Renzhong Zhang for alerting us of the problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=420 +- bugfix: RainerScript getenv() function caused segfault when var was + not found. + Thanks to Philippe Muller for the patch. +- bugfix: several issues in imkmsg + see bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=421#c8 +- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter + To use that functionality, legacy rsyslog.conf syntax had to be used. + Also, the doc was missing information on the "ParseTrusted" set of + config directives. +- doc bugfix: rsyslog.conf man page had invalid file format info + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=418 +---------------------------------------------------------------------------- +Version 7.2.6 [v7-stable] 2013-03-05 - slightly improved config parser error messages when invalid escapes happen +- bugfix: include files got included in the wrong order + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411 + This happens if an $IncludeConfig directive was done on multiple + files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf). + In that case, the order of include file processing is reversed, which + could lead to all sorts of problems. + Thanks to Nathan Stratton Treadway for his great analysis of the problem, + which made bug fixing really easy. - bugfix: omelasticsearch failed when authentication data was provided ... at least in most cases it emitted an error message: "snprintf failed when trying to build auth string" @@ -1151,6 +1247,7 @@ expected that interfaces, even new ones, break during the initial [ported from v4] --------------------------------------------------------------------------- Version 5.10.2 [V5-STABLE], 201?-??-?? +- updated systemd files to match current systemd source - bugfix: spurios error messages from imuxsock about (non-error) EAGAIN Thanks to Marius Tomaschewski for the patch. - imklog: added $klogParseKernelTimestamp option diff --git a/Makefile.am b/Makefile.am index c4499406..f99fa9c7 100644 --- a/Makefile.am +++ b/Makefile.am @@ -177,6 +177,10 @@ if ENABLE_OMHDFS SUBDIRS += plugins/omhdfs endif +if ENABLE_OMJOURNAL +SUBDIRS += plugins/omjournal +endif + if ENABLE_ELASTICSEARCH SUBDIRS += plugins/omelasticsearch endif @@ -225,6 +229,10 @@ if ENABLE_MMAUDIT SUBDIRS += plugins/mmaudit endif +if ENABLE_MMANON +SUBDIRS += plugins/mmanon +endif + if ENABLE_ORACLE SUBDIRS += plugins/omoracle endif @@ -255,7 +255,7 @@ actionResetQueueParams(void) cs.iActionQueueDeqBatchSize = 16; /* default batch size */ cs.iActionQHighWtrMark = 800; /* high water mark for disk-assisted queues */ cs.iActionQLowWtrMark = 200; /* low water mark for disk-assisted queues */ - cs.iActionQDiscardMark = 9800; /* begin to discard messages */ + cs.iActionQDiscardMark = 980; /* begin to discard messages */ cs.iActionQDiscardSeverity = 8; /* discard warning and above */ cs.iActionQueueNumWorkers = 1; /* number of worker threads for the mm queue above */ cs.iActionQueMaxFileSize = 1024*1024; diff --git a/configure.ac b/configure.ac index 450f06ae..650def0f 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[7.3.6],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[7.3.8],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -32,7 +32,7 @@ AC_CANONICAL_HOST PKG_PROG_PKG_CONFIG # modules we require -PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.2) +PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.5) PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0) PKG_CHECK_MODULES([JSON_C], [json]) @@ -66,6 +66,8 @@ save_LIBS=$LIBS LIBS= AC_SEARCH_LIBS(clock_gettime, rt) RT_LIBS=$LIBS +AC_SEARCH_LIBS(mq_getattr, rt) +RT_LIBS="$RT_LIBS $LIBS" LIBS= AC_SEARCH_LIBS(dlopen, dl) DL_LIBS=$LIBS @@ -122,7 +124,9 @@ AC_CHECK_DECL([SCM_CREDENTIALS], [AC_DEFINE(HAVE_SCM_CREDENTIALS, [1], [set defi #include <sys/socket.h>]) AC_CHECK_DECL([SO_TIMESTAMP], [AC_DEFINE(HAVE_SO_TIMESTAMP, [1], [set define])], [], [#include <sys/types.h> #include <sys/socket.h>]) +AC_CHECK_DECL([SYS_gettid], [AC_DEFINE(HAVE_SYS_gettid, [1], [set define])], [], [#include <sys/syscall.h>]) AC_CHECK_MEMBER([struct sysinfo.uptime], [AC_DEFINE(HAVE_SYSINFO_UPTIME, [1], [set define])], [], [#include <sys/sysinfo.h>]) +AC_CHECK_DECL([GLOB_NOMAGIC], [AC_DEFINE(HAVE_GLOB_NOMAGIC, [1], [set define])], [], [#include <glob.h>]) # Check for MAXHOSTNAMELEN AC_MSG_CHECKING(for MAXHOSTNAMELEN) @@ -658,6 +662,11 @@ if test "x$enable_libdbi" = "xyes"; then [dbi_initialize_r], [AC_DEFINE([HAVE_DBI_R], [1], [Define to 1 if libdbi supports the new plugin-safe interface])] ) + AC_CHECK_LIB( + [dbi], + [dbi_conn_transaction_begin], + [AC_DEFINE([HAVE_DBI_TXSUPP], [1], [Define to 1 if libdbi supports transactions])] + ) fi AM_CONDITIONAL(ENABLE_OMLIBDBI, test x$enable_libdbi = xyes) AC_SUBST(LIBDBI_CFLAGS) @@ -875,7 +884,6 @@ fi AM_CONDITIONAL(ENABLE_MMJSONPARSE, test x$enable_mmjsonparse = xyes) - # mmaudit AC_ARG_ENABLE(mmaudit, [AS_HELP_STRING([--enable-mmaudit],[Enable building mmaudit support @<:@default=no@:>@])], @@ -892,6 +900,19 @@ fi AM_CONDITIONAL(ENABLE_MMAUDIT, test x$enable_mmaudit = xyes) +# mmanon +AC_ARG_ENABLE(mmanon, + [AS_HELP_STRING([--enable-mmanon],[Enable building mmanon support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmanon="yes" ;; + no) enable_mmanon="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmanon) ;; + esac], + [enable_mmanon=no] +) +AM_CONDITIONAL(ENABLE_MMANON, test x$enable_mmanon = xyes) + + # RELP support AC_ARG_ENABLE(relp, [AS_HELP_STRING([--enable-relp],[Enable RELP support @<:@default=no@:>@])], @@ -903,7 +924,7 @@ AC_ARG_ENABLE(relp, [enable_relp=no] ) if test "x$enable_relp" = "xyes"; then - PKG_CHECK_MODULES(RELP, relp >= 1.0.1) + PKG_CHECK_MODULES(RELP, relp >= 1.0.2) fi AM_CONDITIONAL(ENABLE_RELP, test x$enable_relp = xyes) @@ -1072,6 +1093,21 @@ AC_ARG_ENABLE(omstdout, ) AM_CONDITIONAL(ENABLE_OMSTDOUT, test x$enable_omstdout = xyes) +# settings for omjournal +AC_ARG_ENABLE(omjournal, + [AS_HELP_STRING([--enable-omjournal],[Compiles omjournal @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_omjournal="yes" ;; + no) enable_omjournal="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-omjournal) ;; + esac], + [enable_omjournal=no] +) +if test "x$enable_omjournal" = "xyes"; then + PKG_CHECK_MODULES([LIBSYSTEMD_JOURNAL], [libsystemd-journal >= 197]) +fi +AM_CONDITIONAL(ENABLE_OMJOURNAL, test x$enable_omjournal = xyes) + # settings for pmlastmsg AC_ARG_ENABLE(pmlastmsg, @@ -1313,6 +1349,7 @@ AC_CONFIG_FILES([Makefile \ plugins/omhdfs/Makefile \ plugins/omprog/Makefile \ plugins/omstdout/Makefile \ + plugins/omjournal/Makefile \ plugins/pmrfc3164sd/Makefile \ plugins/pmlastmsg/Makefile \ plugins/pmcisconames/Makefile \ @@ -1327,7 +1364,7 @@ AC_CONFIG_FILES([Makefile \ plugins/impstats/Makefile \ plugins/imrelp/Makefile \ plugins/imdiag/Makefile \ - plugins/imzmq3/Makefile \ + plugins/imzmq3/Makefile \ plugins/omtesting/Makefile \ plugins/omgssapi/Makefile \ plugins/ommysql/Makefile \ @@ -1339,11 +1376,12 @@ AC_CONFIG_FILES([Makefile \ plugins/omoracle/Makefile \ plugins/omudpspoof/Makefile \ plugins/ommongodb/Makefile \ - plugins/omhiredis/Makefile \ - plugins/omzmq3/Makefile \ + plugins/omhiredis/Makefile \ + plugins/omzmq3/Makefile \ plugins/mmnormalize/Makefile \ plugins/mmjsonparse/Makefile \ plugins/mmaudit/Makefile \ + plugins/mmanon/Makefile \ plugins/omelasticsearch/Makefile \ plugins/sm_cust_bindcdr/Makefile \ plugins/mmsnmptrapd/Makefile \ @@ -1364,6 +1402,7 @@ echo " GUI components will be built: $enable_gui" echo " Unlimited select() support enabled: $enable_unlimited_select" echo " uuid support enabled: $enable_uuid" echo " GuardTime signature support enabled: $enable_guardtime" +echo " anonymization support enabled: $enable_mmanon" echo echo "---{ input plugins }---" echo " Klog functionality enabled: $enable_klog ($os_type)" @@ -1380,6 +1419,7 @@ echo "---{ output plugins }---" echo " Mail support enabled: $enable_mail" echo " omprog module will be compiled: $enable_omprog" echo " omstdout module will be compiled: $enable_omstdout" +echo " omjournal module will be compiled: $enable_omjournal" echo " omhdfs module will be compiled: $enable_omhdfs" echo " omelasticsearch module will be compiled: $enable_elasticsearch" echo " omruleset module will be compiled: $enable_omruleset" diff --git a/doc/Makefile.am b/doc/Makefile.am index 6501cf6c..de2e1df5 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -34,6 +34,8 @@ html_files = \ ompipe.html \ omfwd.html \ omfile.html \ + omjournal.html \ + mmanon.html \ omusrmsg.html \ omstdout.html \ omudpspoof.html \ diff --git a/doc/debug.html b/doc/debug.html index 8b104d80..996bf5c8 100644 --- a/doc/debug.html +++ b/doc/debug.html @@ -75,6 +75,26 @@ rsyslog core, we get a number of data structures wrong. <p>For these reasons, we utilize environment variables to initialize and configure the debugging system. We understand this may be somewhat painful, but now you know there are at least some good reasons for doing so. +<p>HOWEVER, if you have a too hard time to set debug instructions using the environment +variables, there is a cure, described in the next paragraph. + +<h2>Enabling Debug via rsyslog.conf</h2> +<p>As described in the previous paragraph, enabling debug via rsyslog.conf +may not be perfect for some debugging needs, but basic debug output will work - and +that is what most often is requried. There are limited options available, but these +cover the most important use cases. +<p>Debug processing is done via legacy config statements. There currently +is no plan to move these over to the v6+ config system. Availabe settings are +<ul> +<li>$DebugFile <filename> - sets the debug file name +<li>$DebugLevel <0|1|2> - sets the respective debug level, where +0 means debug off, 1 is debug on demand activated (but debug mode off) +and 2 is full debug mode. +</ul> +<p>Note that in theory it is forbidden to specify these parameters more +than once. However, we do not enforce that and if it happens results +are undefined. + <h2>Getting debug information from a running Instance</h2> <p>It is possible to obtain debugging information from a running instance, but this requires some setup. We assume that the instance runs in the background, so debug output to @@ -143,7 +163,7 @@ instance of rsyslogd can be aborted by pressing ctl-c. <p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body> diff --git a/doc/imfile.html b/doc/imfile.html index 0997e382..f37f7055 100644 --- a/doc/imfile.html +++ b/doc/imfile.html @@ -77,7 +77,9 @@ created in the rsyslog working directory (configurable via $WorkDirectory). Be careful to use unique names for different files being monitored. If there are duplicates, all sorts of "interesting" things may happen. Rsyslog currently does not check if a name is -specified multiple times.</li> +specified multiple times. +Note that when $WorkDirectory is not set or set to a non-writable +location, the state file will not be generated.</li> <li><span style="font-weight: bold;">Facility facility</span><br> The syslog facility to be assigned to lines read. Can be specified in @@ -141,17 +143,17 @@ your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.<br> </p> -<textarea rows="15" cols="60">module(load="folder/to/rsyslog/plugins/imfile/.libs/imfile" PollingInterval="10") #needs to be done just once +<textarea rows="15" cols="60">module(load="imfile" PollingInterval="10") #needs to be done just once # File 1 input(type="imfile" File="/path/to/file1" -Tag="tag1" -StateFile="/var/spool/rsyslog/statefile1" -Severity="error" -Facility="local7") + Tag="tag1" + StateFile="/var/spool/rsyslog/statefile1" + Severity="error" + Facility="local7") # File 2 input(type="imfile" File="/path/to/file2" -Tag="tag2" -StateFile="/var/spool/rsyslog/statefile2") + Tag="tag2" + StateFile="/var/spool/rsyslog/statefile2") # ... and so on ... # </textarea> @@ -210,8 +212,7 @@ your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.<br> </p> -<textarea rows="15" cols="60">$ModLoad imfile # -needs to be done just once +<textarea rows="15" cols="60">$ModLoad imfile # needs to be done just once # File 1 $InputFileName /path/to/file1 $InputFileTag tag1: diff --git a/doc/imrelp.html b/doc/imrelp.html index 856aff82..9f3e4875 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -47,7 +47,7 @@ not specific ones. This is due to a currently existing limitation in librelp. <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> </p> -<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imrelp/.libs/imrelp") # needs to be done just once +<textarea rows="15" cols="60">module(load="imrelp") # needs to be done just once input(type="imrelp" port="20514") </textarea> diff --git a/doc/imudp.html b/doc/imudp.html index f2d04d65..a86c9939 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -33,7 +33,7 @@ the value, the less precise the timestamp. <li><b>SchedulingPolicy</b> <rr/fifo/other><br> Can be used the set the scheduler priority, if the necessary functionality is provided by the platform. Most useful to select "fifo" for real-time -processing under Linux (and thus reduce chance of packet loss). +processing under Linux (and thus reduce chance of packet loss). <li><b>SchedulingPriority</b> <number><br> Scheduling priority to use. </ul> @@ -57,7 +57,11 @@ burst in number of messages. Default is 10,000. </ul> <b>Caveats/Known Bugs:</b> <ul> -<li>currently none known</li> +<li>Scheduling parameters are set <b>after</b> privileges have been dropped. +In most cases, this means that setting them will not be possible after +privilege drop. This may be worked around by using a sufficiently-privileged +user account. +</li> </ul> <p><b>Sample:</b></p> <p>This sets up an UPD server on port 514:<br> diff --git a/doc/imuxsock.html b/doc/imuxsock.html index a962f814..0affe8c3 100644 --- a/doc/imuxsock.html +++ b/doc/imuxsock.html @@ -65,6 +65,12 @@ you must turn it on (via SysSock.Annotate and Annotate). <li><b>SysSock.IgnoreTimestamp</b> [<b>on</b>/off]<br> Ignore timestamps included in the messages, applies to messages received via the system log socket. </li> +<li><b>SysSock.IgnoreOwnMessages</b> [<b>on</b>/off] (available since 7.3.7)<br> +Ignores messages that originated from the same instance of rsyslogd. There usually +is no reason to receive messages from ourselfs. This setting is vital +when writing messages to the Linux journal. See <a href="omjournal.html">omjournal</a> +module documentation for a more in-depth description. +</li> <li><b>SysSock.Use</b> (imuxsock) [on/<b>off</b>] do NOT listen for the local log socket. This is most useful if you run multiple instances of rsyslogd where only one shall handle the system log socket. @@ -92,12 +98,28 @@ messages that shall be rate-limited. </li> <li><b>SysSock.Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the system log socket.</li> +<li><b>SysSock.ParseTrusted</b> <on/<b>off</b>> if Annotation is turned on, create +JSON/lumberjack properties out of the trusted properties (which can be accessed +via RainerScript JSON Variables, e.g. "$!pid") instead of adding them to the message. +</li> +<li><b>SysSock.Unlink</b> <<b>on</b>/off> (available since 7.3.9)<br> +if turned on (default), the system socket is unlinked and re-created when +opened and also unlinked when finally closed. Note that this setting has +no effect when running under systemd control (because systemd handles +the socket). +</li> </ul> <p><b>Input Instance Parameters</b></p> <ul> <li><b>IgnoreTimestamp</b> [<b>on</b>/off] <br>Ignore timestamps included in the message. Applies to the next socket being added.</li> +<li><b>IgnoreOwnMessages</b> [<b>on</b>/off] (available since 7.3.7)<br> +Ignore messages that originated from the same instance of rsyslogd. There usually +is no reason to receive messages from ourselfs. This setting is vital +when writing messages to the Linux journal. See <a href="omjournal.html">omjournal</a> +module documentation for a more in-depth description. +</li> <li><b>FlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied to the next socket.</li> <li><b>RateLimit.Interval</b> [number] - specifies the rate-limiting @@ -148,6 +170,15 @@ will only affect the next one and then automatically be reset. This functionalit that the local hostname can be overridden in cases where that is desired.</li> <li><b>Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the non-system log socket in question.</li> +<li><b>ParseTrusted</b> <on/<b>off</b>> equivalent to the SysSock.ParseTrusted module +parameter, but applies to the input that is being defined. +<li><b>Unlink</b> <<b>on</b>/off> (available since 7.3.9)<br> +if turned on (default), the socket is unlinked and re-created when +opened and also unlinked when finally closed. Set it to off if you +handle socket creation yourself. Note that handling socket creation +oneself has the advantage that a limited amount of messages may be +queued by the OS if rsyslog is not running. +</li> </ul> <b>Caveats/Known Bugs:</b><br> @@ -160,12 +191,20 @@ change the array size in imuxsock.c. <p>The following sample is the minimum setup required to accept syslog messages from applications running on the local system.<br> </p> -<textarea rows="2" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="2" cols="70">module(load="imuxsock" # needs to be done just once SysSock.FlowControl="on") # enable flow control (use if needed) </textarea> + +<p>The following sample is similiar to the first one, but enables trusted +properties, which are put into JSON/lumberjack variables. +<br> +</p> +<textarea rows="2" cols="70">module(load="imuxsock" SysSock.Annotate="on" SysSock.ParseTrusted="on") +</textarea> + <p>The following sample is a configuration where rsyslogd pulls logs from two jails, and assigns different hostnames to each of the jails: </p> -<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once +<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once input(type="imuxsock" HostName="jail1.example.net" Socket="/jail/1/dev/log") input(type="imuxsock" HostName="jail2.example.net" Socket="/jail/2/dev/log") @@ -176,18 +215,18 @@ system. As rsyslogd starts up before the sshd, it needs to create the socket directories, because it otherwise can not open the socket and thus not listen to openssh messages. Note that it is vital not to place any other socket between the CreatePath and the Socket.</p> -<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once +<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once input(type="imuxsock" Socket="/var/run/sshd/dev/log" CreatePath="on") </textarea> <p>The following sample is used to turn off input rate limiting on the system log socket. -<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once SysSock.RateLimit.Interval="0") # turn off rate limiting </textarea> <p>The following sample is used activate message annotation and thus trusted properties on the system log socket. -<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once SysSock.Annotate="on") </textarea> @@ -231,6 +270,7 @@ equivalent to: SysSock.IgnoreTimestamp.</li> <li><b>$InputUnixListenSocketHostName</b> <hostname> equivalent to: HostName.</li> <li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> equivalent to: Annotate.</li> <li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> equivalent to: SysSock.Annotate.</li> +<li><b>$SystemLogSocketParseTrusted</b> <on/<b>off</b>> equivalent to: SysSock.ParseTrusted.</li> </ul> <b>Caveats/Known Bugs:</b><br> @@ -283,7 +323,7 @@ $SystemLogSocketAnnotate on <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/manual.html b/doc/manual.html index 05a03e29..ca54a04a 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ professional services</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 7.3.6 (devel branch) of rsyslog.</b> +<p><b>This documentation is for version 7.3.8 (devel branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might diff --git a/doc/mmanon.html b/doc/mmanon.html new file mode 100644 index 00000000..16065a1f --- /dev/null +++ b/doc/mmanon.html @@ -0,0 +1,119 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>IP Address Anonimization Module (mmanon)</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>IP Address Anonimization Module (mmanon)</h1> +<p><b>Module Name: mmanon</b></p> +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Available since</b>: 7.3.7</p> +<p><b>Description</b>:</p> +<p>The mmanon module permits to anonymize IP addresses. It is a message +modification module that actually changes the IP address inside the message, +so after calling mmanon, the original message can no longer be obtained. +Note that anonymization will break digital signatures on the message, if +they exist. +<p><i>How are IP-Addresses defined?</i> +<p>We assume that an IP address consists of four octets in dotted notation, +where each of the octets has a value between 0 and 255, inclusively. After +the last octet, there must be either a space or a colon. So, for example, +"1.2.3.4 Test" and "1.2.3.4:514 Test" are detected as containing valid IP +addresses, whereas this is not the case for "1.2.300.4 Test" or +"1.2.3.4-Test". The message text may contain multiple addresses. If so, +each of them is anonimized (according to the same rules). +<b>Important:</b> We may change the set of acceptable characters after +the last octet in the future, if there are good reasons to do so. +<p> </p> + +<p><b>Module Configuration Parameters</b>:</p> +<p>Currently none. +<p> </p> +<p><b>Action Confguration Parameters</b>:</p> +<ul> +<li><b>mode</b> - default "rewrite"<br> +There exists the "simple" and "rewrite" mode. In simple mode, only octets +as whole can be anonymized and the length of the message is never changed. +This means that when the last three octets of the address 10.1.12.123 are +anonymized, the result will be 10.0.00.000. This means that the length of the +original octets is still visible and may be used to draw some privacy-evasive +conclusions. This mode is slightly faster than "overwrite" mode, and this +may matter in high throughput environments.<br> +The default "rewrite" mode will do full anonymization of any number of bits +and it will also normlize the address, so that no information about the +original IP address is available. So in the above example, 10.1.12.123 would +be anonymized to 10.0.0.0. +<li><b>ipv4.bits</b> - default 16<br> +This set the number of bits that should be anonymized (bits are from the +right, so lower bits are anonymized first). This setting permits to save +network information while still anonymizing user-specific data. The more +bits you discard, the better the anonymization obviously is. The default +of 16 bits reflects what German data privacy rules consider as being +sufficinetly anonymized. We assume, this can also be used as a rough +but conservative guideline for other countries.<br> +Note: when in simple mode, only bits on a byte boundary can be specified. +As such, any value other than 8, 16, 24 or 32 is invalid. If an invalid +value is given, it is rounded to the next byte boundary (so we favor stronger +anonymization in that case). For example, a bit value of 12 will become 16 in +simple mode (an error message is also emitted). +<li><b>replacementChar</b> - default "x"<br> +In simple mode, this sets the character +that the to-be-anonymized part of the IP address is to be overwritten +with. In rewrite mode, this parameter is <b>not permitted</b>, as in +this case we need not necessarily rewrite full octets. As such, the anonymized +part is always zero-filled and replacementChar is of no use. If it is +specified, an error message is emitted and the parameter ignored. +</ul> + +<p><b>Caveats/Known Bugs:</b> +<ul> +<li><b>only IPv4</b> is supported +</ul> + +<p><b>Samples:</b></p> +<p>In this snippet, we write one file without anonymization and another one +with the message anonymized. Note that once mmanon has run, access to the +original message is no longer possible (execept if stored in user +variables before anonymization). +<p><textarea rows="5" cols="60">module(load="mmanon") +action(type="omfile" file="/path/to/non-anon.log") +action(type="mmanon") +action(type="omfile" file="/path/to/anon.log") +</textarea> + +<p>This next snippet is almost identical to the first one, but +here we anonymize the full IPv4 address. Note that by +modifying the number of bits, you can anonymize different parts +of the address. Keep in mind that in simple mode (used here), the bit values +must match IP address bytes, so for IPv4 only the values 8, 16, 24 and +32 are valid. Also, in this example the replacement is done +via asterisks instead of lower-case "x"-letters. Also keep in mind that +"replacementChar" can only be set in simple mode. +<p><textarea rows="5" cols="60">module(load="mmanon") +action(type="omfile" file="/path/to/non-anon.log") +action(type="mmanon" ipv4.bits="32" mode="simple" replacementChar="*") +action(type="omfile" file="/path/to/anon.log") +</textarea> + +<p>The next snippet is also based on the first one, but anonimzes an +"odd" number of bits, 12. The value of 12 is used by some folks as a +compromise between keeping privacy and still permiting to gain some +more in-depth insight from log files. Note that anonymizing 12 bits +may be insufficient to fulfill legal requirements (if such exist). +<p><textarea rows="5" cols="60">module(load="mmanon") +action(type="omfile" file="/path/to/non-anon.log") +action(type="mmanon" ipv4.bits="12") +action(type="omfile" file="/path/to/anon.log") +</textarea> + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/omfile.html b/doc/omfile.html index 75ac5f49..2c5ab97a 100644 --- a/doc/omfile.html +++ b/doc/omfile.html @@ -90,7 +90,7 @@ <p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> <p><b>Sample:</b></p> <p>The following command writes all syslog messages into a file.</p> -<textarea rows="5" cols="60">Module (path="builtin:omfile") +<textarea rows="5" cols="60">Module (load="builtin:omfile") *.* action(type="omfile" DirCreateMode="0700" FileCreateMode="0644" diff --git a/doc/omjournal.html b/doc/omjournal.html new file mode 100644 index 00000000..c42d9841 --- /dev/null +++ b/doc/omjournal.html @@ -0,0 +1,83 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> +<meta http-equiv="Content-Language" content="en"> +<title>Linux Journal Output Module (omjournal)</title></head> + +<body> +<a href="rsyslog_conf_modules.html">back</a> + +<h1>Linux Journal Output Module (omjournal)</h1> +<p><b>Module Name: omjournal</b></p> +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Available since</b>: 7.3.7</p> +<p><b>Description</b>:</p> +<p>The omjournal output module provides an interface to the Linux journal. +It is meant to be used in those cases where the Linux journal is being used +as the sole system log database. With omjournal, messages from various +sources (e.g. files and remote devices) can also be written to the journal +and processed by its tools. +<p>A typical use case we had on our mind is a SOHO environment, where the +user wants to include syslog data obtained from the local router to be +part of the journal data. +<p> </p> + +<p><b>Module Configuration Parameters</b>:</p> +<p>Currently none. +<p> </p> +<p><b>Action Confguration Parameters</b>:</p> +<p>Currently none. + +<p><b>Caveats/Known Bugs:</b> +<ul> +<li>One needs to be careful that no message routing loop is created. The +systemd journal forwards messages it receives to the traditional syslog +system (if present). That means rsyslog will receive the same message that +it just wrote as new input on imuxsock. If not handled specially and assuming +all messages be written to the journal, the message would be emitted to the +journal again and a deadly loop is started. +<p>To prevent that, imuxsock by default does not accept messages originating +from its own process ID, aka it ignores messages from the current instance of +rsyslogd. However, this setting can be changed, and if so the problem may occur. +</ul> + +<p><b>Sample:</b></p> +<p>We assume we have a DSL router inside the network and would like to +receive its syslog message into the journal. Note that this configuration can be +used without havoing any other syslog functionality at all (most importantly, there +is no need to write any file to /var/log!). We assume syslog over UDP, as this +is the most probable choice for the SOHO environment that this use case reflects. +To log to syslog data to the journal, add the following snippet to rsyslog.conf: +<textarea rows="20" cols="60">/* first, we make sure all necessary + * modules are present: + */ +module(load="imudp") # input module for UDP syslog +module(load="omjournal") # output module for journal + +/* then, define the actual server that listens to the + * router. Note that 514 is the default port for UDP + * syslog and that we use a dedicated ruleset to + * avoid mixing messages with the local log stream + * (if there is any). + */ +input(type="imudp" port="514" ruleset="writeToJournal") + +/* inside that ruleset, we just write data to the journal: */ +ruleset(name="writeToJournal") { + action(type="omjournal") +} +</textarea> +<p>Note that this can be your sole rsyslog.conf if you do not use rsyslog +for anything else than receving the router syslog messages. +<p>If you do not receive messages, <b>you probably need to enable inbound UDP +syslog traffic in your firewall</b>. + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual +index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL +version 3 or higher.</font></p> + +</body></html> diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html index fad1b110..00a700d4 100644 --- a/doc/rsyslog_conf_basic_structure.html +++ b/doc/rsyslog_conf_basic_structure.html @@ -49,7 +49,8 @@ after the stop statement are never evaluated. <h3>Data Manipulation Statements</h3> <ul> -<li><b>set</b> - sets a user variable +<li><b>set</b> - <a href="http://www.rsyslog.com/how-to-set-variables-in-rsyslog-v7/">sets</a> +a user variable <li><b>unset</b> - deletes a previously set user variable </ul> @@ -80,6 +81,14 @@ a message comes in via that input, the "program" (ruleset) bound to it will be e (but not any other!). <p>There is detail documentation available for <a href="multi_ruleset">rsyslog rulesets</a>. +<p>For quick reference, rulesets are defined as follows: +<pre> +ruleset(name="rulesetname") { + action(type="omfile" file="/path/to/file") + action(type="..." ...) + /* and so on... */ +} +</pre> <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html index cb31bd9a..554b20f4 100644 --- a/doc/rsyslog_conf_modules.html +++ b/doc/rsyslog_conf_modules.html @@ -53,6 +53,7 @@ and messages be transmitted to various different targets. <ul> <li><a href="omfile.html">omfile</a> - file output module</li> <li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li> +<li><a href="omjournal.html">omjournal</a> - Linux journal output module</li> <li><a href="ompipe.html">ompipe</a> - named pipe output module</li> <li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li> <li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> @@ -100,16 +101,14 @@ They can be implemented using either the output module or the parser module inte From the rsyslog core's point of view, they actually are output or parser modules, it is their implementation that makes them special. <p>Currently, there exists only a limited set of such modules, but new ones could be written with -the methods the engine provides. They could be used, for example, to: -<ul> -<li>anonymize message content -<li>add dynamically computed content to message (fields) -</ul> +the methods the engine provides. They could be used, for example, to +add dynamically computed content to message (fields). <p>Message modification modules are usually written for one specific task and thus usually are not generic enough to be reused. However, existing module's code is probably an excellent starting base for writing a new module. Currently, the following modules exist inside the source tree: <ul> +<li><a href="mmanon.html">mmanon</a> - used to anonymize log messages. <li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages. Note that this actually is a <b>generic</b> module. <li><a href="mmjsonparse.html">mmjsonparse</a> - used to interpret CEE/lumberjack @@ -182,7 +181,7 @@ filter settings. This graphic above is a high-level message flow diagram. [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body> diff --git a/grammar/lexer.l b/grammar/lexer.l index e1f5a9c3..237eb2a6 100644 --- a/grammar/lexer.l +++ b/grammar/lexer.l @@ -310,6 +310,7 @@ cnfSetLexFile(char *fname) currbs = bs; cnfcurrfn = bs->fn; yylineno = 1; + dbgprintf("config parser: pushed file %s on top of stack\n", fname); done: if(r != 0) { @@ -337,6 +338,7 @@ popfile(void) * necessary, as otherwise we may provide wrong file name information * at the end of include files as well. -- rgerhards, 2011-07-22 */ + dbgprintf("config parser: reached end of file %s\n", bs->fn); yy_delete_buffer(bs->bs); if(bs->prev != NULL) free(bs->fn); @@ -346,12 +348,16 @@ popfile(void) currbs = bs->prev; free(bs); - if(currbs == NULL) + if(currbs == NULL) { + dbgprintf("config parser: parsing completed\n"); return 1; /* all processed */ + } yy_switch_to_buffer(currbs->bs); yylineno = currbs->lineno; cnfcurrfn = currbs->fn; + dbgprintf("config parser: resume parsing of file %s at line %d\n", + cnfcurrfn, yylineno); return 0; } diff --git a/grammar/rainerscript.c b/grammar/rainerscript.c index 6dbdad63..8ef7429d 100644 --- a/grammar/rainerscript.c +++ b/grammar/rainerscript.c @@ -1273,8 +1273,12 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) estr = var2String(&r[0], &bMustFree); str = (char*) es_str2cstr(estr, NULL); envvar = getenv(str); + if(envvar == NULL) { + ret->d.estr = es_newStr(0); + } else { + ret->d.estr = es_newStrFromCStr(envvar, strlen(envvar)); + } ret->datatype = 'S'; - ret->d.estr = es_newStrFromCStr(envvar, strlen(envvar)); if(bMustFree) es_deleteStr(estr); if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); free(str); @@ -2635,7 +2639,7 @@ cnfexprOptimize_CMP_severity_facility(struct cnfexpr *expr) /* optimize a comparison with a variable as left-hand operand * NOTE: Currently support CMP_EQ, CMP_NE only and code NEEDS - * TO BE CHANGED for other comparisons! + * TO BE CHANGED fgr other comparisons! */ static inline struct cnfexpr* cnfexprOptimize_CMP_var(struct cnfexpr *expr) @@ -2790,10 +2794,10 @@ cnfexprOptimize(struct cnfexpr *expr) expr->l = expr->r; expr->r = exprswap; } - } else if(expr->l->nodetype == 'V') { - expr = cnfexprOptimize_CMP_var(expr); } - if(expr->r->nodetype == 'A') { + if(expr->l->nodetype == 'V') { + expr = cnfexprOptimize_CMP_var(expr); + } else if(expr->r->nodetype == 'A') { cnfexprOptimize_CMPEQ_arr((struct cnfarray *)expr->r); } break; @@ -3233,7 +3237,7 @@ cnfDoInclude(char *name) { char *cfgFile; char *finalName; - unsigned i; + int i; int result; glob_t cfgFiles; struct stat fileInfo; @@ -3252,12 +3256,16 @@ cnfDoInclude(char *name) /* Use GLOB_MARK to append a trailing slash for directories. */ /* Use GLOB_NOMAGIC to detect wildcards that match nothing. */ - result = glob(finalName, GLOB_MARK | GLOB_NOMAGIC, NULL, &cfgFiles); - +#ifdef HAVE_GLOB_NOMAGIC /* Silently ignore wildcards that match nothing */ + result = glob(finalName, GLOB_MARK | GLOB_NOMAGIC, NULL, &cfgFiles); if(result == GLOB_NOMATCH) { - return 0; - } +#else + result = glob(finalName, GLOB_MARK, NULL, &cfgFiles); + if(result == GLOB_NOMATCH && containsGlobWildcard(finalName)) { +#endif /* HAVE_GLOB_NOMAGIC */ + return 0; + } if(result == GLOB_NOSPACE || result == GLOB_ABORTED) { char errStr[1024]; @@ -3269,7 +3277,12 @@ cnfDoInclude(char *name) return 1; } - for(i = 0; i < cfgFiles.gl_pathc; i++) { + /* note: bison "stacks" the files, so we need to submit them + * in reverse order to the *stack* in order to get the proper + * parsing order. Also see + * http://bugzilla.adiscon.com/show_bug.cgi?id=411 + */ + for(i = cfgFiles.gl_pathc - 1; i >= 0 ; i--) { cfgFile = cfgFiles.gl_pathv[i]; if(stat(cfgFile, &fileInfo) != 0) { char errStr[1024]; diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index 0f155c10..349acead 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -235,6 +235,7 @@ openFile(fileInfo_t *pThis) /* read back in the object */ CHKiRet(obj.Deserialize(&pThis->pStrm, (uchar*) "strm", psSF, NULL, pThis)); + strm.CheckFileChange(pThis->pStrm); CHKiRet(strm.SeekCurrOffs(pThis->pStrm)); /* note: we do not delete the state file, so that the last position remains @@ -738,12 +739,20 @@ persistStrmState(fileInfo_t *pInfo) CHKiRet(strm.ConstructFinalize(psSF)); CHKiRet(strm.Serialize(pInfo->pStrm, psSF)); + CHKiRet(strm.Flush(psSF)); CHKiRet(strm.Destruct(&psSF)); finalize_it: if(psSF != NULL) strm.Destruct(&psSF); + + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "imfile: could not persist state " + "file %s - data may be repeated on next " + "startup. Is WorkDirectory set?", + pInfo->pszStateFile); + } RETiRet; } diff --git a/plugins/imkmsg/kmsg.c b/plugins/imkmsg/kmsg.c index b771d68a..f1815f25 100644 --- a/plugins/imkmsg/kmsg.c +++ b/plugins/imkmsg/kmsg.c @@ -89,6 +89,10 @@ submitSyslog(uchar *buf) for (; isdigit(*buf); buf++) { timestamp += (timestamp * 10) + (*buf - '0'); } + + while (*buf != ';') { + buf++; /* skip everything till the first ; */ + } buf++; /* skip ; */ /* get message */ @@ -103,7 +107,7 @@ submitSyslog(uchar *buf) if (*buf != '\0') /* message has appended properties, skip \n */ buf++; - while (strlen((char *)buf)) { + while (*buf) { /* get name of the property */ buf++; /* skip ' ' */ offs = 0; @@ -174,18 +178,22 @@ static void readkmsg(void) { int i; - uchar pRcv[8096+1]; + uchar pRcv[8192+1]; char errmsg[2048]; for (;;) { dbgprintf("imkmsg waiting for kernel log line\n"); /* every read() from the opened device node receives one record of the printk buffer */ - i = read(fklog, pRcv, 8096); + i = read(fklog, pRcv, 8192); if (i > 0) { /* successful read of message of nonzero length */ pRcv[i] = '\0'; + } else if (i == -EPIPE) { + imkmsgLogIntMsg(LOG_WARNING, + "imkmsg: some messages in circular buffer got overwritten"); + continue; } else { /* something went wrong - error or zero length message */ if (i < 0 && errno != EINTR && errno != EAGAIN) { diff --git a/plugins/imrelp/imrelp.c b/plugins/imrelp/imrelp.c index dc67f4fe..5e0ae552 100644 --- a/plugins/imrelp/imrelp.c +++ b/plugins/imrelp/imrelp.c @@ -208,6 +208,7 @@ addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) if(pRelpEngine == NULL) { CHKiRet(relpEngineConstruct(&pRelpEngine)); CHKiRet(relpEngineSetDbgprint(pRelpEngine, dbgprintf)); + CHKiRet(relpEngineSetFamily(pRelpEngine, glbl.GetDefPFFamily())); CHKiRet(relpEngineSetEnableCmd(pRelpEngine, (uchar*) "syslog", eRelpCmdState_Required)); CHKiRet(relpEngineSetSyslogRcv(pRelpEngine, onSyslogRcv)); if (!glbl.GetDisableDNS()) { diff --git a/plugins/imudp/imudp.c b/plugins/imudp/imudp.c index dde8f105..7e11a80e 100644 --- a/plugins/imudp/imudp.c +++ b/plugins/imudp/imudp.c @@ -853,7 +853,6 @@ CODESTARTactivateCnfPrePrivDrop ABORT_FINALIZE(RS_RET_NO_RUN); } - setSchedParams(pModConf); finalize_it: ENDactivateCnfPrePrivDrop @@ -886,6 +885,15 @@ ENDfreeCnf */ BEGINrunInput CODESTARTrunInput + /* Note well: the setting of scheduling parameters will not work + * when we dropped privileges (if the user is not sufficently + * privileged, of course). Howerver, we can't change the + * scheduling params in PrePrivDrop(), as at that point our thread + * is not yet created. So at least as an interim solution, we do + * NOT support both setting sched parameters and dropping + * privileges within the same instance. + */ + setSchedParams(runModConf); iRet = rcvMainLoop(pThrd); ENDrunInput diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index 66a1c648..f15773fc 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -6,7 +6,7 @@ * * File begun on 2007-12-20 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -144,7 +144,9 @@ typedef struct lstn_s { sbool bAnnotate; /* annotate events with trusted properties */ sbool bParseTrusted; /* parse trusted properties */ sbool bWritePid; /* write original PID into tag */ + sbool bDiscardOwnMsgs; /* discard messages that originated from ourselves */ sbool bUseSysTimeStamp; /* use timestamp from system (instead of from message) */ + sbool bUnlink; /* unlink&re-create socket at start and end of processing */ } lstn_t; static lstn_t listeners[MAXFUNIX]; @@ -199,6 +201,8 @@ struct instanceConf_s { int ratelimitSeverity; int bAnnotate; /* annotate trusted properties */ int bParseTrusted; /* parse trusted properties */ + sbool bDiscardOwnMsgs; /* discard messages that originated from our own pid? */ + sbool bUnlink; struct instanceConf_s *next; }; @@ -216,7 +220,9 @@ struct modConfData_s { sbool bOmitLocalLogging; sbool bWritePidSysSock; sbool bUseSysTimeStamp; + sbool bDiscardOwnMsgs; sbool configSetViaV2Method; + sbool bUnlink; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ @@ -225,10 +231,13 @@ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current lo static struct cnfparamdescr modpdescr[] = { { "syssock.use", eCmdHdlrBinary, 0 }, { "syssock.name", eCmdHdlrGetWord, 0 }, + { "syssock.unlink", eCmdHdlrBinary, 0 }, { "syssock.ignoretimestamp", eCmdHdlrBinary, 0 }, + { "syssock.ignoreownmessages", eCmdHdlrBinary, 0 }, { "syssock.flowcontrol", eCmdHdlrBinary, 0 }, { "syssock.usesystimestamp", eCmdHdlrBinary, 0 }, { "syssock.annotate", eCmdHdlrBinary, 0 }, + { "syssock.parsetrusted", eCmdHdlrBinary, 0 }, { "syssock.usepidfromsystem", eCmdHdlrBinary, 0 }, { "syssock.ratelimit.interval", eCmdHdlrInt, 0 }, { "syssock.ratelimit.burst", eCmdHdlrInt, 0 }, @@ -243,8 +252,10 @@ static struct cnfparamblk modpblk = /* input instance parameters */ static struct cnfparamdescr inppdescr[] = { { "socket", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: addunixlistensocket */ + { "unlink", eCmdHdlrBinary, 0 }, { "createpath", eCmdHdlrBinary, 0 }, { "parsetrusted", eCmdHdlrBinary, 0 }, + { "ignoreownmessages", eCmdHdlrBinary, 0 }, { "hostname", eCmdHdlrString, 0 }, { "ignoretimestamp", eCmdHdlrBinary, 0 }, { "flowcontrol", eCmdHdlrBinary, 0 }, @@ -288,6 +299,8 @@ createInstance(instanceConf_t **pinst) inst->bWritePid = 0; inst->bAnnotate = 0; inst->bParseTrusted = 0; + inst->bDiscardOwnMsgs = 1; + inst->bUnlink = 1; inst->next = NULL; /* node created, let's add to config */ @@ -388,9 +401,11 @@ addListner(instanceConf_t *inst) listeners[nfd].flags = inst->bIgnoreTimestamp ? IGNDATE : NOFLAG; listeners[nfd].bCreatePath = inst->bCreatePath; listeners[nfd].sockName = ustrdup(inst->sockName); - listeners[nfd].bUseCreds = (inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0; + listeners[nfd].bUseCreds = (inst->bDiscardOwnMsgs || inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0; listeners[nfd].bAnnotate = inst->bAnnotate; listeners[nfd].bParseTrusted = inst->bParseTrusted; + listeners[nfd].bDiscardOwnMsgs = inst->bDiscardOwnMsgs; + listeners[nfd].bUnlink = inst->bUnlink; listeners[nfd].bWritePid = inst->bWritePid; listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp; CHKiRet(ratelimitNew(&listeners[nfd].dflt_ratelimiter, "imuxsock", NULL)); @@ -441,7 +456,8 @@ createLogSocket(lstn_t *pLstn) struct sockaddr_un sunx; DEFiRet; - unlink((char*)pLstn->sockName); + if(pLstn->bUnlink) + unlink((char*)pLstn->sockName); memset(&sunx, 0, sizeof(sunx)); sunx.sun_family = AF_UNIX; if(pLstn->bCreatePath) { @@ -732,6 +748,11 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim struct json_object *json = NULL, *jval; DEFiRet; + if(pLstn->bDiscardOwnMsgs && cred != NULL && cred->pid == glblGetOurPid()) { + DBGPRINTF("imuxsock: discarding message from our own pid\n"); + FINALIZE; + } + /* TODO: handle format errors?? */ /* we need to parse the pri first, because we need the severity for * rate-limiting as well. @@ -1037,10 +1058,12 @@ activateListeners() listeners[0].ratelimitInterval = runModConf->ratelimitIntervalSysSock; listeners[0].ratelimitBurst = runModConf->ratelimitBurstSysSock; listeners[0].ratelimitSev = runModConf->ratelimitSeveritySysSock; - listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock) ? 1 : 0; + listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock || runModConf->bDiscardOwnMsgs) ? 1 : 0; listeners[0].bWritePid = runModConf->bWritePidSysSock; listeners[0].bAnnotate = runModConf->bAnnotateSysSock; listeners[0].bParseTrusted = runModConf->bParseTrusted; + listeners[0].bDiscardOwnMsgs = runModConf->bDiscardOwnMsgs; + listeners[0].bUnlink = runModConf->bUnlink; listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp; listeners[0].flags = runModConf->bIgnoreTimestamp ? IGNDATE : NOFLAG; listeners[0].flowCtl = runModConf->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; @@ -1089,6 +1112,8 @@ CODESTARTbeginCnfLoad pModConf->bWritePidSysSock = 0; pModConf->bAnnotateSysSock = 0; pModConf->bParseTrusted = 0; + pModConf->bDiscardOwnMsgs = 1; + pModConf->bUnlink = 1; pModConf->ratelimitIntervalSysSock = DFLT_ratelimitInterval; pModConf->ratelimitBurstSysSock = DFLT_ratelimitBurst; pModConf->ratelimitSeveritySysSock = DFLT_ratelimitSeverity; @@ -1123,12 +1148,18 @@ CODESTARTsetModCnf loadModConf->pLogSockName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(modpblk.descr[i].name, "syssock.ignoretimestamp")) { loadModConf->bIgnoreTimestamp = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.ignoreownmessages")) { + loadModConf->bDiscardOwnMsgs = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.unlink")) { + loadModConf->bUnlink = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.flowcontrol")) { loadModConf->bUseFlowCtl = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.usesystimestamp")) { loadModConf->bUseSysTimeStamp = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.annotate")) { loadModConf->bAnnotateSysSock = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.parsetrusted")) { + loadModConf->bParseTrusted = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.usepidfromsystem")) { loadModConf->bWritePidSysSock = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.ratelimit.interval")) { @@ -1183,6 +1214,10 @@ CODESTARTnewInpInst inst->bCreatePath = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "parsetrusted")) { inst->bParseTrusted = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "ignoreownmessages")) { + inst->bDiscardOwnMsgs = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "unlink")) { + inst->bUnlink = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "hostname")) { inst->pLogHostName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(modpblk.descr[i].name, "ignoretimestamp")) { @@ -1222,6 +1257,9 @@ CODESTARTendCnfLoad loadModConf->bUseFlowCtl = cs.bUseFlowCtlSysSock; loadModConf->bAnnotateSysSock = cs.bAnnotateSysSock; loadModConf->bParseTrusted = cs.bParseTrusted; + loadModConf->ratelimitIntervalSysSock = cs.ratelimitIntervalSysSock; + loadModConf->ratelimitBurstSysSock = cs.ratelimitBurstSysSock; + loadModConf->ratelimitSeveritySysSock = cs.ratelimitSeveritySysSock; } loadModConf = NULL; /* done loading */ @@ -1359,8 +1397,10 @@ CODESTARTafterRun listeners[i].fd < SD_LISTEN_FDS_START + sd_fds) continue; - DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName); - unlink((char*) listeners[i].sockName); + if(listeners[i].bUnlink) { + DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName); + unlink((char*) listeners[i].sockName); + } } discardLogSockets(); @@ -1472,8 +1512,17 @@ CODEmodInit_QueryRegCFSLineHdlr listeners[0].bUseCreds = 0; listeners[0].bAnnotate = 0; listeners[0].bParseTrusted = 0; + listeners[0].bDiscardOwnMsgs = 1; + listeners[0].bUnlink = 1; listeners[0].bCreatePath = 0; listeners[0].bUseSysTimeStamp = 1; + if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, + (void(*)(void*))ratelimitDestruct)) == NULL) { + /* in this case, we simply turn off rate-limiting */ + DBGPRINTF("imuxsock: turning off rate limiting for system socket " + "because we could not create hash table\n"); + listeners[0].ratelimitInterval = 0; + } /* initialize socket names */ for(i = 1 ; i < MAXFUNIX ; ++i) { diff --git a/plugins/mmanon/Makefile.am b/plugins/mmanon/Makefile.am new file mode 100644 index 00000000..98f0da24 --- /dev/null +++ b/plugins/mmanon/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmanon.la + +mmanon_la_SOURCES = mmanon.c +mmanon_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmanon_la_LDFLAGS = -module -avoid-version +mmanon_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmanon/mmanon.c b/plugins/mmanon/mmanon.c new file mode 100644 index 00000000..a1c99d09 --- /dev/null +++ b/plugins/mmanon/mmanon.c @@ -0,0 +1,401 @@ +/* mmanon.c + * anonnymize IP addresses inside the syslog message part + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmanon") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* config variables */ + +/* precomputed table of IPv4 anonymization masks */ +static const uint32_t ipv4masks[33] = { + 0xffffffff, 0xfffffffe, 0xfffffffc, 0xfffffff8, + 0xfffffff0, 0xffffffe0, 0xffffffc0, 0xffffff80, + 0xffffff00, 0xfffffe00, 0xfffffc00, 0xfffff800, + 0xfffff000, 0xffffe000, 0xffffc000, 0xffff8000, + 0xffff0000, 0xfffe0000, 0xfffc0000, 0xfff80000, + 0xfff00000, 0xffe00000, 0xffc00000, 0xff800000, + 0xff000000, 0xfe000000, 0xfc000000, 0xf8000000, + 0xf0000000, 0xe0000000, 0xc0000000, 0x80000000, + 0x00000000 + }; + +/* define operation modes we have */ +#define SIMPLE_MODE 0 /* just overwrite */ +#define REWRITE_MODE 1 /* rewrite IP address, canoninized */ +typedef struct _instanceData { + char replChar; + int8_t mode; + struct { + int8_t bits; + } ipv4; +} instanceData; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "mode", eCmdHdlrGetWord, 0 }, + { "replacementchar", eCmdHdlrGetChar, 0 }, + { "ipv4.bits", eCmdHdlrInt, 0 }, +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance +ENDfreeInstance + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->mode = REWRITE_MODE; + pData->replChar = 'x'; + pData->ipv4.bits = 16; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; + sbool bHadBitsErr; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmanon)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "mode")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"simple", + sizeof("simple")-1)) { + pData->mode = SIMPLE_MODE; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"rewrite", + sizeof("rewrite")-1)) { + pData->mode = REWRITE_MODE; + } else { + char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_INVLD_MODE, + "mmanon: invalid anonymization mode '%s' - ignored", + cstr); + free(cstr); + } + pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0]; + } else if(!strcmp(actpblk.descr[i].name, "replacementchar")) { + pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0]; + } else if(!strcmp(actpblk.descr[i].name, "ipv4.bits")) { + pData->ipv4.bits = (int8_t) pvals[i].val.d.n; + } else { + dbgprintf("mmanon: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if(pData->mode == SIMPLE_MODE) { + bHadBitsErr = 0; + if(pData->ipv4.bits < 8) { + pData->ipv4.bits = 8; + bHadBitsErr = 1; + } else if(pData->ipv4.bits < 16) { + pData->ipv4.bits = 16; + bHadBitsErr = 1; + } else if(pData->ipv4.bits < 24) { + pData->ipv4.bits = 24; + bHadBitsErr = 1; + } else if(pData->ipv4.bits != 32) { + pData->ipv4.bits = 32; + bHadBitsErr = 1; + } + if(bHadBitsErr) + errmsg.LogError(0, RS_RET_INVLD_ANON_BITS, + "mmanon: invalid number of ipv4 bits " + "in simple mode, corrected to %d", + pData->ipv4.bits); + } else { /* REWRITE_MODE */ + if(pData->ipv4.bits < 1 || pData->ipv4.bits > 32) { + pData->ipv4.bits = 32; + errmsg.LogError(0, RS_RET_INVLD_ANON_BITS, + "mmanon: invalid number of ipv4 bits " + "in rewrite mode, corrected to %d", + pData->ipv4.bits); + } + if(pData->replChar != 'x') { + errmsg.LogError(0, RS_RET_REPLCHAR_IGNORED, + "mmanon: replacementChar parameter is ignored " + "in rewrite mode"); + } + } + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +static int +getnum(uchar *msg, int lenMsg, int *idx) +{ + int num = 0; + int i = *idx; + + while(i < lenMsg && msg[i] >= '0' && msg[i] <= '9') { + num = num * 10 + msg[i] - '0'; + ++i; + } + + *idx = i; + return num; +} + + +/* write an IP address octet to the output position */ +static int +writeOctet(uchar *msg, int idx, int *nxtidx, uint8_t octet) +{ + if(octet > 99) { + msg[idx++] = '0' + octet / 100; + octet = octet % 100; + } + if(octet > 9) { + msg[idx++] = '0' + octet / 10; + octet = octet % 10; + } + msg[idx++] = '0' + octet; + + if(nxtidx != NULL) { + if(idx + 1 != *nxtidx) { + /* we got shorter, fix it! */ + msg[idx] = '.'; + *nxtidx = idx + 1; + } + } + return idx; +} + +/* currently works for IPv4 only! */ +void +anonip(instanceData *pData, uchar *msg, int *pLenMsg, int *idx) +{ + int i = *idx; + int octet; + uint32_t ipv4addr; + int ipstart[4]; + int j; + int endpos; + int lenMsg = *pLenMsg; + + while(i < lenMsg && (msg[i] <= '0' || msg[i] >= '9')) { + ++i; /* skip to first number */ + } + if(i >= lenMsg) + goto done; + + /* got digit, let's see if ip */ + ipstart[0] = i; + octet = getnum(msg, lenMsg, &i); + if(octet > 255 || msg[i] != '.') goto done; + ipv4addr = octet << 24; + ++i; + ipstart[1] = i; + octet = getnum(msg, lenMsg, &i); + if(octet > 255 || msg[i] != '.') goto done; + ipv4addr |= octet << 16; + ++i; + ipstart[2] = i; + octet = getnum(msg, lenMsg, &i); + if(octet > 255 || msg[i] != '.') goto done; + ipv4addr |= octet << 8; + ++i; + ipstart[3] = i; + octet = getnum(msg, lenMsg, &i); + if(octet > 255 || !(msg[i] == ' ' || msg[i] == ':')) goto done; + ipv4addr |= octet; + + /* OK, we now found an ip address */ + if(pData->mode == SIMPLE_MODE) { + if(pData->ipv4.bits == 8) + j = ipstart[3]; + else if(pData->ipv4.bits == 16) + j = ipstart[2]; + else if(pData->ipv4.bits == 24) + j = ipstart[1]; + else /* due to our checks, this *must* be 32 */ + j = ipstart[0]; + while(j < i) { + if(msg[j] != '.') + msg[j] = pData->replChar; + ++j; + } + } else { /* REWRITE_MODE */ + ipv4addr &= ipv4masks[pData->ipv4.bits]; + if(pData->ipv4.bits > 24) + writeOctet(msg, ipstart[0], &(ipstart[1]), ipv4addr >> 24); + if(pData->ipv4.bits > 16) + writeOctet(msg, ipstart[1], &(ipstart[2]), (ipv4addr >> 16) & 0xff); + if(pData->ipv4.bits > 8) + writeOctet(msg, ipstart[2], &(ipstart[3]), (ipv4addr >> 8) & 0xff); + endpos = writeOctet(msg, ipstart[3], NULL, ipv4addr & 0xff); + /* if we had truncation, we need to shrink the msg */ + dbgprintf("existing i %d, endpos %d\n", i, endpos); + if(i - endpos > 0) { + *pLenMsg = lenMsg - (i - endpos); + memmove(msg+endpos, msg+i, lenMsg - i + 1); + } + } + +done: *idx = i; + return; +} + + +BEGINdoAction + msg_t *pMsg; + uchar *msg; + int lenMsg; + int i; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + lenMsg = getMSGLen(pMsg); + msg = getMSG(pMsg); + for(i = 0 ; i < lenMsg ; ++i) { + anonip(pData, msg, &lenMsg, &i); + } + if(lenMsg != getMSGLen(pMsg)) + setMSGLen(pMsg, lenMsg); +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmanon:", sizeof(":mmanon:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmanon supports only v6+ config format, use: " + "action(type=\"mmanon\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmanon: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/mmnormalize/mmnormalize.c b/plugins/mmnormalize/mmnormalize.c index fd2004a3..fcadc328 100644 --- a/plugins/mmnormalize/mmnormalize.c +++ b/plugins/mmnormalize/mmnormalize.c @@ -397,7 +397,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizerulebase", 0, eCmdHdlrGetWord, setRuleBase, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizeuserawmsg", 0, eCmdHdlrInt, + CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizeuserawmsg", 0, eCmdHdlrBinary, NULL, &cs.bUseRawMsg, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); diff --git a/plugins/omjournal/Makefile.am b/plugins/omjournal/Makefile.am new file mode 100644 index 00000000..4cfbbd96 --- /dev/null +++ b/plugins/omjournal/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = omjournal.la + +omjournal_la_SOURCES = omjournal.c +omjournal_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBSYSTEMD_JOURNAL_CFLAGS) +omjournal_la_LDFLAGS = -module -avoid-version +omjournal_la_LIBADD = $(LIBSYSTEMD_JOURNAL_LIBS) + +EXTRA_DIST = diff --git a/plugins/omjournal/omjournal.c b/plugins/omjournal/omjournal.c new file mode 100644 index 00000000..c340287f --- /dev/null +++ b/plugins/omjournal/omjournal.c @@ -0,0 +1,185 @@ +/* omjournal.c + * send messages to the Linux Journal. This is meant to be used + * in cases where journal serves as the whole system log database. + * Note that we may get into a loop if journald re-injects messages + * into the syslog stream and we read that via imuxsock. Thus there + * is an option in imuxsock to ignore messages from ourselves + * (actually from our pid). So there are some module-interdependencies. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include <systemd/sd-journal.h> + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("omjournal") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* config variables */ + + +typedef struct _instanceData { +} instanceData; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance +ENDfreeInstance + + +BEGINnewActInst +CODESTARTnewActInst + /* Note: we currently do not have any parameters, so we do not need + * the lst ptr. However, we will most probably need params in the + * future. + */ + DBGPRINTF("newActInst (mmjournal)\n"); + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + /*setInstParamDefaults(pData);*/ +CODE_STD_FINALIZERnewActInst +/* cnfparamvalsDestruct(pvals, &actpblk);*/ +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + +BEGINdoAction + msg_t *pMsg; + uchar *tag; + int lenTag; + int sev; + int r; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + MsgGetSeverity(pMsg, &sev); + getTAG(pMsg, &tag, &lenTag); + /* we can use more properties here, but let's see if there + * is some real user interest. We can always add later... + */ + r = sd_journal_send("MESSAGE=%s", getMSG(pMsg), + "PRIORITY=%d", sev, + "SYSLOG_FACILITY=%d", pMsg->iFacility, + "SYSLOG_IDENTIFIER=%s", tag, + NULL); + /* FIXME: think about what to do with errors ;) */ +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":omjournal:", sizeof(":omjournal:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "omjournal supports only v6+ config format, use: " + "action(type=\"omjournal\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("omjournal: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/omlibdbi/omlibdbi.c b/plugins/omlibdbi/omlibdbi.c index 390e59d5..6e27ad22 100644 --- a/plugins/omlibdbi/omlibdbi.c +++ b/plugins/omlibdbi/omlibdbi.c @@ -70,6 +70,7 @@ typedef struct _instanceData { uchar *dbName; /* database to use */ unsigned uLastDBErrno; /* last errno returned by libdbi or 0 if all is well */ uchar *tplName; /* format template to use */ + int txSupport; /* transaction support */ } instanceData; typedef struct configSettings_s { @@ -261,7 +262,7 @@ static rsRetVal initConn(instanceData *pData, int bSilent) # endif if(pData->conn == NULL) { errmsg.LogError(0, RS_RET_SUSPENDED, "can not initialize libdbi connection"); - iRet = RS_RET_SUSPENDED; + ABORT_FINALIZE(RS_RET_SUSPENDED); } else { /* we could get the handle, now on with work... */ /* Connect to database */ dbi_conn_set_option(pData->conn, "host", (char*) pData->host); @@ -272,8 +273,9 @@ static rsRetVal initConn(instanceData *pData, int bSilent) if(dbi_conn_connect(pData->conn) < 0) { reportDBError(pData, bSilent); closeConn(pData); /* ignore any error we may get */ - iRet = RS_RET_SUSPENDED; + ABORT_FINALIZE(RS_RET_SUSPENDED); } + pData->txSupport = dbi_conn_cap_get(pData->conn, "transaction_support"); } finalize_it: @@ -329,12 +331,46 @@ CODESTARTtryResume } ENDtryResume +/* transaction support 2013-03 */ +BEGINbeginTransaction +CODESTARTbeginTransaction + if(pData->conn == NULL) { + CHKiRet(initConn(pData, 0)); + } +# if HAVE_DBI_TXSUPP + if (pData->txSupport == 1) { + if (dbi_conn_transaction_begin(pData->conn) != 0) { + dbgprintf("libdbi server error: begin transaction not successful\n"); + iRet = RS_RET_SUSPENDED; + } + } +# endif +finalize_it: +ENDbeginTransaction +/* end transaction */ + BEGINdoAction CODESTARTdoAction - dbgprintf("\n"); - iRet = writeDB(ppString[0], pData); + CHKiRet(writeDB(ppString[0], pData)); +# if HAVE_DBI_TXSUPP + if (pData->txSupport == 1) { + iRet = RS_RET_DEFER_COMMIT; + } +# endif +finalize_it: ENDdoAction +/* transaction support 2013-03 */ +BEGINendTransaction +CODESTARTendTransaction +# if HAVE_DBI_TXSUPP + if (dbi_conn_transaction_commit(pData->conn) != 0) { + dbgprintf("libdbi server error: transaction not committed\n"); + iRet = RS_RET_SUSPENDED; + } +# endif +ENDendTransaction +/* end transaction */ BEGINbeginCnfLoad CODESTARTbeginCnfLoad @@ -427,7 +463,6 @@ CODESTARTnewActInst CHKiRet(createInstance(&pData)); setInstParamDefaults(pData); - CODE_STD_STRING_REQUESTnewActInst(1) for(i = 0 ; i < actpblk.nParams ; ++i) { if(!pvals[i].bUsed) @@ -468,7 +503,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* ok, if we reach this point, we have something for us */ CHKiRet(createInstance(&pData)); - /* no create the instance based on what we currently have */ if(cs.drvrName == NULL) { errmsg.LogError(0, RS_RET_NO_DRIVERNAME, "omlibdbi: no db driver name given - action can not be created"); @@ -513,6 +547,7 @@ CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt @@ -542,6 +577,10 @@ CODESTARTmodInit INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr +# ifndef HAVE_DBI_TXSUPP + DBGPRINTF("omlibdbi: no transaction support in libdbi\n"); +# warning libdbi too old - transactions are not enabled (use 0.9 or later) +# endif CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(regCfSysLineHdlr2((uchar *)"actionlibdbidriverdirectory", 0, eCmdHdlrGetWord, NULL, &cs.dbiDrvrDir, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidriver", 0, eCmdHdlrGetWord, NULL, &cs.drvrName, STD_LOADABLE_MODULE_ID)); diff --git a/plugins/omstdout/omstdout.c b/plugins/omstdout/omstdout.c index 59f9c8bb..a84a7593 100644 --- a/plugins/omstdout/omstdout.c +++ b/plugins/omstdout/omstdout.c @@ -105,6 +105,7 @@ BEGINdoAction int iBuf; char szBuf[65564]; size_t len; + int r; CODESTARTdoAction if(pData->bUseArrayInterface) { /* if we use array passing, we need to put together a string @@ -140,9 +141,15 @@ CODESTARTdoAction * actually intends to use this module in production (why???), this code * needs to be more solid. -- rgerhards, 2012-11-28 */ - if(write(1, toWrite, len)) {}; /* 1 is stdout! */ + if((r = write(1, toWrite, len)) != (int) len) { /* 1 is stdout! */ + DBGPRINTF("omstdout: error %d writing to stdout[%d]: %s\n", + r, len, toWrite); + } if(pData->bEnsureLFEnding && toWrite[len-1] != '\n') { - if(write(1, "\n", 1)) {}; /* write missing LF */ + if((r = write(1, "\n", 1)) != 1) { /* write missing LF */ + DBGPRINTF("omstdout: error %d writing \\n to stdout\n", + r); + } } ENDdoAction diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c index 6b06d427..a437b7f8 100644 --- a/runtime/cfsysline.c +++ b/runtime/cfsysline.c @@ -350,8 +350,9 @@ static rsRetVal doGetGID(uchar **pp, rsRetVal (*pSetHdlr)(void*, uid_t), void *p struct group gBuf; DEFiRet; uchar szName[256]; - int bufSize = 2048; + int bufSize = 1024; char * stringBuf = NULL; + int err; assert(pp != NULL); assert(*pp != NULL); @@ -361,20 +362,21 @@ static rsRetVal doGetGID(uchar **pp, rsRetVal (*pSetHdlr)(void*, uid_t), void *p ABORT_FINALIZE(RS_RET_NOT_FOUND); } - - CHKmalloc(stringBuf = malloc(bufSize)); - while(pgBuf == NULL) { - errno = 0; - getgrnam_r((char*)szName, &gBuf, stringBuf, bufSize, &pgBuf); - if((pgBuf == NULL) && (errno == ERANGE)) { - /* Increase bufsize and try again.*/ - bufSize *= 2; - CHKmalloc(stringBuf = realloc(stringBuf, bufSize)); - } - } + do { + /* Increase bufsize and try again.*/ + bufSize *= 2; + CHKmalloc(stringBuf = realloc(stringBuf, bufSize)); + err = getgrnam_r((char*)szName, &gBuf, stringBuf, bufSize, &pgBuf); + } while((pgBuf == NULL) && (err == ERANGE)); if(pgBuf == NULL) { - errmsg.LogError(0, RS_RET_NOT_FOUND, "ID for group '%s' could not be found or error", (char*)szName); + if (err != 0) { + rs_strerror_r(err, stringBuf, bufSize); + errmsg.LogError(0, RS_RET_NOT_FOUND, "Query for group '%s' resulted in an error: %s\n", + (char*)szName, stringBuf); + } else { + errmsg.LogError(0, RS_RET_NOT_FOUND, "ID for group '%s' could not be found", (char*)szName); + } iRet = RS_RET_NOT_FOUND; } else { if(pSetHdlr == NULL) { diff --git a/runtime/debug.c b/runtime/debug.c index fa39e7fe..876f61d0 100644 --- a/runtime/debug.c +++ b/runtime/debug.c @@ -303,7 +303,7 @@ static inline void dbgFuncDBRemoveMutexLock(dbgFuncDB_t *pFuncDB, pthread_mutex_ void dbgOutputTID(char* name) { -# ifdef HAVE_SYSCALL +# if defined(HAVE_SYSCALL) && defined(HAVE_SYS_gettid) if(bOutputTidToStderr) fprintf(stderr, "thread tid %u, name '%s'\n", (unsigned)syscall(SYS_gettid), name); @@ -1315,6 +1315,15 @@ dbgmalloc(size_t size) } +/* report fd used for debug log. This is needed in case of + * auto-backgrounding, where the debug log shall not be closed. + */ +int +dbgGetDbglogFd(void) +{ + return altdbg; +} + /* read in the runtime options * rgerhards, 2008-02-28 */ @@ -1398,6 +1407,25 @@ dbgGetRuntimeOptions(void) } +void +dbgSetDebugLevel(int level) +{ + Debug = level; + debugging_on = (level == DEBUG_FULL) ? 1 : 0; +} + +void +dbgSetDebugFile(uchar *fn) +{ + if(altdbg != -1) { + dbgprintf("switching to debug file %s\n", fn); + close(altdbg); + } + if((altdbg = open((char*)fn, O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, S_IRUSR|S_IWUSR)) == -1) { + fprintf(stderr, "alternate debug file could not be opened, ignoring. Error: %s\n", strerror(errno)); + } +} + /* end support system to set debug options at runtime */ rsRetVal dbgClassInit(void) diff --git a/runtime/debug.h b/runtime/debug.h index f802e8c1..f3226098 100644 --- a/runtime/debug.h +++ b/runtime/debug.h @@ -89,6 +89,8 @@ typedef struct dbgCallStack_s { /* prototypes */ rsRetVal dbgClassInit(void); rsRetVal dbgClassExit(void); +void dbgSetDebugFile(uchar *fn); +void dbgSetDebugLevel(int level); void sigsegvHdlr(int signum); void dbgoprint(obj_t *pObj, char *fmt, ...) __attribute__((format(printf, 2, 3))); void dbgprintf(char *fmt, ...) __attribute__((format(printf, 1, 2))); @@ -105,6 +107,7 @@ void dbgSetThrdName(uchar *pszName); void dbgPrintAllDebugInfo(void); void *dbgmalloc(size_t size); void dbgOutputTID(char* name); +int dbgGetDbglogFd(void); /* macros */ #ifdef DEBUGLESS diff --git a/runtime/glbl.c b/runtime/glbl.c index 0e5cac20..b3fe3a1d 100644 --- a/runtime/glbl.c +++ b/runtime/glbl.c @@ -7,7 +7,7 @@ * * Module begun 2008-04-16 by Rainer Gerhards * - * Copyright 2008-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -82,6 +82,7 @@ static uchar *pszDfltNetstrmDrvrCAF = NULL; /* default CA file for the netstrm d static uchar *pszDfltNetstrmDrvrKeyFile = NULL; /* default key file for the netstrm driver (server) */ static uchar *pszDfltNetstrmDrvrCertFile = NULL; /* default cert file for the netstrm driver (server) */ static int bTerminateInputs = 0; /* global switch that inputs shall terminate ASAP (1=> terminate) */ +pid_t glbl_ourpid; #ifndef HAVE_ATOMIC_BUILTINS static DEF_ATOMIC_HELPER_MUT(mutTerminateInputs); #endif @@ -278,6 +279,28 @@ finalize_it: RETiRet; } + +static rsRetVal +setDebugFile(void __attribute__((unused)) *pVal, uchar *pNewVal) +{ + DEFiRet; + dbgSetDebugFile(pNewVal); + free(pNewVal); + RETiRet; +} + + +static rsRetVal +setDebugLevel(void __attribute__((unused)) *pVal, int level) +{ + DEFiRet; + dbgSetDebugLevel(level); + dbgprintf("debug level %d set via config file\n", level); + dbgprintf("This is rsyslog version " VERSION "\n"); + RETiRet; +} + + /* return our local IP. * If no local IP is set, "127.0.0.1" is selected *and* set. This * is an intensional side effect that we do in order to keep things @@ -610,6 +633,8 @@ BEGINAbstractObjClassInit(glbl, 1, OBJ_IS_CORE_MODULE) /* class, version */ CHKiRet(objUse(errmsg, CORE_COMPONENT)); /* config handlers are never unregistered and need not be - we are always loaded ;) */ + CHKiRet(regCfSysLineHdlr((uchar *)"debugfile", 0, eCmdHdlrGetWord, setDebugFile, NULL, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"debuglevel", 0, eCmdHdlrInt, setDebugLevel, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"workdirectory", 0, eCmdHdlrGetWord, setWorkDir, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"dropmsgswithmaliciousdnsptrrecords", 0, eCmdHdlrBinary, NULL, &bDropMalPTRMsgs, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"defaultnetstreamdriver", 0, eCmdHdlrGetWord, NULL, &pszDfltNetstrmDrvr, NULL)); diff --git a/runtime/glbl.h b/runtime/glbl.h index d2d1e66a..e95e48f7 100644 --- a/runtime/glbl.h +++ b/runtime/glbl.h @@ -30,11 +30,14 @@ #ifndef GLBL_H_INCLUDED #define GLBL_H_INCLUDED +#include <sys/types.h> #include "rainerscript.h" #include "prop.h" #define glblGetIOBufSize() 4096 /* size of the IO buffer, e.g. for strm class */ +extern pid_t glbl_ourpid; + /* interfaces */ BEGINinterface(glbl) /* name must also be changed in ENDinterface macro! */ uchar* (*GetWorkDir)(void); @@ -86,6 +89,9 @@ ENDinterface(glbl) /* the remaining prototypes */ PROTOTYPEObj(glbl); +static inline pid_t glblGetOurPid(void) { return glbl_ourpid; } +static inline void glblSetOurPid(pid_t pid) { glbl_ourpid = pid; } + void glblPrepCnf(void); void glblProcessCnf(struct cnfobj *o); void glblDoneLoadCnf(void); diff --git a/runtime/msg.c b/runtime/msg.c index 68577ad0..c302a050 100644 --- a/runtime/msg.c +++ b/runtime/msg.c @@ -1468,6 +1468,14 @@ getRawMsg(msg_t *pM, uchar **pBuf, int *piLen) } +/* note: setMSGLen() is only for friends who really know what they + * do. Setting an invalid length can be desasterous! + */ +void setMSGLen(msg_t *pM, int lenMsg) +{ + pM->iLenMSG = lenMsg; +} + int getMSGLen(msg_t *pM) { return((pM == NULL) ? 0 : pM->iLenMSG); diff --git a/runtime/msg.h b/runtime/msg.h index 564441b6..edf5ed98 100644 --- a/runtime/msg.h +++ b/runtime/msg.h @@ -198,6 +198,7 @@ uchar *getMSG(msg_t *pM); char *getHOSTNAME(msg_t *pM); char *getPROCID(msg_t *pM, sbool bLockMutex); char *getAPPNAME(msg_t *pM, sbool bLockMutex); +void setMSGLen(msg_t *pM, int lenMsg); int getMSGLen(msg_t *pM); char *getHOSTNAME(msg_t *pM); diff --git a/runtime/queue.c b/runtime/queue.c index 6df1c95e..4c8d3ac5 100644 --- a/runtime/queue.c +++ b/runtime/queue.c @@ -1356,7 +1356,7 @@ qqueueSetDefaultsActionQueue(qqueue_t *pThis) pThis->iDeqBatchSize = 128; /* default batch size */ pThis->iHighWtrMrk = 800; /* high water mark for disk-assisted queues */ pThis->iLowWtrMrk = 200; /* low water mark for disk-assisted queues */ - pThis->iDiscardMrk = 9800; /* begin to discard messages */ + pThis->iDiscardMrk = 980; /* begin to discard messages */ pThis->iDiscardSeverity = 8; /* turn off */ pThis->iNumWorkerThreads = 1; /* number of worker threads for the mm queue above */ pThis->iMaxFileSize = 1024*1024; diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index 42c61579..5ba6ede7 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -399,6 +399,9 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_DS_PROP_SEQ_ERR = -2308,/**< property sequence error deserializing object */ RS_RET_TPL_INVLD_PROP = -2309,/**< property name error in template (unknown name) */ RS_RET_NO_RULEBASE = -2310,/**< mmnormalize: rulebase can not be found or otherwise invalid */ + RS_RET_INVLD_MODE = -2311,/**< invalid mode specified in configuration */ + RS_RET_INVLD_ANON_BITS = -2312,/**< mmanon: invalid number of bits to anonymize specified */ + RS_RET_REPLCHAR_IGNORED = -2313,/**< mmanon: replacementChar parameter is ignored */ RS_RET_SIGPROV_ERR = -2320,/**< error in signature provider */ /* RainerScript error messages (range 1000.. 1999) */ diff --git a/runtime/sd-daemon.c b/runtime/sd-daemon.c index 9c23b917..79d8ca37 100644 --- a/runtime/sd-daemon.c +++ b/runtime/sd-daemon.c @@ -25,14 +25,18 @@ ***/ #ifndef _GNU_SOURCE -#define _GNU_SOURCE +# define _GNU_SOURCE #endif #include <sys/types.h> #include <sys/stat.h> #include <sys/socket.h> #include <sys/un.h> -#include <sys/fcntl.h> +#ifdef __BIONIC__ +# include <linux/fcntl.h> +#else +# include <sys/fcntl.h> +#endif #include <netinet/in.h> #include <stdlib.h> #include <errno.h> @@ -40,10 +44,28 @@ #include <string.h> #include <stdarg.h> #include <stdio.h> +#include <stddef.h> +#include <limits.h> + +#if defined(__linux__) +# include <mqueue.h> +#endif #include "sd-daemon.h" -int sd_listen_fds(int unset_environment) { +#if (__GNUC__ >= 4) +# ifdef SD_EXPORT_SYMBOLS +/* Export symbols */ +# define _sd_export_ __attribute__ ((visibility("default"))) +# else +/* Don't export the symbols */ +# define _sd_export_ __attribute__ ((visibility("hidden"))) +# endif +#else +# define _sd_export_ +#endif + +_sd_export_ int sd_listen_fds(int unset_environment) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) return 0; @@ -53,7 +75,8 @@ int sd_listen_fds(int unset_environment) { char *p = NULL; unsigned long l; - if (!(e = getenv("LISTEN_PID"))) { + e = getenv("LISTEN_PID"); + if (!e) { r = 0; goto finish; } @@ -66,7 +89,7 @@ int sd_listen_fds(int unset_environment) { goto finish; } - if (!p || *p || l <= 0) { + if (!p || p == e || *p || l <= 0) { r = -EINVAL; goto finish; } @@ -77,7 +100,8 @@ int sd_listen_fds(int unset_environment) { goto finish; } - if (!(e = getenv("LISTEN_FDS"))) { + e = getenv("LISTEN_FDS"); + if (!e) { r = 0; goto finish; } @@ -90,7 +114,7 @@ int sd_listen_fds(int unset_environment) { goto finish; } - if (!p || *p) { + if (!p || p == e || *p) { r = -EINVAL; goto finish; } @@ -98,7 +122,8 @@ int sd_listen_fds(int unset_environment) { for (fd = SD_LISTEN_FDS_START; fd < SD_LISTEN_FDS_START + (int) l; fd ++) { int flags; - if ((flags = fcntl(fd, F_GETFD)) < 0) { + flags = fcntl(fd, F_GETFD); + if (flags < 0) { r = -errno; goto finish; } @@ -124,13 +149,12 @@ finish: #endif } -int sd_is_fifo(int fd, const char *path) { +_sd_export_ int sd_is_fifo(int fd, const char *path) { struct stat st_fd; if (fd < 0) return -EINVAL; - memset(&st_fd, 0, sizeof(st_fd)); if (fstat(fd, &st_fd) < 0) return -errno; @@ -140,7 +164,6 @@ int sd_is_fifo(int fd, const char *path) { if (path) { struct stat st_path; - memset(&st_path, 0, sizeof(st_path)); if (stat(path, &st_path) < 0) { if (errno == ENOENT || errno == ENOTDIR) @@ -157,6 +180,42 @@ int sd_is_fifo(int fd, const char *path) { return 1; } +_sd_export_ int sd_is_special(int fd, const char *path) { + struct stat st_fd; + + if (fd < 0) + return -EINVAL; + + if (fstat(fd, &st_fd) < 0) + return -errno; + + if (!S_ISREG(st_fd.st_mode) && !S_ISCHR(st_fd.st_mode)) + return 0; + + if (path) { + struct stat st_path; + + if (stat(path, &st_path) < 0) { + + if (errno == ENOENT || errno == ENOTDIR) + return 0; + + return -errno; + } + + if (S_ISREG(st_fd.st_mode) && S_ISREG(st_path.st_mode)) + return + st_path.st_dev == st_fd.st_dev && + st_path.st_ino == st_fd.st_ino; + else if (S_ISCHR(st_fd.st_mode) && S_ISCHR(st_path.st_mode)) + return st_path.st_rdev == st_fd.st_rdev; + else + return 0; + } + + return 1; +} + static int sd_is_socket_internal(int fd, int type, int listening) { struct stat st_fd; @@ -208,13 +267,14 @@ union sockaddr_union { struct sockaddr_storage storage; }; -int sd_is_socket(int fd, int family, int type, int listening) { +_sd_export_ int sd_is_socket(int fd, int family, int type, int listening) { int r; if (family < 0) return -EINVAL; - if ((r = sd_is_socket_internal(fd, type, listening)) <= 0) + r = sd_is_socket_internal(fd, type, listening); + if (r <= 0) return r; if (family > 0) { @@ -236,7 +296,7 @@ int sd_is_socket(int fd, int family, int type, int listening) { return 1; } -int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port) { +_sd_export_ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port) { union sockaddr_union sockaddr; socklen_t l; int r; @@ -244,7 +304,8 @@ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port if (family != 0 && family != AF_INET && family != AF_INET6) return -EINVAL; - if ((r = sd_is_socket_internal(fd, type, listening)) <= 0) + r = sd_is_socket_internal(fd, type, listening); + if (r <= 0) return r; memset(&sockaddr, 0, sizeof(sockaddr)); @@ -281,12 +342,13 @@ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port return 1; } -int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t length) { +_sd_export_ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t length) { union sockaddr_union sockaddr; socklen_t l; int r; - if ((r = sd_is_socket_internal(fd, type, listening)) <= 0) + r = sd_is_socket_internal(fd, type, listening); + if (r <= 0) return r; memset(&sockaddr, 0, sizeof(sockaddr)); @@ -302,29 +364,66 @@ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t return 0; if (path) { - if (length <= 0) + if (length == 0) length = strlen(path); - if (length <= 0) + if (length == 0) /* Unnamed socket */ - return l == sizeof(sa_family_t); + return l == offsetof(struct sockaddr_un, sun_path); if (path[0]) /* Normal path socket */ return - (l >= sizeof(sa_family_t) + length + 1) && + (l >= offsetof(struct sockaddr_un, sun_path) + length + 1) && memcmp(path, sockaddr.un.sun_path, length+1) == 0; else /* Abstract namespace socket */ return - (l == sizeof(sa_family_t) + length) && + (l == offsetof(struct sockaddr_un, sun_path) + length) && memcmp(path, sockaddr.un.sun_path, length) == 0; } return 1; } -int sd_notify(int unset_environment, const char *state) { +_sd_export_ int sd_is_mq(int fd, const char *path) { +#if !defined(__linux__) + return 0; +#else + struct mq_attr attr; + + if (fd < 0) + return -EINVAL; + + if (mq_getattr(fd, &attr) < 0) + return -errno; + + if (path) { + char fpath[PATH_MAX]; + struct stat a, b; + + if (path[0] != '/') + return -EINVAL; + + if (fstat(fd, &a) < 0) + return -errno; + + strncpy(stpcpy(fpath, "/dev/mqueue"), path, sizeof(fpath) - 12); + fpath[sizeof(fpath)-1] = 0; + + if (stat(fpath, &b) < 0) + return -errno; + + if (a.st_dev != b.st_dev || + a.st_ino != b.st_ino) + return 0; + } + + return 1; +#endif +} + +_sd_export_ int sd_notify(int unset_environment, const char *state) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) || !defined(SOCK_CLOEXEC) return 0; #else @@ -339,7 +438,8 @@ int sd_notify(int unset_environment, const char *state) { goto finish; } - if (!(e = getenv("NOTIFY_SOCKET"))) + e = getenv("NOTIFY_SOCKET"); + if (!e) return 0; /* Must be an abstract socket, or an absolute path */ @@ -348,7 +448,8 @@ int sd_notify(int unset_environment, const char *state) { goto finish; } - if ((fd = socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0)) < 0) { + fd = socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0); + if (fd < 0) { r = -errno; goto finish; } @@ -366,7 +467,7 @@ int sd_notify(int unset_environment, const char *state) { memset(&msghdr, 0, sizeof(msghdr)); msghdr.msg_name = &sockaddr; - msghdr.msg_namelen = sizeof(sa_family_t) + strlen(e); + msghdr.msg_namelen = offsetof(struct sockaddr_un, sun_path) + strlen(e); if (msghdr.msg_namelen > sizeof(struct sockaddr_un)) msghdr.msg_namelen = sizeof(struct sockaddr_un); @@ -392,7 +493,7 @@ finish: #endif } -int sd_notifyf(int unset_environment, const char *format, ...) { +_sd_export_ int sd_notifyf(int unset_environment, const char *format, ...) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) return 0; #else @@ -414,22 +515,19 @@ int sd_notifyf(int unset_environment, const char *format, ...) { #endif } -int sd_booted(void) { +_sd_export_ int sd_booted(void) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) return 0; #else + struct stat st; - struct stat a, b; - - /* We simply test whether the systemd cgroup hierarchy is - * mounted */ - - if (lstat("/sys/fs/cgroup", &a) < 0) - return 0; + /* We test whether the runtime unit file directory has been + * created. This takes place in mount-setup.c, so is + * guaranteed to happen very early during boot. */ - if (lstat("/sys/fs/cgroup/systemd", &b) < 0) + if (lstat("/run/systemd/system/", &st) < 0) return 0; - return a.st_dev != b.st_dev; + return !!S_ISDIR(st.st_mode); #endif } diff --git a/runtime/srUtils.h b/runtime/srUtils.h index 3169fd94..8626a4bb 100644 --- a/runtime/srUtils.h +++ b/runtime/srUtils.h @@ -91,6 +91,7 @@ char *rs_strerror_r(int errnum, char *buf, size_t buflen); int decodeSyslogName(uchar *name, syslogName_t *codetab); int getSubString(uchar **ppSrc, char *pDst, size_t DstSize, char cSep); rsRetVal getFileSize(uchar *pszName, off_t *pSize); +int containsGlobWildcard(char *str); /* mutex operations */ /* some useful constants */ diff --git a/runtime/srutils.c b/runtime/srutils.c index 7b485b23..6a509b4a 100644 --- a/runtime/srutils.c +++ b/runtime/srutils.c @@ -630,6 +630,28 @@ finalize_it: RETiRet; } +/* Returns 1 if the given string contains a non-escaped glob(3) + * wildcard character and 0 otherwise (or if the string is empty). + */ +int +containsGlobWildcard(char *str) +{ + char *p; + if(!str) { + return 0; + } + /* From Linux Programmer's Guide: + * "A string is a wildcard pattern if it contains one of the characters '?', '*' or '['" + * "One can remove the special meaning of '?', '*' and '[' by preceding them by a backslash" + */ + for(p = str; *p != '\0'; p++) { + if((*p == '?' || *p == '*' || *p == '[') && + (p == str || *(p-1) != '\\')) { + return 1; + } + } + return 0; +} /* vim:set ai: */ diff --git a/runtime/stream.c b/runtime/stream.c index 3e890c71..00afcdaa 100644 --- a/runtime/stream.c +++ b/runtime/stream.c @@ -81,6 +81,7 @@ static void *asyncWriterThread(void *pPtr); static rsRetVal doZipWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf, int bFlush); static rsRetVal doZipFinish(strm_t *pThis); static rsRetVal strmPhysWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf); +static rsRetVal strmSeekCurrOffs(strm_t *pThis); /* methods */ @@ -197,6 +198,7 @@ static rsRetVal doPhysOpen(strm_t *pThis) { int iFlags = 0; + struct stat statOpen; DEFiRet; ISOBJ_TYPE_assert(pThis, strm); @@ -234,15 +236,71 @@ doPhysOpen(strm_t *pThis) ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); else ABORT_FINALIZE(RS_RET_IO_ERROR); + } + + if(pThis->tOperationsMode == STREAMMODE_READ) { + if(fstat(pThis->fd, &statOpen) == -1) { + DBGPRINTF("Error: cannot obtain inode# for file %s\n", pThis->pszCurrFName); + ABORT_FINALIZE(RS_RET_IO_ERROR); + } + pThis->inode = statOpen.st_ino; + } + + if(!ustrcmp(pThis->pszCurrFName, UCHAR_CONSTANT(_PATH_CONSOLE)) || isatty(pThis->fd)) { + DBGPRINTF("file %d is a tty-type file\n", pThis->fd); + pThis->bIsTTY = 1; } else { - if(!ustrcmp(pThis->pszCurrFName, UCHAR_CONSTANT(_PATH_CONSOLE)) || isatty(pThis->fd)) { - DBGPRINTF("file %d is a tty-type file\n", pThis->fd); - pThis->bIsTTY = 1; + pThis->bIsTTY = 0; + } + +finalize_it: + RETiRet; +} + + +static rsRetVal +strmSetCurrFName(strm_t *pThis) +{ + DEFiRet; + + if(pThis->sType == STREAMTYPE_FILE_CIRCULAR) { + CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, + pThis->pszFName, pThis->lenFName, pThis->iCurrFNum, pThis->iFileNumDigits)); + } else { + if(pThis->pszDir == NULL) { + if((pThis->pszCurrFName = ustrdup(pThis->pszFName)) == NULL) + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); } else { - pThis->bIsTTY = 0; + CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, + pThis->pszFName, pThis->lenFName, -1, 0)); } } +finalize_it: + RETiRet; +} + +/* This function checks if the actual file has changed and, if so, resets the + * offset. This is support for monitoring files. It should be called after + * deserializing the strm object and before doing any other operation on it + * (most importantly not an open or seek!). + */ +static rsRetVal +CheckFileChange(strm_t *pThis) +{ + struct stat statName; + DEFiRet; + CHKiRet(strmSetCurrFName(pThis)); + if(stat((char*) pThis->pszCurrFName, &statName) == -1) + ABORT_FINALIZE(RS_RET_IO_ERROR); + DBGPRINTF("stream/after deserialize checking for file change on '%s', " + "inode %u/%u, size/currOffs %llu/%llu\n", + pThis->pszCurrFName, (unsigned) pThis->inode, + (unsigned) statName.st_ino, statName.st_size, pThis->iCurrOffs); + if(pThis->inode != statName.st_ino || statName.st_size < pThis->iCurrOffs) { + DBGPRINTF("stream: file %s has changed\n", pThis->pszCurrFName); + pThis->iCurrOffs = 0; + } finalize_it: RETiRet; } @@ -265,19 +323,8 @@ static rsRetVal strmOpenFile(strm_t *pThis) if(pThis->pszFName == NULL) ABORT_FINALIZE(RS_RET_FILE_PREFIX_MISSING); - if(pThis->sType == STREAMTYPE_FILE_CIRCULAR) { - CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, - pThis->pszFName, pThis->lenFName, pThis->iCurrFNum, pThis->iFileNumDigits)); - } else { - if(pThis->pszDir == NULL) { - if((pThis->pszCurrFName = ustrdup(pThis->pszFName)) == NULL) - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - } else { - CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, - pThis->pszFName, pThis->lenFName, -1, 0)); - } - } - + CHKiRet(strmSetCurrFName(pThis)); + CHKiRet(doPhysOpen(pThis)); pThis->iCurrOffs = 0; @@ -357,6 +404,7 @@ static rsRetVal strmCloseFile(strm_t *pThis) if(pThis->fd != -1) { close(pThis->fd); pThis->fd = -1; + pThis->inode = 0; } if(pThis->fdDir != -1) { @@ -432,18 +480,15 @@ static rsRetVal strmHandleEOFMonitor(strm_t *pThis) { DEFiRet; - struct stat statOpen; struct stat statName; ISOBJ_TYPE_assert(pThis, strm); - if(fstat(pThis->fd, &statOpen) == -1) - ABORT_FINALIZE(RS_RET_IO_ERROR); if(stat((char*) pThis->pszCurrFName, &statName) == -1) ABORT_FINALIZE(RS_RET_IO_ERROR); - DBGPRINTF("stream checking for file change on '%s', inode %u/%u", - pThis->pszCurrFName, (unsigned) statOpen.st_ino, + DBGPRINTF("stream checking for file change on '%s', inode %u/%u\n", + pThis->pszCurrFName, (unsigned) pThis->inode, (unsigned) statName.st_ino); - if(statOpen.st_ino == statName.st_ino) { + if(pThis->inode == statName.st_ino) { ABORT_FINALIZE(RS_RET_EOF); } else { /* we had a file change! */ @@ -1343,7 +1388,11 @@ static rsRetVal strmSeek(strm_t *pThis, off64_t offs) } long long i; DBGOPRINT((obj_t*) pThis, "file %d seek, pos %llu\n", pThis->fd, (long long unsigned) offs); - i = lseek64(pThis->fd, offs, SEEK_SET); // TODO: check error! + i = lseek64(pThis->fd, offs, SEEK_SET); + if(i != offs) { + DBGPRINTF("strmSeek: error %lld seeking to offset %lld\n", i, offs); + ABORT_FINALIZE(RS_RET_IO_ERROR); + } pThis->iCurrOffs = offs; /* we are now at *this* offset */ pThis->iBufPtr = 0; /* buffer invalidated */ @@ -1697,6 +1746,9 @@ static rsRetVal strmSerialize(strm_t *pThis, strm_t *pStrm) l = pThis->iCurrOffs; objSerializeSCALAR_VAR(pStrm, iCurrOffs, INT64, l); + l = pThis->inode; + objSerializeSCALAR_VAR(pStrm, inode, INT64, l); + objSerializePTR(pStrm, prevLineSegment, PSZ); CHKiRet(obj.EndSerialize(pStrm)); @@ -1796,6 +1848,8 @@ static rsRetVal strmSetProperty(strm_t *pThis, var_t *pProp) CHKiRet(strmSettOpenMode(pThis, pProp->val.num)); } else if(isProp("iCurrOffs")) { pThis->iCurrOffs = pProp->val.num; + } else if(isProp("inode")) { + pThis->inode = (ino_t) pProp->val.num; } else if(isProp("iMaxFileSize")) { CHKiRet(strmSetiMaxFileSize(pThis, pProp->val.num)); } else if(isProp("iMaxFiles")) { @@ -1865,6 +1919,7 @@ CODESTARTobjQueryInterface(strm) pIf->GetCurrOffset = strmGetCurrOffset; pIf->Dup = strmDup; pIf->SetWCntr = strmSetWCntr; + pIf->CheckFileChange = CheckFileChange; /* set methods */ pIf->SetbDeleteOnClose = strmSetbDeleteOnClose; pIf->SetiMaxFileSize = strmSetiMaxFileSize; diff --git a/runtime/stream.h b/runtime/stream.h index b7e74074..b7cc6d36 100644 --- a/runtime/stream.h +++ b/runtime/stream.h @@ -112,6 +112,7 @@ typedef struct strm_s { int lenDir; int fd; /* the file descriptor, -1 if closed */ int fdDir; /* the directory's descriptor, in case bSync is requested (-1 if closed) */ + ino_t inode; /* current inode for files being monitored (undefined else) */ uchar *pszCurrFName; /* name of current file (if open) */ uchar *pIOBuf; /* the iobuffer currently in use to gather data */ size_t iBufPtrMax; /* current max Ptr in Buffer (if partial read!) */ @@ -187,8 +188,10 @@ BEGINinterface(strm) /* name must also be changed in ENDinterface macro! */ rsRetVal (*ReadLine)(strm_t *pThis, cstr_t **ppCStr, int mode); /* v7 added 2012-09-14 */ INTERFACEpropSetMeth(strm, bVeryReliableZip, int); + /* v8 added 2013-03-21 */ + rsRetVal (*CheckFileChange)(strm_t *pThis); ENDinterface(strm) -#define strmCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ +#define strmCURR_IF_VERSION 8 /* increment whenever you change the interface structure! */ static inline int strmGetCurrFileNum(strm_t *pStrm) { diff --git a/runtime/wtp.c b/runtime/wtp.c index f8d3588b..19151e7c 100644 --- a/runtime/wtp.c +++ b/runtime/wtp.c @@ -381,9 +381,9 @@ wtpWorker(void *arg) /* the arg is actually a wti object, even though we are in if(prctl(PR_SET_NAME, thrdName, 0, 0, 0) != 0) { DBGPRINTF("prctl failed, not setting thread name for '%s'\n", wtpGetDbgHdr(pThis)); } + dbgOutputTID((char*)thrdName); # endif - dbgOutputTID((char*)thrdName); pthread_cleanup_push(wtpWrkrExecCancelCleanup, pWti); wtiWorker(pWti); pthread_cleanup_pop(0); diff --git a/tools/pmrfc3164.c b/tools/pmrfc3164.c index bcded428..5dfa74f0 100644 --- a/tools/pmrfc3164.c +++ b/tools/pmrfc3164.c @@ -138,7 +138,7 @@ CODESTARTparse */ if(lenMsg > 0 && pMsg->msgFlags & PARSE_HOSTNAME) { i = 0; - while(i < lenMsg && (isalnum(p2parse[i]) || p2parse[i] == '.' || p2parse[i] == '.' + while(i < lenMsg && (isalnum(p2parse[i]) || p2parse[i] == '.' || p2parse[i] == '_' || p2parse[i] == '-') && i < (CONF_HOSTNAME_MAXSIZE - 1)) { bufParseHOSTNAME[i] = p2parse[i]; ++i; diff --git a/tools/rsyslog.conf.5 b/tools/rsyslog.conf.5 index fe9e083b..07da6ffd 100644 --- a/tools/rsyslog.conf.5 +++ b/tools/rsyslog.conf.5 @@ -218,7 +218,7 @@ beginning with a slash ('/'). .B Example: .RS -*.* /var/log/traditionalfile.log;RSYSLOG_TraditionalFormat # log to a file in the traditional format +*.* /var/log/traditionalfile.log;RSYSLOG_TraditionalFileFormat # log to a file in the traditional format .RE Note: if you would like to use high-precision timestamps in your log files, diff --git a/tools/syslogd.c b/tools/syslogd.c index ae1b3de5..e291ba47 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -195,7 +195,6 @@ static prop_t *pInternalInputName = NULL; /* there is only one global inputName static uchar *ConfFile = (uchar*) _PATH_LOGCONF; /* read-only after startup */ static char *PidFile = _PATH_LOGPID; /* read-only after startup */ -static pid_t myPid; /* our pid for use in self-generated messages, e.g. on startup */ /* mypid is read-only after the initial fork() */ static int bHadHUP = 0; /* did we have a HUP? */ @@ -219,7 +218,7 @@ static ratelimit_t *dflt_ratelimiter = NULL; /* ratelimiter for submits without static ratelimit_t *internalMsg_ratelimiter = NULL; /* ratelimiter for rsyslog-own messages */ int MarkInterval = 20 * 60; /* interval between marks in seconds - read-only after startup */ int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ -static int NoFork = 0; /* don't fork - don't run in daemon mode - read-only after startup */ +static int doFork = 1; /* fork - run in daemon mode - read-only after startup */ int bHaveMainQueue = 0;/* set to 1 if the main queue - in queueing mode - is available * If the main queue is either not yet ready or not running in * queueing mode (mode DIRECT!), then this is set to 0. @@ -474,7 +473,7 @@ logmsgInternal(int iErr, int pri, uchar *msg, int flags) * permits us to process unmodified config files which otherwise contain a * supressor statement. */ - if(((Debug == DEBUG_FULL || NoFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { + if(((Debug == DEBUG_FULL || !doFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { if(LOG_PRI(pri) == LOG_ERR) fprintf(stderr, "rsyslogd: %s\n", msg); } @@ -795,7 +794,7 @@ die(int sig) (void) snprintf(buf, sizeof(buf) / sizeof(char), " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"]" " exiting on signal %d.", - (int) myPid, sig); + (int) glblGetOurPid(), sig); errno = 0; logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)buf, 0); } @@ -1169,7 +1168,7 @@ init(void) snprintf(bufStartUpMsg, sizeof(bufStartUpMsg)/sizeof(char), " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] start", - (int) myPid); + (int) glblGetOurPid()); logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)bufStartUpMsg, 0); } @@ -1248,7 +1247,7 @@ doHUP(void) snprintf(buf, sizeof(buf) / sizeof(char), " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] rsyslogd was HUPed", - (int) myPid); + (int) glblGetOurPid()); errno = 0; logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)buf, 0); } @@ -1615,8 +1614,7 @@ doGlblProcessInit(void) thrdInit(); - if( !(Debug == DEBUG_FULL || NoFork) ) - { + if(doFork) { DBGPRINTF("Checking pidfile '%s'.\n", PidFile); if (!check_pid(PidFile)) { @@ -1628,16 +1626,23 @@ doGlblProcessInit(void) /* stop writing debug messages to stdout (if debugging is on) */ stddbg = -1; + dbgprintf("ready for forking\n"); if (fork()) { /* Parent process */ - sleep(300); - /* Not reached unless something major went wrong. 5 - * minutes should be a fair amount of time to wait. - * Please note that this procedure is important since - * the father must not exit before syslogd isn't - * initialized or the klogd won't be able to flush its - * logs. -Joey + dbgprintf("parent process going to sleep for 60 secs\n"); + sleep(60); + /* Not reached unless something major went wrong. 1 + * minute should be a fair amount of time to wait. + * The parent should not exit before rsyslogd is + * properly initilized (at least almost) or the init + * system may get a wrong impression of our readyness. + * Note that we exit before being completely initialized, + * but at this point it is very, very unlikely that something + * bad can happen. We do this here, because otherwise we would + * need to have much more code to handle priv drop (which we + * don't consider worth for the init system, especially as it + * is going away on the majority of distros). */ exit(1); /* "good" exit - after forking, not diasabling anything */ } @@ -1646,6 +1651,7 @@ doGlblProcessInit(void) close(0); /* we keep stdout and stderr open in case we have to emit something */ i = 3; + dbgprintf("in child, finalizing initialization\n"); /* if (sd_booted()) */ { const char *e; @@ -1679,7 +1685,8 @@ doGlblProcessInit(void) i = SD_LISTEN_FDS_START + sd_fds; } for ( ; i < num_fds; i++) - (void) close(i); + if(i != dbgGetDbglogFd()) + close(i); untty(); } else { @@ -1704,7 +1711,7 @@ doGlblProcessInit(void) fputs("Pidfile (and pid) already exist.\n", stderr); exit(1); /* exit during startup - questionable */ } - myPid = getpid(); /* save our pid for further testing (also used for messages) */ + glblSetOurPid(getpid()); memset(&sigAct, 0, sizeof (sigAct)); sigemptyset(&sigAct.sa_mask); @@ -1890,7 +1897,7 @@ int realMain(int argc, char **argv) fprintf(stderr, "rsyslogd: error -m is no longer supported - use immark instead"); break; case 'n': /* don't fork */ - NoFork = 1; + doFork = 0; break; case 'N': /* enable config verify mode */ iConfigVerify = atoi(arg); @@ -1994,17 +2001,16 @@ int realMain(int argc, char **argv) if(!iConfigVerify) CHKiRet(doGlblProcessInit()); + /* Send a signal to the parent so it can terminate. */ + if(glblGetOurPid() != ppid) + kill(ppid, SIGTERM); + CHKiRet(init()); if(Debug && debugging_on) { dbgprintf("Debugging enabled, SIGUSR1 to turn off debugging.\n"); } - /* Send a signal to the parent so it can terminate. */ - if(myPid != ppid) - kill(ppid, SIGTERM); - - /* END OF INTIALIZATION */ DBGPRINTF("initialization completed, transitioning to regular run mode\n"); @@ -2015,7 +2021,7 @@ int realMain(int argc, char **argv) * is still in its infancy (and not really done), we currently accept this issue. * rgerhards, 2009-06-29 */ - if(!(Debug == DEBUG_FULL || NoFork)) { + if(doFork) { close(1); close(2); ourConf->globals.bErrMsgToStderr = 0; |