diff options
82 files changed, 7332 insertions, 441 deletions
@@ -1,5 +1,83 @@ --------------------------------------------------------------------------- -Version 7.3.7 [devel] 2013-02-?? +Version 7.3.11 [devel] 2013-04-?? +- added support for encrypting log files +- bugfix: $PreserveFQDN is not properly working + Thanks to Louis Bouchard for the patch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=426 +- bugfix: imuxsock aborted due to problem in ratelimiting code + Thanks to Tomas Heinrich for the patch. +- bugfix: imuxsock aborted under some conditions + regression from ratelimiting enhancements - this was a different one + to the one Tomas Heinrich patched. +--------------------------------------------------------------------------- +Version 7.3.10 [devel] 2013-04-10 +- added RainerScript re_extract() function +- omrelp: added support for RainerScript-based configuration +- omrelp: added ability to specify session timeout +- templates now permit substring extraction relative to end-of-string +- bugfix: failover/action suspend did not work correctly + This was experienced if the retry action took more than one second + to complete. For suspending, a cached timestamp was used, and if the + retry took longer, that timestamp was already in the past. As a + result, the action never was kept in suspended state, and as such + no failover happened. The suspend functionalit now does no longer use + the cached timestamp (should not have any performance implication, as + action suspend occurs very infrequently). +- bugfix: gnutls RFC5425 driver had some undersized buffers + Thanks to Tomas Heinrich for the patch. +- bugfix: nested if/prifilt conditions did not work properly + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=415 +- bugfix: imuxsock aborted under some conditions + regression from ratelimiting enhancements +- bugfix: build problems on Solaris + Thanks to Martin Carpenter for the patches. +--------------------------------------------------------------------------- +Version 7.3.9 [devel] 2013-03-27 +- support for signing logs added +- imudp: now supports user-selectable inputname +- omlibdbi: now supports transaction interface + if recent enough lbdbi is present +- imuxsock: add ability to NOT create/delete sockets during startup and + shutdown + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=259 +- imfile: errors persisting state file are now reported + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=292 +- imfile: now detects file change when rsyslog was inactive + Previosly, this case could not be detected, so if a file was overwritten + or rotated away while rsyslog was stopped, some data was missing. This + is now detected and the new file being forwarded right from the + beginning. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=228 +- updated systemd files to match current systemd source +- bugfix: imudp scheduling parameters did affect main thread, not imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409 +- bugfix: build problem on platforms without GLOB_NOMAGIC +- bugfix: build problems on non-Linux platforms +- bugfix: stdout/stderr were not closed on forking + but were closed when running in the forground - this was just reversed + of what it should be. This is a regression of a recent change. +--------------------------------------------------------------------------- +Version 7.3.8 [devel] 2013-03-18 +- imrelp: now supports listening to IPv4/v6 only instead of always both + build now requires librelp 1.0.2 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=378 +- bugfix: mmanon did not build on some platforms (e.g. Ubuntu) +- bugfix: segfault in expression optimizer + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=423 +- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter + To use that functionality, legacy rsyslog.conf syntax had to be used. + Also, the doc was missing information on the "ParseTrusted" set of + config directives. +- bugfix: include files got included in the wrong order + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411 + This happens if an $IncludeConfig directive was done on multiple + files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf). + In that case, the order of include file processing is reversed, which + could lead to all sorts of problems. + Thanks to Nathan Stratton Treadway for his great analysis of the problem, + which made bug fixing really easy. +--------------------------------------------------------------------------- +Version 7.3.7 [devel] 2013-03-12 - add support for anonymizing IPv4 addresses - add support for writing to the Linux Journal (omjournal) - imuxsock: add capability to ignore messages from ourselfes @@ -158,8 +236,70 @@ Version 7.3.0 [devel] 2012-10-09 This was achieved by somewhat reducing the robustness of the zip archive. This is controlled by the new action parameter "VeryReliableZip". ---------------------------------------------------------------------------- -Version 7.2.6 [v7-stable] 2013-01-?? +Version 7.2.7 [v7-stable] 2013-04-17 +- rsyslogd startup information is now properly conveyed back to init + when privileges are beging dropped + Actually, we have moved termination of the parent in front of the + priv drop. So it shall work now in all cases. See code comments in + commit for more details. +- If forking, the parent now waits for a maximum of 60 seconds for + termination by the child +- improved debugging support in forked (auto-backgrounding) mode + The rsyslog debug log file is now continued to be written across the + fork. +- updated systemd files to match current systemd source +- bugfix: failover/action suspend did not work correctly + This was experienced if the retry action took more than one second + to complete. For suspending, a cached timestamp was used, and if the + retry took longer, that timestamp was already in the past. As a + result, the action never was kept in suspended state, and as such + no failover happened. The suspend functionalit now does no longer use + the cached timestamp (should not have any performance implication, as + action suspend occurs very infrequently). +- bugfix: nested if/prifilt conditions did not work properly + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=415 +- bugfix: script == comparison did not work properly on JSON objects + [backport from 7.3 branch] +- bugfix: imudp scheduling parameters did affect main thread, not imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409 +- bugfix: imuxsock rate-limiting could not be configured via legacy conf + Rate-limiting for the system socket could not be configured via legacy + configuration directives. However, the new-style RainerScript config + options worked. + Thanks to Milan Bartos for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=390 +- bugfix: using group resolution could lead to endless loop + Thanks to Tomas Heinrich for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 +- bugfix: $mmnormalizeuseramsg paramter was specified with wrong type + Thank to Renzhong Zhang for alerting us of the problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=420 +- bugfix: RainerScript getenv() function caused segfault when var was + not found. + Thanks to Philippe Muller for the patch. +- bugfix: several issues in imkmsg + see bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=421#c8 +- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter + To use that functionality, legacy rsyslog.conf syntax had to be used. + Also, the doc was missing information on the "ParseTrusted" set of + config directives. +- bugfix: parameter action.execOnlyWhenPreviousIsSuspended was accidently + of integer-type. For obvious reasons, it needs to be boolean. Note + that this change can break existing configurations if they circumvented + the problem by using 0/1 values. +- doc bugfix: rsyslog.conf man page had invalid file format info + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=418 +---------------------------------------------------------------------------- +Version 7.2.6 [v7-stable] 2013-03-05 - slightly improved config parser error messages when invalid escapes happen +- bugfix: include files got included in the wrong order + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411 + This happens if an $IncludeConfig directive was done on multiple + files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf). + In that case, the order of include file processing is reversed, which + could lead to all sorts of problems. + Thanks to Nathan Stratton Treadway for his great analysis of the problem, + which made bug fixing really easy. - bugfix: omelasticsearch failed when authentication data was provided ... at least in most cases it emitted an error message: "snprintf failed when trying to build auth string" @@ -1158,6 +1298,7 @@ expected that interfaces, even new ones, break during the initial [ported from v4] --------------------------------------------------------------------------- Version 5.10.2 [V5-STABLE], 201?-??-?? +- updated systemd files to match current systemd source - bugfix: spurios error messages from imuxsock about (non-error) EAGAIN Thanks to Marius Tomaschewski for the patch. - imklog: added $klogParseKernelTimestamp option @@ -188,7 +188,7 @@ static struct cnfparamdescr cnfparamdescr[] = { { "action.execonlyeverynthtime", eCmdHdlrInt, 0 }, /* legacy: actionexeconlyeverynthtime */ { "action.execonlyeverynthtimetimeout", eCmdHdlrInt, 0 }, /* legacy: actionexeconlyeverynthtimetimeout */ { "action.execonlyonceeveryinterval", eCmdHdlrInt, 0 }, /* legacy: actionexeconlyonceeveryinterval */ - { "action.execonlywhenpreviousissuspended", eCmdHdlrInt, 0 }, /* legacy: actionexeconlywhenpreviousissuspended */ + { "action.execonlywhenpreviousissuspended", eCmdHdlrBinary, 0 }, /* legacy: actionexeconlywhenpreviousissuspended */ { "action.repeatedmsgcontainsoriginalmsg", eCmdHdlrBinary, 0 }, /* legacy: repeatedmsgcontainsoriginalmsg */ { "action.resumeretrycount", eCmdHdlrInt, 0 }, /* legacy: actionresumeretrycount */ { "action.resumeinterval", eCmdHdlrInt, 0 } @@ -605,13 +605,17 @@ static void actionDisable(action_t *pThis) * CPU time. TODO: maybe a config option for that? * rgerhards, 2007-08-02 */ -static inline void actionSuspend(action_t *pThis, time_t ttNow) +static inline void actionSuspend(action_t *pThis) { - if(ttNow == NO_TIME_PROVIDED) - datetime.GetTime(&ttNow); + time_t ttNow; + + /* note: we can NOT use a cached timestamp, as time may have evolved + * since caching, and this would break logic (and it actually did so!) + */ + datetime.GetTime(&ttNow); pThis->ttResumeRtry = ttNow + pThis->iResumeInterval * (pThis->iNbrResRtry / 10 + 1); actionSetState(pThis, ACT_STATE_SUSP); - DBGPRINTF("earliest retry=%d\n", (int) pThis->ttResumeRtry); + DBGPRINTF("action suspended, earliest retry=%d\n", (int) pThis->ttResumeRtry); } @@ -631,7 +635,7 @@ static inline void actionSuspend(action_t *pThis, time_t ttNow) * of its inability to recover. -- rgerhards, 2010-04-26. */ static inline rsRetVal -actionDoRetry(action_t *pThis, time_t ttNow, int *pbShutdownImmediate) +actionDoRetry(action_t *pThis, int *pbShutdownImmediate) { int iRetries; int iSleepPeriod; @@ -642,7 +646,9 @@ actionDoRetry(action_t *pThis, time_t ttNow, int *pbShutdownImmediate) iRetries = 0; while((*pbShutdownImmediate == 0) && pThis->eState == ACT_STATE_RTRY) { + DBGPRINTF("actionDoRetry: enter loop, iRetries=%d\n", iRetries); iRet = pThis->pMod->tryResume(pThis->pModData); + DBGPRINTF("actionDoRetry: action->tryResume returned %d\n", iRet); if((pThis->iResumeOKinRow > 9) && (pThis->iResumeOKinRow % 10 == 0)) { bTreatOKasSusp = 1; pThis->iResumeOKinRow = 0; @@ -650,16 +656,18 @@ actionDoRetry(action_t *pThis, time_t ttNow, int *pbShutdownImmediate) bTreatOKasSusp = 0; } if((iRet == RS_RET_OK) && (!bTreatOKasSusp)) { + DBGPRINTF("actionDoRetry: had success RDY again (iRet=%d)\n", iRet); actionSetState(pThis, ACT_STATE_RDY); } else if(iRet == RS_RET_SUSPENDED || bTreatOKasSusp) { /* max retries reached? */ + DBGPRINTF("actionDoRetry: check for max retries, iResumeRetryCount %d, iRetries %d\n", + pThis->iResumeRetryCount, iRetries); if((pThis->iResumeRetryCount != -1 && iRetries >= pThis->iResumeRetryCount)) { - actionSuspend(pThis, ttNow); + actionSuspend(pThis); } else { ++pThis->iNbrResRtry; ++iRetries; iSleepPeriod = pThis->iResumeInterval; - ttNow += iSleepPeriod; /* not truly exact, but sufficiently... */ srSleep(iSleepPeriod, 0); if(*pbShutdownImmediate) { ABORT_FINALIZE(RS_RET_FORCE_TERM); @@ -706,7 +714,7 @@ static rsRetVal actionTryResume(action_t *pThis, int *pbShutdownImmediate) if(pThis->eState == ACT_STATE_RTRY) { if(ttNow == NO_TIME_PROVIDED) /* use cached result if we have it */ datetime.GetTime(&ttNow); - CHKiRet(actionDoRetry(pThis, ttNow, pbShutdownImmediate)); + CHKiRet(actionDoRetry(pThis, pbShutdownImmediate)); } if(Debug && (pThis->eState == ACT_STATE_RTRY ||pThis->eState == ACT_STATE_SUSP)) { @@ -953,6 +961,8 @@ actionProcessMessage(action_t *pThis, msg_t *pMsg, void *actParams, int *pbShutd ISOBJ_TYPE_assert(pMsg, msg); CHKiRet(actionPrepare(pThis, pbShutdownImmediate)); + if(pThis->pMod->mod.om.SetShutdownImmdtPtr != NULL) + pThis->pMod->mod.om.SetShutdownImmdtPtr(pThis->pModData, pbShutdownImmediate); if(pThis->eState == ACT_STATE_ITX) CHKiRet(actionCallDoAction(pThis, pMsg, actParams)); @@ -1111,6 +1121,7 @@ submitBatch(action_t *pAction, batch_t *pBatch, int nElem) assert(pBatch != NULL); + DBGPRINTF("submitBatch: enter, nElem %d\n", nElem); wasDoneTo = pBatch->iDoneUpTo; bDone = 0; do { @@ -1132,7 +1143,8 @@ submitBatch(action_t *pAction, batch_t *pBatch, int nElem) || localRet == RS_RET_DEFER_COMMIT) { bDone = 1; } else if(localRet == RS_RET_SUSPENDED) { - ; /* do nothing, this will retry the full batch */ + DBGPRINTF("action ret RS_RET_SUSPENDED - retry full batch\n"); + /* do nothing, this will retry the full batch */ } else if(localRet == RS_RET_ACTION_FAILED) { /* in this case, everything not yet committed is BAD */ for(i = pBatch->iDoneUpTo ; i < wasDoneTo + nElem ; ++i) { @@ -1262,8 +1274,11 @@ processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate) assert(pBatch != NULL); - pbShutdownImmdtSave = pBatch->pbShutdownImmediate; - pBatch->pbShutdownImmediate = pbShutdownImmediate; + if(pbShutdownImmediate != NULL) { + pbShutdownImmdtSave = pBatch->pbShutdownImmediate; + pBatch->pbShutdownImmediate = pbShutdownImmediate; +dbgprintf("DDDD: processBatchMain ShutdownImmediate is %p, was %p\n", pBatch->pbShutdownImmediate, pbShutdownImmdtSave); + } CHKiRet(prepareBatch(pAction, pBatch, &activeSave, &bMustRestoreActivePtr)); /* We now must guard the output module against execution by multiple threads. The @@ -1294,7 +1309,8 @@ processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate) } finalize_it: - pBatch->pbShutdownImmediate = pbShutdownImmdtSave; + if(pbShutdownImmediate != NULL) + pBatch->pbShutdownImmediate = pbShutdownImmdtSave; RETiRet; } #pragma GCC diagnostic warning "-Wempty-body" @@ -1864,7 +1880,7 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, pAction->eState = ACT_STATE_RDY; /* action is enabled */ if(bSuspended) - actionSuspend(pAction, datetime.GetTime(NULL)); /* "good" time call, only during init and unavoidable */ + actionSuspend(pAction); CHKiRet(actionConstructFinalize(pAction, queueParams)); diff --git a/compat/Makefile.am b/compat/Makefile.am index 635ad280..78c298e0 100644 --- a/compat/Makefile.am +++ b/compat/Makefile.am @@ -1,4 +1,4 @@ -pkglib_LTLIBRARIES = compat.la +noinst_LTLIBRARIES = compat.la compat_la_SOURCES = getifaddrs.c compat_la_CPPFLAGS = -I$(top_srcdir) $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) diff --git a/configure.ac b/configure.ac index 64d0126b..352404d6 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[7.3.6],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[7.3.11],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -28,11 +28,12 @@ fi AC_DISABLE_STATIC AC_PROG_LIBTOOL AC_CANONICAL_HOST +AC_PATH_PROG([RST2MAN], [rst2man]) PKG_PROG_PKG_CONFIG # modules we require -PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.2) +PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.5) PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0) PKG_CHECK_MODULES([JSON_C], [json]) @@ -55,6 +56,10 @@ case "${host}" in AC_DEFINE([OS_SOLARIS], [1], [Indicator for a Solaris OS]) AC_DEFINE([_POSIX_PTHREAD_SEMANTICS], [1], [Use POSIX pthread semantics]) SOL_LIBS="-lsocket -lnsl" + # Solaris libuuid does not ship with a pkgconfig file so override the appropriate + # variables (but only if they have not been set by the user). + LIBUUID_CFLAGS=${LIBUUID_CFLAGS:= } + LIBUUID_LIBS=${LIBUUID_LIBS:=-luuid} AC_SUBST(SOL_LIBS) ;; esac @@ -66,6 +71,8 @@ save_LIBS=$LIBS LIBS= AC_SEARCH_LIBS(clock_gettime, rt) RT_LIBS=$LIBS +AC_SEARCH_LIBS(mq_getattr, rt) +RT_LIBS="$RT_LIBS $LIBS" LIBS= AC_SEARCH_LIBS(dlopen, dl) DL_LIBS=$LIBS @@ -114,7 +121,10 @@ AC_TYPE_SIGNAL AC_FUNC_STAT AC_FUNC_STRERROR_R AC_FUNC_VPRINTF -AC_CHECK_FUNCS([flock basename alarm clock_gettime getifaddrs gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64]) +AC_CHECK_FUNCS([flock basename alarm clock_gettime gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64]) + +# getifaddrs is in libc (mostly) or in libsocket (eg Solaris 11) or not defined (eg Solaris 10) +AC_SEARCH_LIBS([getifaddrs], [socket], [AC_DEFINE(HAVE_GETIFADDRS, [1], [set define])]) # the check below is probably ugly. If someone knows how to do it in a better way, please # let me know! -- rgerhards, 2010-10-06 @@ -122,7 +132,9 @@ AC_CHECK_DECL([SCM_CREDENTIALS], [AC_DEFINE(HAVE_SCM_CREDENTIALS, [1], [set defi #include <sys/socket.h>]) AC_CHECK_DECL([SO_TIMESTAMP], [AC_DEFINE(HAVE_SO_TIMESTAMP, [1], [set define])], [], [#include <sys/types.h> #include <sys/socket.h>]) +AC_CHECK_DECL([SYS_gettid], [AC_DEFINE(HAVE_SYS_gettid, [1], [set define])], [], [#include <sys/syscall.h>]) AC_CHECK_MEMBER([struct sysinfo.uptime], [AC_DEFINE(HAVE_SYSINFO_UPTIME, [1], [set define])], [], [#include <sys/sysinfo.h>]) +AC_CHECK_DECL([GLOB_NOMAGIC], [AC_DEFINE(HAVE_GLOB_NOMAGIC, [1], [set define])], [], [#include <glob.h>]) # Check for MAXHOSTNAMELEN AC_MSG_CHECKING(for MAXHOSTNAMELEN) @@ -672,6 +684,11 @@ if test "x$enable_libdbi" = "xyes"; then [dbi_initialize_r], [AC_DEFINE([HAVE_DBI_R], [1], [Define to 1 if libdbi supports the new plugin-safe interface])] ) + AC_CHECK_LIB( + [dbi], + [dbi_conn_transaction_begin], + [AC_DEFINE([HAVE_DBI_TXSUPP], [1], [Define to 1 if libdbi supports transactions])] + ) fi AM_CONDITIONAL(ENABLE_OMLIBDBI, test x$enable_libdbi = xyes) AC_SUBST(LIBDBI_CFLAGS) @@ -756,6 +773,38 @@ if test "x$enable_gnutls" = "xyes"; then fi AM_CONDITIONAL(ENABLE_GNUTLS, test x$enable_gnutls = xyes) +# libgcrypt support +AC_ARG_ENABLE(libgcrypt, + [AS_HELP_STRING([--enable-libgcrypt],[Enable libgcrypt support @<:@default=yes@:>@])], + [case "${enableval}" in + yes) enable_libgcrypt="yes" ;; + no) enable_libgcrypt="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-libgcrypt) ;; + esac], + [enable_libgcrypt=yes] +) +if test "x$enable_libgcrypt" = "xyes"; then + AC_CHECK_PROG( + [HAVE_LIBGCRYPT_CONFIG], + [libgcrypt-config], + [yes],,, + ) + if test "x${HAVE_LIBGCRYPT_CONFIG}" != "xyes"; then + AC_MSG_FAILURE([libgcrypt-config not found in PATH]) + fi + AC_CHECK_LIB( + [gcrypt], + [gcry_cipher_open], + [LIBGCRYPT_CFLAGS="`libgcrypt-config --cflags`" + LIBGCRYPT_LIBS="`libgcrypt-config --libs`" + ], + [AC_MSG_FAILURE([libgrypt is missing])], + [`libgcrypt-config --libs --cflags`] + ) + AC_DEFINE([ENABLE_LIBGCRYPT], [1], [Indicator that LIBGCRYPT is present]) +fi +AM_CONDITIONAL(ENABLE_LIBGCRYPT, test x$enable_libgcrypt = xyes) + # support for building the rsyslogd runtime AC_ARG_ENABLE(rsyslogrt, @@ -929,11 +978,26 @@ AC_ARG_ENABLE(relp, [enable_relp=no] ) if test "x$enable_relp" = "xyes"; then - PKG_CHECK_MODULES(RELP, relp >= 1.0.1) + PKG_CHECK_MODULES(RELP, relp >= 1.0.3) fi AM_CONDITIONAL(ENABLE_RELP, test x$enable_relp = xyes) +# GuardTime support +AC_ARG_ENABLE(guardtime, + [AS_HELP_STRING([--enable-guardtime],[Enable GuardTime support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_guardtime="yes" ;; + no) enable_guardtime="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-guardtime) ;; + esac], + [enable_guardtime=no] +) +if test "x$enable_guardtime" = "xyes"; then + PKG_CHECK_MODULES(GUARDTIME, libgt >= 0.3.1) +fi +AM_CONDITIONAL(ENABLE_GUARDTIME, test x$enable_guardtime = xyes) + # RFC 3195 support AC_ARG_ENABLE(rfc3195, [AS_HELP_STRING([--enable-rfc3195],[Enable RFC3195 support @<:@default=no@:>@])], @@ -1392,6 +1456,8 @@ echo " rsyslogd will be built: $enable_rsyslogd" echo " GUI components will be built: $enable_gui" echo " Unlimited select() support enabled: $enable_unlimited_select" echo " uuid support enabled: $enable_uuid" +echo " GuardTime signature support enabled: $enable_guardtime" +echo " libgcrypt support enabled: $enable_libgcrypt" echo " anonymization support enabled: $enable_mmanon" echo echo "---{ input plugins }---" diff --git a/doc/Makefile.am b/doc/Makefile.am index de2e1df5..46afd900 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -41,6 +41,7 @@ html_files = \ omudpspoof.html \ omruleset.html \ omsnmp.html \ + sigprov_gt.html \ ommysql.html \ omoracle.html \ omlibdbi.html \ diff --git a/doc/cryprov_gcry.html b/doc/cryprov_gcry.html new file mode 100644 index 00000000..2568add9 --- /dev/null +++ b/doc/cryprov_gcry.html @@ -0,0 +1,121 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> +<head> +<meta http-equiv="Content-Language" content="en"> +<title>libgcryt Log Crypto Provider (gcry)</title> +</head> + +<body> +<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a> + +<h1>libgcrypt Log Crypto Provider (gcry)</h1> +<p><b>Signature Provider Name: gt</b></p> +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Supported Since: </b>since 7.3.10 +<p><b>Description</b>:</p> +<p>Provides encryption support to rsyslog. +</p> + +<p><b>Configuration Parameters</b>:</p> +<p>Crypto providers are loaded by omfile, when the +provider is selected in its "cry.providerName" parameter. +Parameters for the provider are given in the omfile action instance +line. +<p>This provider creates an encryption information file with the same base name but +the extension ".encinfo" for each log file (both for fixed-name files +as well as dynafiles). Both files together form a set. So you need to +archive both in order to prove integrity. +<ul> +<li><b>cry.algo</b> <Encryption Algorithm><br> +The algorithm (cipher) to be used for encryption. +The default algorithm is "AES128". +<br>Currently, the following Algorithms are supported: + <ul> + <li>3DES + <li>CAST5 + <li>BLOWFISH + <li>AES128 + <li>AES192 + <li>AES256 + <li>TWOFISH + <li>TWOFISH128 + <li>ARCFOUR + <li>DES + <li>SERPENT128 + <li>SERPENT192 + <li>SERPENT256 + <li>RFC2268_40 + <li>SEED + <li>CAMELLIA128 + <li>CAMELLIA192 + <li>CAMELLIA256 + </ul> + <br> + The actual availability of an algorithms depends on which ones + are compiled into libgcrypt. Note that some versions of libgcrypt + simply abort the process (rsyslogd in this case!) if a supported + algorithm is select but not available due to libgcrypt build + settings. There is nothing rsyslog can do against this. So in + order to avoid production downtime, always check carefully when + you change the algorithm. +</li> +<li><b>cry.mode</b> <Algorithm Mode><br> +The encryption mode to be used. Default ist Cipher Block Chaining (CBC). +Note that not all encryption modes can be used together with all +algorithms. +<br>Currently, the following modes are supported: + <ul> + <li>ECB + <li>CFB + <li>CBC + <li>STREAM + <li>OFB + <li>CTR + <li>AESWRAP + </ul> +<li><b>cry.key</b> <encryption key><br> + TESTING AID, NOT FOR PRODUCTION USE. This uses the KEY specified + inside rsyslog.conf. This is the actual key, and as such this mode + is highly insecure. However, it can be useful for intial testing + steps. This option may be removed in the future. +</li> +<li><b>cry.keyfile</b> <filename><br> + Reads the key from the specified file. The file must contain the key, only, + no headers or other meta information. Keyfiles can be generated via the + rscrytool utility. +</li> +<li><b>cry.keyprogram</b> <path to program><br> + If given, the key is provided by a so-called "key program". This program + is executed and must return the key to (as well as some meta information) + via stdout. The core idea of key programs is that using this interface the + user can implement as complex (and secure) method to obtain keys as + desired, all without the need to make modifications to rsyslog. +</li> +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>currently none known +</li> +</ul> +<p><b>Samples:</b></p> +<p>This encrypts a log file. Default parameters are used, they key is +provided via a keyfile. +</p> +<textarea rows="3" cols="60"> +action(type="omfile" file="/var/log/somelog" + cry.provider="gcry" keyfile="/secured/path/to/keyfile") +</textarea> +Note that the keyfile can be generated via the rscrytool utility (see its +documentation for how to actually do that). + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2013 by +<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/doc/imfile.html b/doc/imfile.html index 0997e382..f37f7055 100644 --- a/doc/imfile.html +++ b/doc/imfile.html @@ -77,7 +77,9 @@ created in the rsyslog working directory (configurable via $WorkDirectory). Be careful to use unique names for different files being monitored. If there are duplicates, all sorts of "interesting" things may happen. Rsyslog currently does not check if a name is -specified multiple times.</li> +specified multiple times. +Note that when $WorkDirectory is not set or set to a non-writable +location, the state file will not be generated.</li> <li><span style="font-weight: bold;">Facility facility</span><br> The syslog facility to be assigned to lines read. Can be specified in @@ -141,17 +143,17 @@ your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.<br> </p> -<textarea rows="15" cols="60">module(load="folder/to/rsyslog/plugins/imfile/.libs/imfile" PollingInterval="10") #needs to be done just once +<textarea rows="15" cols="60">module(load="imfile" PollingInterval="10") #needs to be done just once # File 1 input(type="imfile" File="/path/to/file1" -Tag="tag1" -StateFile="/var/spool/rsyslog/statefile1" -Severity="error" -Facility="local7") + Tag="tag1" + StateFile="/var/spool/rsyslog/statefile1" + Severity="error" + Facility="local7") # File 2 input(type="imfile" File="/path/to/file2" -Tag="tag2" -StateFile="/var/spool/rsyslog/statefile2") + Tag="tag2" + StateFile="/var/spool/rsyslog/statefile2") # ... and so on ... # </textarea> @@ -210,8 +212,7 @@ your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.<br> </p> -<textarea rows="15" cols="60">$ModLoad imfile # -needs to be done just once +<textarea rows="15" cols="60">$ModLoad imfile # needs to be done just once # File 1 $InputFileName /path/to/file1 $InputFileTag tag1: diff --git a/doc/imrelp.html b/doc/imrelp.html index 856aff82..9f3e4875 100644 --- a/doc/imrelp.html +++ b/doc/imrelp.html @@ -47,7 +47,7 @@ not specific ones. This is due to a currently existing limitation in librelp. <p><b>Sample:</b></p> <p>This sets up a RELP server on port 20514.<br> </p> -<textarea rows="15" cols="60">module(load="/folder/to/rsyslog/plugins/imrelp/.libs/imrelp") # needs to be done just once +<textarea rows="15" cols="60">module(load="imrelp") # needs to be done just once input(type="imrelp" port="20514") </textarea> diff --git a/doc/imudp.html b/doc/imudp.html index f2d04d65..6c949536 100644 --- a/doc/imudp.html +++ b/doc/imudp.html @@ -18,8 +18,8 @@ multiple input actions. </p> -<p><b>Configuration Directives</b>:</p> -<p><b>Global Directives</b>:</p> +<p><b>Configuration Parameters</b>:</p> +<p><b>Module Parameters</b>:</p> <ul> <li><b>TimeRequery</b> <nbr-of-times><br> this is a performance @@ -33,11 +33,11 @@ the value, the less precise the timestamp. <li><b>SchedulingPolicy</b> <rr/fifo/other><br> Can be used the set the scheduler priority, if the necessary functionality is provided by the platform. Most useful to select "fifo" for real-time -processing under Linux (and thus reduce chance of packet loss). +processing under Linux (and thus reduce chance of packet loss). <li><b>SchedulingPriority</b> <number><br> Scheduling priority to use. </ul> -<p><b>Action Directives</b>:</p> +<p><b>Input Parameters</b>:</p> <ul> <li><b>Address</b> <IP><br> local IP address (or name) the UDP listens should bind to</li> @@ -54,18 +54,63 @@ of seconds (5 recommended) to activate rate-limiting. <li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting burst in number of messages. Default is 10,000. </li> +<li><b>InputName</b> [name] - (available since 7.3.9) specifies the value of +the inputname. In older versions, this was always "imudp" for all listeners, +which still i the default. +Starting with 7.3.9 it can be set to different values for each listener. +Note that when a single input statement defines multipe listner ports, the +inputname will be the same for all of them. If you want to differentiate in that +case, use "InputName.AppendPort" to make them unique. +Note that the "InputName" parameter can be an empty string. In that case, the +corresponding inputname property will obviously also be the empty string. This +is primarily meant to be used togehter with "InputName.AppendPort" to set the +inputname equal to the port. +</li> +<li><b>InputName.AppendPort</b> [on/<b>off</b>] - (available since 7.3.9) +appends the port the the inputname. Note that when no inputname is specified, +the default of "imudp" is used and the port is appended to that default. So, +for example, a listner port of 514 in that case will lead to an inputname +of "imudp514". The ability to append a port is most useful when multiple ports +are defined for a single input and each of the inputnames shall be unique. +Note that there currently is no differentiation between IPv4/v6 listners on +the same port. +</li> </ul> <b>Caveats/Known Bugs:</b> <ul> -<li>currently none known</li> +<li>Scheduling parameters are set <b>after</b> privileges have been dropped. +In most cases, this means that setting them will not be possible after +privilege drop. This may be worked around by using a sufficiently-privileged +user account. +</li> </ul> -<p><b>Sample:</b></p> +<p><b>Samples:</b></p> <p>This sets up an UPD server on port 514:<br> </p> -<textarea rows="15" cols="60">module(load="imudp") # needs to be done just once +<textarea rows="3" cols="60">module(load="imudp") # needs to be done just once input(type="imudp" port="514") </textarea> +<p>In the next example, we set up three listners at ports 10514, 10515 and 10516 +and assign a listner name of "udp" to it, followed by the port number: +</p> +<textarea rows="4" cols="60">module(load="imudp") +input(type="imudp" port=["10514","10515","10516"] + inputname="udp" inputname.appendPort="on") +</textarea> + +<p>The next example is almost equal to the previous one, but +now the inputname property will just be set to the port number. +So if a message was received on port 10515, the input name will be +"10515" in this example whereas it was "udp10515" in the previous one. +Note that to do that we set the inputname to the empty string. +</p> +<textarea rows="4" cols="60">module(load="imudp") +input(type="imudp" port=["10514","10515","10516"] + inputname="" inputname.appendPort="on") +</textarea> + + <p><b>Legacy Configuration Directives</b>:</p> <p>Multiple receivers may be configured by specifying $UDPServerRun multiple times. @@ -84,23 +129,20 @@ equivalent to: SchedulingPolicy <li>$IMUDPSchedulingPriority <number> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> equivalent to: SchedulingPriority </ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>currently none known</li> -</ul> <p><b>Sample:</b></p> <p>This sets up an UPD server on port 514:<br> </p> -<textarea rows="15" cols="60">$ModLoad imudp # needs to be done just once +<textarea rows="3" cols="60">$ModLoad imudp # needs to be done just once $UDPServerRun 514 </textarea> + <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and +Copyright © 2009-2013 by +<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> </body></html> diff --git a/doc/imuxsock.html b/doc/imuxsock.html index ee9e2a05..0affe8c3 100644 --- a/doc/imuxsock.html +++ b/doc/imuxsock.html @@ -98,6 +98,16 @@ messages that shall be rate-limited. </li> <li><b>SysSock.Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the system log socket.</li> +<li><b>SysSock.ParseTrusted</b> <on/<b>off</b>> if Annotation is turned on, create +JSON/lumberjack properties out of the trusted properties (which can be accessed +via RainerScript JSON Variables, e.g. "$!pid") instead of adding them to the message. +</li> +<li><b>SysSock.Unlink</b> <<b>on</b>/off> (available since 7.3.9)<br> +if turned on (default), the system socket is unlinked and re-created when +opened and also unlinked when finally closed. Note that this setting has +no effect when running under systemd control (because systemd handles +the socket). +</li> </ul> <p><b>Input Instance Parameters</b></p> @@ -160,6 +170,15 @@ will only affect the next one and then automatically be reset. This functionalit that the local hostname can be overridden in cases where that is desired.</li> <li><b>Annotate</b> <on/<b>off</b>> turn on annotation/trusted properties for the non-system log socket in question.</li> +<li><b>ParseTrusted</b> <on/<b>off</b>> equivalent to the SysSock.ParseTrusted module +parameter, but applies to the input that is being defined. +<li><b>Unlink</b> <<b>on</b>/off> (available since 7.3.9)<br> +if turned on (default), the socket is unlinked and re-created when +opened and also unlinked when finally closed. Set it to off if you +handle socket creation yourself. Note that handling socket creation +oneself has the advantage that a limited amount of messages may be +queued by the OS if rsyslog is not running. +</li> </ul> <b>Caveats/Known Bugs:</b><br> @@ -172,12 +191,20 @@ change the array size in imuxsock.c. <p>The following sample is the minimum setup required to accept syslog messages from applications running on the local system.<br> </p> -<textarea rows="2" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="2" cols="70">module(load="imuxsock" # needs to be done just once SysSock.FlowControl="on") # enable flow control (use if needed) </textarea> + +<p>The following sample is similiar to the first one, but enables trusted +properties, which are put into JSON/lumberjack variables. +<br> +</p> +<textarea rows="2" cols="70">module(load="imuxsock" SysSock.Annotate="on" SysSock.ParseTrusted="on") +</textarea> + <p>The following sample is a configuration where rsyslogd pulls logs from two jails, and assigns different hostnames to each of the jails: </p> -<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once +<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once input(type="imuxsock" HostName="jail1.example.net" Socket="/jail/1/dev/log") input(type="imuxsock" HostName="jail2.example.net" Socket="/jail/2/dev/log") @@ -188,18 +215,18 @@ system. As rsyslogd starts up before the sshd, it needs to create the socket directories, because it otherwise can not open the socket and thus not listen to openssh messages. Note that it is vital not to place any other socket between the CreatePath and the Socket.</p> -<textarea rows="6" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock") # needs to be done just once +<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once input(type="imuxsock" Socket="/var/run/sshd/dev/log" CreatePath="on") </textarea> <p>The following sample is used to turn off input rate limiting on the system log socket. -<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once SysSock.RateLimit.Interval="0") # turn off rate limiting </textarea> <p>The following sample is used activate message annotation and thus trusted properties on the system log socket. -<textarea rows="4" cols="70">module(load="/folder/to/rsyslog/plugins/imuxsock/.libs/imuxsock" # needs to be done just once +<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once SysSock.Annotate="on") </textarea> @@ -243,6 +270,7 @@ equivalent to: SysSock.IgnoreTimestamp.</li> <li><b>$InputUnixListenSocketHostName</b> <hostname> equivalent to: HostName.</li> <li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> equivalent to: Annotate.</li> <li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> equivalent to: SysSock.Annotate.</li> +<li><b>$SystemLogSocketParseTrusted</b> <on/<b>off</b>> equivalent to: SysSock.ParseTrusted.</li> </ul> <b>Caveats/Known Bugs:</b><br> @@ -295,7 +323,7 @@ $SystemLogSocketAnnotate on <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> diff --git a/doc/manual.html b/doc/manual.html index 05a03e29..a1e39c9d 100644 --- a/doc/manual.html +++ b/doc/manual.html @@ -19,7 +19,7 @@ professional services</a> available directly from the source!</p> <p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the project goals.</p> -<p><b>This documentation is for version 7.3.6 (devel branch) of rsyslog.</b> +<p><b>This documentation is for version 7.3.10 (devel branch) of rsyslog.</b> Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> to obtain current version information and project status. </p><p><b>If you like rsyslog, you might diff --git a/doc/omfile.html b/doc/omfile.html index 2c5ab97a..06b738bb 100644 --- a/doc/omfile.html +++ b/doc/omfile.html @@ -13,14 +13,14 @@ <p>The omfile plug-in provides the core functionality of writing messages to files residing inside the local file system (which may actually be remote if methods like NFS are used). Both files named with static names as well files with names based on message content are supported by this module. It is a built-in module that does not need to be loaded. </p> <p> </p> -<p><b>Module Configuration Parameters</b>:</p> +<p><b>Module Parameters</b>:</p> <ul> <li><strong>Template </strong>[templateName]<br> sets a new default template for file actions.<br></li> </ul> <p> </p> -<p><b>Action Confguration Parameters</b>:</p> +<p><b>Action Parameters</b>:</p> <ul> <li><strong>DynaFileCacheSize </strong>(not mandatory, default will be used)<br> Defines a template to be used for the output. <br></li><br> @@ -83,6 +83,16 @@ <li><strong>DynaFile </strong><br> For each message, the file name is generated based on the given template. Then, this file is opened. As with the ``file'' property, data is appended if the file already exists. If the file does not exist, a new file is created. A cache of recent files is kept. Note that this cache can consume quite some memory (especially if large buffer sizes are used). Files are kept open as long as they stay inside the cache. Currently, files are only evicted from the cache when there is need to do so (due to insufficient cache size). To force-close (and evict) a dynafile from cache, send a HUP signal to rsyslogd. <br></li><br> + <li><b>Sig.Provider </b>[ProviderName]<br> + Selects a signature provider for log signing. Currently, + there only is one provider called + "<a href="sigprov_gt.html">gt</a>".<br></li><br> + + <li><b>Cry.Provider </b>[ProviderName]<br> + Selects a crypto provider for log encryption. Currently, + there only is one provider called + "<a href="cryprov_gcry.html">gcry</a>".<br></li><br> + <li><strong>Template </strong>[templateName]<br> sets a new default template for file actions.<br></li><br> diff --git a/doc/omrelp.html b/doc/omrelp.html index 22e6845f..8858f884 100644 --- a/doc/omrelp.html +++ b/doc/omrelp.html @@ -16,10 +16,34 @@ RELP protocol. For RELP's advantages over plain tcp syslog, please see the documentation for <a href="imrelp.html">imrelp</a> (the server counterpart). </p> <span style="font-weight: bold;">Setup</span> -<p>Please note the <a href="http://www.librelp.com">librelp</a> +<p>Please note that <a href="http://www.librelp.com">librelp</a> is required for imrelp (it provides the core relp protocol implementation).</p> -<p><b>Configuration Directives</b>:</p> +<p><b>Action Configuration Parameters</b>:</p> +<p>This module supports RainerScript configuration starting with +rsyslog 7.3.10. For older versions, legacy configuration directives +must be used. +<ul> + <li><b>target </b>(mandatory)<br> + The target server to connect to. + </li> + <li><b>template </b>(not mandatory, default "RSYSLOG_ForwardFormat")<br> + Defines the template to be used for the output. + </li> + <li><b>timeout </b>(not mandatory, default 90)<br> + Timeout for relp sessions. If set too low, valid sessions + may be considered dead and tried to recover. + </li> +</ul> +<p><b>Sample:</b></p> +<p>The following sample sends all messages to the central server +"centralserv" at port 2514 (note that that server must run imrelp on +port 2514). +</p> +<textarea rows="3" cols="60">module(load="omrelp") +action(type="omrelp" target="centralserv" port="2514") +</textarea> +<p><b>Legacy Configuration Directives</b>:</p> <p>This module uses old-style action configuration to keep consistent with the forwarding rule. So far, no additional configuration directives can be specified. To send a message via RELP, @@ -33,18 +57,15 @@ use</p> <b>Caveats/Known Bugs:</b> <p>See <a href="imrelp.html">imrelp</a>, which documents them. </p> -<p><b>Sample:</b></p> +<p><b>Legacy Sample:</b></p> <p>The following sample sends all messages to the central server "centralserv" at port 2514 (note that that server must run imrelp on -port 2514). Rsyslog's high-precision timestamp format is used, thus the -special "RSYSLOG_ForwardFormat" (case sensitive!) template is used.<br> +port 2514). </p> -<textarea rows="15" cols="60">$ModLoad omrelp -# forward messages to the remote server "myserv" on -# port 2514 -*.* :omrelp:centralserv:2514;RSYSLOG_ForwardFormat +<textarea rows="3" cols="60">$ModLoad omrelp +*.* :omrelp:centralserv:2514 </textarea> -Note: to use IPv6 addresses, encode them in [::1] format. +<p>Note: to use IPv6 addresses, encode them in [::1] format. <p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the diff --git a/doc/property_replacer.html b/doc/property_replacer.html index c7624b2d..13ff41c3 100644 --- a/doc/property_replacer.html +++ b/doc/property_replacer.html @@ -413,6 +413,12 @@ option when forwarding to remote hosts - they may treat the date as invalid <td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td> </tr> <tr> +<td>pos-end-relative</td> + <td>the from and to position is relative to the end of the string + instead of the usual start of string. (available since rsyslog v7.3.10) + </td> +</tr> +<tr> <td><b>ControlCharacters</b></td> <td>Option values for how to process control characters</td> </tr> diff --git a/doc/rainerscript.html b/doc/rainerscript.html index d76316ed..7cbbfa9f 100644 --- a/doc/rainerscript.html +++ b/doc/rainerscript.html @@ -66,6 +66,15 @@ variable, if it exists. Returns an empty string if it does not exist. <li>cstr(expr) - converts expr to a string value <li>cnum(expr) - converts expr to a number (integer) <li>re_match(expr, re) - returns 1, if expr matches re, 0 otherwise +<li>re_extract(expr, re, match, submatch, no-found) - extracts +data from a string (property) via a regular expression match. +POSIX ERE regular expressions are used. The variable "match" contains +the number of the match to use. This permits to pick up more than the +first expression match. Submatch is the submatch to match (max 50 supported). +The "no-found" parameter specifies which string is to be returned in case when +the regular expression is not found. Note that match and submatch start with +zero. It currently is not possible to extract more than one submatch with +a single call. <li>field(str, delim, matchnbr) - returns a field-based substring. str is the string to search, delim is the delimiter and matchnbr is the match to search for (the first match starts at 1). This works similar as the field based diff --git a/doc/rsyslog_conf_actions.html b/doc/rsyslog_conf_actions.html index 0c7705f8..fa240d97 100644 --- a/doc/rsyslog_conf_actions.html +++ b/doc/rsyslog_conf_actions.html @@ -3,13 +3,83 @@ <body> <p>This is a part of the rsyslog.conf documentation.</p> <a href="rsyslog_conf.html">back</a> -<h2>Actions (legacy format)</h2> -<p>The action field of a rule describes what to do with the -message. In general, message content is written to a kind of "logfile". -But also other actions might be done, like writing to a database table -or forwarding to another host.<br> -<br> -Templates can be used with all actions. If used, the specified template +<h2>Actions</h2> +Action object describe what is to be done with a message. They are +implemented via <a href="rsyslog_conf_modules.html#om">outpout modules</a>. +<p>The action object has different parameters: +<ul> +<li>those that apply to all actions and are action specific. These + are documented below. +<li>parameters for the action queue. While they also apply to + all parameters, they are queue-specific, not action-specific (they + are the same that are used in rulesets, for example). +<li>action-specific parameters. These are specific to a certain + type of actions. They are documented by the output module + in question. +</ul> +<h3>General Action Parameters</h3> +<ul> + <li><b>name</b> word + <br>used for statistics gathering and documentation + <li><b>type</b> string + <br>Mandatory parameter for every action. The name of the module that should be used. </li> + <li><b>action.writeAllMarkMessages</b> on/off + <br>Normally, mark messages are written to actions only if the action was not recently executed (by default, recently means within the past 20 minutes). If this setting is switched to "on", mark messages are always sent to actions, no matter how recently they have been executed. In this mode, mark messages can be used as a kind of heartbeat. Note that this option auto-resets to "off", so if you intend to use it with multiple actions, it must be specified in front off all selector lines that should provide this functionality. </li> + <li><b>action.execOnlyEveryNthTime</b> integer + <br>If configured, the next action will only be executed every n-th time. For example, if configured to 3, the first two messages that go into the action will be dropped, the 3rd will actually cause the action to execute, the 4th and 5th will be dropped, the 6th executed under the action, ... and so on. Note: this setting is automatically re-set when the actual action is defined.</li> + <li><b>action.execOnlyEveryNthTimeout</b> integer + <br>Has a meaning only if Action.ExecOnlyEveryNthTime is also configured for the same action. If so, the timeout setting specifies after which period the counting of "previous actions" expires and a new action count is begun. Specify 0 (the default) to disable timeouts. +Why is this option needed? Consider this case: a message comes in at, eg., 10am. That's count 1. Then, nothing happens for the next 10 hours. At 8pm, the next one occurs. That's count 2. Another 5 hours later, the next message occurs, bringing the total count to 3. Thus, this message now triggers the rule. +The question is if this is desired behavior? Or should the rule only be triggered if the messages occur within an e.g. 20 minute window? If the later is the case, you need a +<br>Action.ExecOnlyEveryNthTimeTimeout="1200" +<br>This directive will timeout previous messages seen if they are older than 20 minutes. In the example above, the count would now be always 1 and consequently no rule would ever be triggered. </li> + <li><b>action.execOnlyOnceEveryInterval</b> integer + <br>Execute action only if the last execute is at last <seconds> seconds in the past (more info in ommail, but may be used with any action)</li> + <li><b>action.execOnlyWhenpReviousIsSuspended</b> on/off + <br>This directive allows to specify if actions should always be executed ("off," the default) or only if the previous action is suspended ("on"). This directive works hand-in-hand with the multiple actions per selector feature. It can be used, for example, to create rules that automatically switch destination servers or databases to a (set of) backup(s), if the primary server fails. Note that this feature depends on proper implementation of the suspend feature in the output module. All built-in output modules properly support it (most importantly the database write and the syslog message forwarder).</li> + <li><b>action.repeatedmsgcontainsoriginalmsg</b> on/off + <br>"last message repeated n times" messages, if generated, have a different format that contains the message that is being repeated. Note that only the first "n" characters are included, with n to be at least 80 characters, most probably more (this may change from version to version, thus no specific limit is given). The bottom line is that n is large enough to get a good idea which message was repeated but it is not necessarily large enough for the whole message. (Introduced with 4.1.5). Once set, it affects all following actions.</li> + <li><b>action.resumeRetryCount</b> integer + <br>[default 0, -1 means eternal]</li> + <li><b>action.resumeInterval</b> integer + <br>Sets the ActionResumeInterval for the action. The interval provided is always in seconds. Thus, multiply by 60 if you need minutes and 3,600 if you need hours (not recommended). +When an action is suspended (e.g. destination can not be connected), the action is resumed for the configured interval. Thereafter, it is retried. If multiple retires fail, the interval is automatically extended. This is to prevent excessive ressource use for retires. After each 10 retries, the interval is extended by itself. To be precise, the actual interval is (numRetries / 10 + 1) * Action.ResumeInterval. so after the 10th try, it by default is 60 and after the 100th try it is 330.</li> +</ul> + + +<h2>Legacy Format</h2> +<p><b>Be warned that legacy action format is hard to get right. It is +recommended to use RainerScript-Style action format whenever possible!</b> +A key problem with legacy format is that a single action is defined via +multiple configurations lines, which may be spread all across rsyslog.conf. +Even the definition of multiple actions may be intermixed (often not +intentional!). If legacy actions format needs to be used (e.g. some modules +may not yet implement the RainerScript format), it is strongly recommended +to place all configuration statements pertaining to a single action +closely together. +<p>Please also note that legacy action parameters <b>do not</b> affect +RainerScript action objects. So if you define for example: + +<code><pre> +$actionResumeRetryCount 10 +action(type="omfwd" target="server1.example.net") +@@server2.example.net +</pre></code> + +server1's "action.resumeRetryCount" parameter is <b>not</b> set, instead +server2's is! +<p>A goal of the new RainerScript action format was to avoid confusion +which parameters are actually used. As such, it would be counter-productive +to honor legacy action parameters inside a RainerScript definition. As +result, both types of action definitions are strictly (and nicely) +separated from each other. The bottom line is that if RainerScript actions +are used, one does not need to care about which legacy action parameters may +(still...) be in effect. +<p> +<p>Note that not all modules necessarily support legacy action format. +Especially newer modules are recommended to NOT support it. +<h3>Legacy Description</h3> +<p>Templates can be used with many actions. If used, the specified template is used to generate the message content (instead of the default template). To specify a template, write a semicolon after the action value immediately followed by the template name.<br> @@ -331,13 +401,12 @@ one template name for each given action. The default template is specific to each action. For a description of what a template is and what you can do with it, see "TEMPLATES" at the top of this document.</p> - <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> <p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 2 or higher.</font></p> </body> diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html index fad1b110..00a700d4 100644 --- a/doc/rsyslog_conf_basic_structure.html +++ b/doc/rsyslog_conf_basic_structure.html @@ -49,7 +49,8 @@ after the stop statement are never evaluated. <h3>Data Manipulation Statements</h3> <ul> -<li><b>set</b> - sets a user variable +<li><b>set</b> - <a href="http://www.rsyslog.com/how-to-set-variables-in-rsyslog-v7/">sets</a> +a user variable <li><b>unset</b> - deletes a previously set user variable </ul> @@ -80,6 +81,14 @@ a message comes in via that input, the "program" (ruleset) bound to it will be e (but not any other!). <p>There is detail documentation available for <a href="multi_ruleset">rsyslog rulesets</a>. +<p>For quick reference, rulesets are defined as follows: +<pre> +ruleset(name="rulesetname") { + action(type="omfile" file="/path/to/file") + action(type="..." ...) + /* and so on... */ +} +</pre> <p>[<a href="manual.html">manual index</a>] [<a href="rsyslog_conf.html">rsyslog.conf</a>] diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html index 0c189100..9a6e1619 100644 --- a/doc/rsyslog_conf_templates.html +++ b/doc/rsyslog_conf_templates.html @@ -134,6 +134,8 @@ csv-data is generated, "json", which formats proper json content (but without a header) and "jsonf", which formats as a complete json field. <li>position.from - obtain substring starting from this position (1 is the first position) <li>position.to - obtain substring up to this position +<li>position.relativeToEnd - the from and to position is relative to the end of the string + instead of the usual start of string. (available since rsyslog v7.3.10) <li>field.number - obtain this field match <li>field.delimiter - decimal value of delimiter character for field extraction <li>regex.expression - expression to use diff --git a/doc/sigprov_gt.html b/doc/sigprov_gt.html new file mode 100644 index 00000000..caeee116 --- /dev/null +++ b/doc/sigprov_gt.html @@ -0,0 +1,100 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> +<head> +<meta http-equiv="Content-Language" content="en"> +<title>GuardTime Log Signature Provider (gt)</title> +</head> + +<body> +<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a> + +<h1>GuardTime Log Signature Provider (gt)</h1> +<p><b>Signature Provider Name: gt</b></p> +<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> +<p><b>Supported Since: </b>since 7.3.9 +<p><b>Description</b>:</p> +<p>Provides the ability to sign syslog messages via the +GuardTime signature services. +</p> + +<p><b>Configuration Parameters</b>:</p> +<p>Signature providers are loaded by omfile, when the +provider is selected in its "sig.providerName" parameter. +Parameters for the provider are given in the omfile action instance +line. +<p>This provider creates a signature file with the same base name but +the extension ".gtsig" for each log file (both for fixed-name files +as well as dynafiles). Both files together form a set. So you need to +archive both in order to prove integrity. +<ul> +<li><b>sig.hashFunction</b> <Hash Algorithm><br> +The following hash algorithms are currently supported: + <ul> + <li>SHA1 + <li>RIPEMD-160 + <li>SHA2-224 + <li>SHA2-256 + <li>SHA2-384 + <li>SHA2-512 + </ul> +</li> +<li><b>sig.timestampService</b> <timestamper URL><br> +This provides the URL of the timestamper service. If not selected, +a default server is selected. This may not necessarily be a good +one for your region. +</li> +<li><b>sig.block.sizeLimit</b> <nbr-records><br> +The maximum number of records inside a single signature block. By +default, there is no size limit, so the signature is only written +on file closure. Note that a signature request typically takes between +one and two seconds. So signing to frequently is probably not a good +idea. +</li> +<li><b>sig.keepRecordHashes</b> <on/<b>off</b>><br> +Controls if record hashes are written to the .gtsig file. This +enhances the ability to spot the location of a signature breach, +but costs considerable disk space (65 bytes for each log record +for SHA2-512 hashes, for example). +</li> +<li><b>sig.keepTreeHashes</b> <on/<b>off</b>><br> +Controls if tree (intermediate) hashes are written to the .gtsig file. This +enhances the ability to spot the location of a signature breach, +but costs considerable disk space (a bit mire than the amount +sig.keepRecordHashes requries). Note that both Tree and Record +hashes can be kept inside the signature file. +</li> +</ul> +<b>Caveats/Known Bugs:</b> +<ul> +<li>currently none known +</li> +</ul> +<p><b>Samples:</b></p> +<p>This writes a log file with it's associated signature file. Default +parameters are used. +</p> +<textarea rows="3" cols="60"> +action(type="omfile" file="/var/log/somelog" + sig.provider="gt") +</textarea> + +<p>In the next sample, we use the more secure SHA2-512 hash function, +sign every 10,000 records and Tree and Record hashes are kept. +<textarea rows="3" cols="60"> +action(type="omfile" file="/var/log/somelog" + sig.provider="gt" sig.hashfunction="SHA2-512" + sig.block.sizelimit="10000" + sig.keepTreeHashes="on" sig.keepRecordHashes="on") +</textarea> + + +<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] +[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> +<p><font size="2">This documentation is part of the +<a href="http://www.rsyslog.com/">rsyslog</a> +project.<br> +Copyright © 2013 by +<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and +<a href="http://www.adiscon.com/">Adiscon</a>. +Released under the GNU GPL version 3 or higher.</font></p> +</body></html> diff --git a/grammar/lexer.l b/grammar/lexer.l index e1f5a9c3..237eb2a6 100644 --- a/grammar/lexer.l +++ b/grammar/lexer.l @@ -310,6 +310,7 @@ cnfSetLexFile(char *fname) currbs = bs; cnfcurrfn = bs->fn; yylineno = 1; + dbgprintf("config parser: pushed file %s on top of stack\n", fname); done: if(r != 0) { @@ -337,6 +338,7 @@ popfile(void) * necessary, as otherwise we may provide wrong file name information * at the end of include files as well. -- rgerhards, 2011-07-22 */ + dbgprintf("config parser: reached end of file %s\n", bs->fn); yy_delete_buffer(bs->bs); if(bs->prev != NULL) free(bs->fn); @@ -346,12 +348,16 @@ popfile(void) currbs = bs->prev; free(bs); - if(currbs == NULL) + if(currbs == NULL) { + dbgprintf("config parser: parsing completed\n"); return 1; /* all processed */ + } yy_switch_to_buffer(currbs->bs); yylineno = currbs->lineno; cnfcurrfn = currbs->fn; + dbgprintf("config parser: resume parsing of file %s at line %d\n", + cnfcurrfn, yylineno); return 0; } diff --git a/grammar/rainerscript.c b/grammar/rainerscript.c index 6dbdad63..89cf946c 100644 --- a/grammar/rainerscript.c +++ b/grammar/rainerscript.c @@ -2,7 +2,7 @@ * * Module begun 2011-07-01 by Rainer Gerhards * - * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -1228,6 +1228,96 @@ finalize_it: RETiRet; } +static inline void +doFunc_re_extract(struct cnffunc *func, struct var *ret, void* usrptr) +{ + size_t submatchnbr; + short matchnbr; + regmatch_t pmatch[50]; + int bMustFree; + es_str_t *estr; + char *str; + struct var r[CNFFUNC_MAX_ARGS]; + int iLenBuf; + unsigned iOffs; + short iTry = 0; + uchar bFound = 0; + iOffs = 0; + sbool bHadNoMatch = 0; + + cnfexprEval(func->expr[0], &r[0], usrptr); + /* search string is already part of the compiled regex, so we don't + * need it here! + */ + cnfexprEval(func->expr[2], &r[2], usrptr); + cnfexprEval(func->expr[3], &r[3], usrptr); + str = (char*) var2CString(&r[0], &bMustFree); + matchnbr = (short) var2Number(&r[2], NULL); + submatchnbr = (size_t) var2Number(&r[3], NULL); + if(submatchnbr > sizeof(pmatch)/sizeof(regmatch_t)) { + DBGPRINTF("re_extract() submatch %d is too large\n", submatchnbr); + bHadNoMatch = 1; + goto finalize_it; + } + + /* first see if we find a match, iterating through the series of + * potential matches over the string. + */ + while(!bFound) { + int iREstat; + iREstat = regexp.regexec(func->funcdata, (char*)(str + iOffs), + submatchnbr+1, pmatch, 0); + dbgprintf("re_extract: regexec return is %d\n", iREstat); + if(iREstat == 0) { + if(pmatch[0].rm_so == -1) { + dbgprintf("oops ... start offset of successful regexec is -1\n"); + break; + } + if(iTry == matchnbr) { + bFound = 1; + } else { + dbgprintf("re_extract: regex found at offset %d, new offset %d, tries %d\n", + iOffs, (int) (iOffs + pmatch[0].rm_eo), iTry); + iOffs += pmatch[0].rm_eo; + ++iTry; + } + } else { + break; + } + } + dbgprintf("re_extract: regex: end search, found %d\n", bFound); + if(!bFound) { + bHadNoMatch = 1; + goto finalize_it; + } else { + /* Match- but did it match the one we wanted? */ + /* we got no match! */ + if(pmatch[submatchnbr].rm_so == -1) { + bHadNoMatch = 1; + goto finalize_it; + } + /* OK, we have a usable match - we now need to malloc pB */ + iLenBuf = pmatch[submatchnbr].rm_eo - pmatch[submatchnbr].rm_so; + estr = es_newStrFromBuf(str + iOffs + pmatch[submatchnbr].rm_so, + iLenBuf); + } + + if(bMustFree) free(str); + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + if(r[2].datatype == 'S') es_deleteStr(r[2].d.estr); + if(r[3].datatype == 'S') es_deleteStr(r[3].d.estr); +finalize_it: + if(bHadNoMatch) { + cnfexprEval(func->expr[4], &r[4], usrptr); + estr = var2String(&r[4], &bMustFree); + if(r[4].datatype == 'S') es_deleteStr(r[4].d.estr); + } + ret->datatype = 'S'; + ret->d.estr = estr; + return; +} + + /* Perform a function call. This has been moved out of cnfExprEval in order * to keep the code small and easier to maintain. */ @@ -1273,8 +1363,12 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) estr = var2String(&r[0], &bMustFree); str = (char*) es_str2cstr(estr, NULL); envvar = getenv(str); + if(envvar == NULL) { + ret->d.estr = es_newStr(0); + } else { + ret->d.estr = es_newStrFromCStr(envvar, strlen(envvar)); + } ret->datatype = 'S'; - ret->d.estr = es_newStrFromCStr(envvar, strlen(envvar)); if(bMustFree) es_deleteStr(estr); if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); free(str); @@ -1327,6 +1421,9 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) if(bMustFree) free(str); if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); break; + case CNFFUNC_RE_EXTRACT: + doFunc_re_extract(func, ret, usrptr); + break; case CNFFUNC_FIELD: cnfexprEval(func->expr[0], &r[0], usrptr); cnfexprEval(func->expr[1], &r[1], usrptr); @@ -1904,6 +2001,7 @@ cnffuncDestruct(struct cnffunc *func) /* some functions require special destruction */ switch(func->fID) { case CNFFUNC_RE_MATCH: + case CNFFUNC_RE_EXTRACT: if(func->funcdata != NULL) regexp.regfree(func->funcdata); break; @@ -2154,31 +2252,33 @@ cnfexprPrint(struct cnfexpr *expr, int indent) break; } } +/* print only the given stmt + * if "subtree" equals 1, the full statement subtree is printed, else + * really only the statement. + */ void -cnfstmtPrint(struct cnfstmt *root, int indent) +cnfstmtPrintOnly(struct cnfstmt *stmt, int indent, sbool subtree) { - struct cnfstmt *stmt; char *cstr; - //dbgprintf("stmt %p, indent %d, type '%c'\n", expr, indent, expr->nodetype); - for(stmt = root ; stmt != NULL ; stmt = stmt->next) { - switch(stmt->nodetype) { - case S_NOP: - doIndent(indent); dbgprintf("NOP\n"); - break; - case S_STOP: - doIndent(indent); dbgprintf("STOP\n"); - break; - case S_CALL: - cstr = es_str2cstr(stmt->d.s_call.name, NULL); - doIndent(indent); dbgprintf("CALL [%s]\n", cstr); - free(cstr); - break; - case S_ACT: - doIndent(indent); dbgprintf("ACTION %p [%s]\n", stmt->d.act, stmt->printable); - break; - case S_IF: - doIndent(indent); dbgprintf("IF\n"); - cnfexprPrint(stmt->d.s_if.expr, indent+1); + switch(stmt->nodetype) { + case S_NOP: + doIndent(indent); dbgprintf("NOP\n"); + break; + case S_STOP: + doIndent(indent); dbgprintf("STOP\n"); + break; + case S_CALL: + cstr = es_str2cstr(stmt->d.s_call.name, NULL); + doIndent(indent); dbgprintf("CALL [%s]\n", cstr); + free(cstr); + break; + case S_ACT: + doIndent(indent); dbgprintf("ACTION %p [%s]\n", stmt->d.act, stmt->printable); + break; + case S_IF: + doIndent(indent); dbgprintf("IF\n"); + cnfexprPrint(stmt->d.s_if.expr, indent+1); + if(subtree) { doIndent(indent); dbgprintf("THEN\n"); cnfstmtPrint(stmt->d.s_if.t_then, indent+1); if(stmt->d.s_if.t_else != NULL) { @@ -2186,54 +2286,67 @@ cnfstmtPrint(struct cnfstmt *root, int indent) cnfstmtPrint(stmt->d.s_if.t_else, indent+1); } doIndent(indent); dbgprintf("END IF\n"); - break; - case S_SET: - doIndent(indent); dbgprintf("SET %s =\n", - stmt->d.s_set.varname); - cnfexprPrint(stmt->d.s_set.expr, indent+1); - doIndent(indent); dbgprintf("END SET\n"); - break; - case S_UNSET: - doIndent(indent); dbgprintf("UNSET %s\n", - stmt->d.s_unset.varname); - break; - case S_PRIFILT: - doIndent(indent); dbgprintf("PRIFILT '%s'\n", stmt->printable); - pmaskPrint(stmt->d.s_prifilt.pmask, indent); + } + break; + case S_SET: + doIndent(indent); dbgprintf("SET %s =\n", + stmt->d.s_set.varname); + cnfexprPrint(stmt->d.s_set.expr, indent+1); + doIndent(indent); dbgprintf("END SET\n"); + break; + case S_UNSET: + doIndent(indent); dbgprintf("UNSET %s\n", + stmt->d.s_unset.varname); + break; + case S_PRIFILT: + doIndent(indent); dbgprintf("PRIFILT '%s'\n", stmt->printable); + pmaskPrint(stmt->d.s_prifilt.pmask, indent); + if(subtree) { cnfstmtPrint(stmt->d.s_prifilt.t_then, indent+1); if(stmt->d.s_prifilt.t_else != NULL) { doIndent(indent); dbgprintf("ELSE\n"); cnfstmtPrint(stmt->d.s_prifilt.t_else, indent+1); } doIndent(indent); dbgprintf("END PRIFILT\n"); - break; - case S_PROPFILT: - doIndent(indent); dbgprintf("PROPFILT\n"); - doIndent(indent); dbgprintf("\tProperty.: '%s'\n", - propIDToName(stmt->d.s_propfilt.propID)); - if(stmt->d.s_propfilt.propName != NULL) { - cstr = es_str2cstr(stmt->d.s_propfilt.propName, NULL); - doIndent(indent); - dbgprintf("\tCEE-Prop.: '%s'\n", cstr); - free(cstr); - } - doIndent(indent); dbgprintf("\tOperation: "); - if(stmt->d.s_propfilt.isNegated) - dbgprintf("NOT "); - dbgprintf("'%s'\n", getFIOPName(stmt->d.s_propfilt.operation)); - if(stmt->d.s_propfilt.pCSCompValue != NULL) { - doIndent(indent); dbgprintf("\tValue....: '%s'\n", - rsCStrGetSzStrNoNULL(stmt->d.s_propfilt.pCSCompValue)); - } + } + break; + case S_PROPFILT: + doIndent(indent); dbgprintf("PROPFILT\n"); + doIndent(indent); dbgprintf("\tProperty.: '%s'\n", + propIDToName(stmt->d.s_propfilt.propID)); + if(stmt->d.s_propfilt.propName != NULL) { + cstr = es_str2cstr(stmt->d.s_propfilt.propName, NULL); + doIndent(indent); + dbgprintf("\tCEE-Prop.: '%s'\n", cstr); + free(cstr); + } + doIndent(indent); dbgprintf("\tOperation: "); + if(stmt->d.s_propfilt.isNegated) + dbgprintf("NOT "); + dbgprintf("'%s'\n", getFIOPName(stmt->d.s_propfilt.operation)); + if(stmt->d.s_propfilt.pCSCompValue != NULL) { + doIndent(indent); dbgprintf("\tValue....: '%s'\n", + rsCStrGetSzStrNoNULL(stmt->d.s_propfilt.pCSCompValue)); + } + if(subtree) { doIndent(indent); dbgprintf("THEN\n"); cnfstmtPrint(stmt->d.s_propfilt.t_then, indent+1); doIndent(indent); dbgprintf("END PROPFILT\n"); - break; - default: - dbgprintf("error: unknown stmt type %u\n", - (unsigned) stmt->nodetype); - break; } + break; + default: + dbgprintf("error: unknown stmt type %u\n", + (unsigned) stmt->nodetype); + break; + } +} +void +cnfstmtPrint(struct cnfstmt *root, int indent) +{ + struct cnfstmt *stmt; + //dbgprintf("stmt %p, indent %d, type '%c'\n", expr, indent, expr->nodetype); + for(stmt = root ; stmt != NULL ; stmt = stmt->next) { + cnfstmtPrintOnly(stmt, indent, 1); } } @@ -2635,7 +2748,7 @@ cnfexprOptimize_CMP_severity_facility(struct cnfexpr *expr) /* optimize a comparison with a variable as left-hand operand * NOTE: Currently support CMP_EQ, CMP_NE only and code NEEDS - * TO BE CHANGED for other comparisons! + * TO BE CHANGED fgr other comparisons! */ static inline struct cnfexpr* cnfexprOptimize_CMP_var(struct cnfexpr *expr) @@ -2790,10 +2903,10 @@ cnfexprOptimize(struct cnfexpr *expr) expr->l = expr->r; expr->r = exprswap; } - } else if(expr->l->nodetype == 'V') { - expr = cnfexprOptimize_CMP_var(expr); } - if(expr->r->nodetype == 'A') { + if(expr->l->nodetype == 'V') { + expr = cnfexprOptimize_CMP_var(expr); + } else if(expr->r->nodetype == 'A') { cnfexprOptimize_CMPEQ_arr((struct cnfarray *)expr->r); } break; @@ -2984,7 +3097,6 @@ cnfstmtOptimize(struct cnfstmt *root) struct cnfstmt *stmt; if(root == NULL) goto done; for(stmt = root ; stmt != NULL ; stmt = stmt->next) { -dbgprintf("RRRR: stmtOptimize: stmt %p, nodetype %u\n", stmt, stmt->nodetype); switch(stmt->nodetype) { case S_IF: cnfstmtOptimizeIf(stmt); @@ -3084,6 +3196,13 @@ funcName2ID(es_str_t *fname, unsigned short nParams) return CNFFUNC_INVALID; } return CNFFUNC_RE_MATCH; + } else if(!es_strbufcmp(fname, (unsigned char*)"re_extract", sizeof("re_extract") - 1)) { + if(nParams != 5) { + parser_errmsg("number of parameters for re_extract() must be five " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_RE_EXTRACT; } else if(!es_strbufcmp(fname, (unsigned char*)"field", sizeof("field") - 1)) { if(nParams != 3) { parser_errmsg("number of parameters for field() must be three " @@ -3114,7 +3233,7 @@ initFunc_re_match(struct cnffunc *func) func->funcdata = NULL; if(func->expr[1]->nodetype != 'S') { - parser_errmsg("param 2 of re_match() must be a constant string"); + parser_errmsg("param 2 of re_match/extract() must be a constant string"); FINALIZE; } @@ -3192,6 +3311,7 @@ cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) /* some functions require special initialization */ switch(func->fID) { case CNFFUNC_RE_MATCH: + case CNFFUNC_RE_EXTRACT: /* need to compile the regexp in param 2, so this MUST be a constant */ initFunc_re_match(func); break; @@ -3233,7 +3353,7 @@ cnfDoInclude(char *name) { char *cfgFile; char *finalName; - unsigned i; + int i; int result; glob_t cfgFiles; struct stat fileInfo; @@ -3252,12 +3372,16 @@ cnfDoInclude(char *name) /* Use GLOB_MARK to append a trailing slash for directories. */ /* Use GLOB_NOMAGIC to detect wildcards that match nothing. */ - result = glob(finalName, GLOB_MARK | GLOB_NOMAGIC, NULL, &cfgFiles); - +#ifdef HAVE_GLOB_NOMAGIC /* Silently ignore wildcards that match nothing */ + result = glob(finalName, GLOB_MARK | GLOB_NOMAGIC, NULL, &cfgFiles); if(result == GLOB_NOMATCH) { - return 0; - } +#else + result = glob(finalName, GLOB_MARK, NULL, &cfgFiles); + if(result == GLOB_NOMATCH && containsGlobWildcard(finalName)) { +#endif /* HAVE_GLOB_NOMAGIC */ + return 0; + } if(result == GLOB_NOSPACE || result == GLOB_ABORTED) { char errStr[1024]; @@ -3269,7 +3393,12 @@ cnfDoInclude(char *name) return 1; } - for(i = 0; i < cfgFiles.gl_pathc; i++) { + /* note: bison "stacks" the files, so we need to submit them + * in reverse order to the *stack* in order to get the proper + * parsing order. Also see + * http://bugzilla.adiscon.com/show_bug.cgi?id=411 + */ + for(i = cfgFiles.gl_pathc - 1; i >= 0 ; i--) { cfgFile = cfgFiles.gl_pathv[i]; if(stat(cfgFile, &fileInfo) != 0) { char errStr[1024]; @@ -3507,3 +3636,52 @@ unescapeStr(uchar *s, int len) s[iDst] = '\0'; } } + +char * +tokenval2str(int tok) +{ + if(tok < 256) return ""; + switch(tok) { + case NAME: return "NAME"; + case FUNC: return "FUNC"; + case BEGINOBJ: return "BEGINOBJ"; + case ENDOBJ: return "ENDOBJ"; + case BEGIN_ACTION: return "BEGIN_ACTION"; + case BEGIN_PROPERTY: return "BEGIN_PROPERTY"; + case BEGIN_CONSTANT: return "BEGIN_CONSTANT"; + case BEGIN_TPL: return "BEGIN_TPL"; + case BEGIN_RULESET: return "BEGIN_RULESET"; + case STOP: return "STOP"; + case SET: return "SET"; + case UNSET: return "UNSET"; + case CONTINUE: return "CONTINUE"; + case CALL: return "CALL"; + case LEGACY_ACTION: return "LEGACY_ACTION"; + case LEGACY_RULESET: return "LEGACY_RULESET"; + case PRIFILT: return "PRIFILT"; + case PROPFILT: return "PROPFILT"; + case BSD_TAG_SELECTOR: return "BSD_TAG_SELECTOR"; + case BSD_HOST_SELECTOR: return "BSD_HOST_SELECTOR"; + case IF: return "IF"; + case THEN: return "THEN"; + case ELSE: return "ELSE"; + case OR: return "OR"; + case AND: return "AND"; + case NOT: return "NOT"; + case VAR: return "VAR"; + case STRING: return "STRING"; + case NUMBER: return "NUMBER"; + case CMP_EQ: return "CMP_EQ"; + case CMP_NE: return "CMP_NE"; + case CMP_LE: return "CMP_LE"; + case CMP_GE: return "CMP_GE"; + case CMP_LT: return "CMP_LT"; + case CMP_GT: return "CMP_GT"; + case CMP_CONTAINS: return "CMP_CONTAINS"; + case CMP_CONTAINSI: return "CMP_CONTAINSI"; + case CMP_STARTSWITH: return "CMP_STARTSWITH"; + case CMP_STARTSWITHI: return "CMP_STARTSWITHI"; + case UMINUS: return "UMINUS"; + default: return "UNKNOWN TOKEN"; + } +} diff --git a/grammar/rainerscript.h b/grammar/rainerscript.h index 59ce53f3..31b2eb93 100644 --- a/grammar/rainerscript.h +++ b/grammar/rainerscript.h @@ -226,6 +226,7 @@ enum cnffuncid { CNFFUNC_CSTR, CNFFUNC_CNUM, CNFFUNC_RE_MATCH, + CNFFUNC_RE_EXTRACT, CNFFUNC_FIELD, CNFFUNC_PRIFILT }; @@ -317,6 +318,7 @@ int cnfparamvalsIsSet(struct cnfparamblk *params, struct cnfparamvals *vals); void varDelete(struct var *v); void cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk); struct cnfstmt * cnfstmtNew(unsigned s_type); +void cnfstmtPrintOnly(struct cnfstmt *stmt, int indent, sbool subtree); void cnfstmtPrint(struct cnfstmt *stmt, int indent); struct cnfstmt* scriptAddStmt(struct cnfstmt *root, struct cnfstmt *s); struct objlst* objlstAdd(struct objlst *root, struct cnfobj *o); @@ -338,6 +340,7 @@ void cnfarrayContentDestruct(struct cnfarray *ar); char* getFIOPName(unsigned iFIOP); rsRetVal initRainerscript(void); void unescapeStr(uchar *s, int len); +char * tokenval2str(int tok); /* debug helper */ void cstrPrint(char *text, es_str_t *estr); diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index 0f155c10..349acead 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -235,6 +235,7 @@ openFile(fileInfo_t *pThis) /* read back in the object */ CHKiRet(obj.Deserialize(&pThis->pStrm, (uchar*) "strm", psSF, NULL, pThis)); + strm.CheckFileChange(pThis->pStrm); CHKiRet(strm.SeekCurrOffs(pThis->pStrm)); /* note: we do not delete the state file, so that the last position remains @@ -738,12 +739,20 @@ persistStrmState(fileInfo_t *pInfo) CHKiRet(strm.ConstructFinalize(psSF)); CHKiRet(strm.Serialize(pInfo->pStrm, psSF)); + CHKiRet(strm.Flush(psSF)); CHKiRet(strm.Destruct(&psSF)); finalize_it: if(psSF != NULL) strm.Destruct(&psSF); + + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "imfile: could not persist state " + "file %s - data may be repeated on next " + "startup. Is WorkDirectory set?", + pInfo->pszStateFile); + } RETiRet; } diff --git a/plugins/imkmsg/kmsg.c b/plugins/imkmsg/kmsg.c index b771d68a..f1815f25 100644 --- a/plugins/imkmsg/kmsg.c +++ b/plugins/imkmsg/kmsg.c @@ -89,6 +89,10 @@ submitSyslog(uchar *buf) for (; isdigit(*buf); buf++) { timestamp += (timestamp * 10) + (*buf - '0'); } + + while (*buf != ';') { + buf++; /* skip everything till the first ; */ + } buf++; /* skip ; */ /* get message */ @@ -103,7 +107,7 @@ submitSyslog(uchar *buf) if (*buf != '\0') /* message has appended properties, skip \n */ buf++; - while (strlen((char *)buf)) { + while (*buf) { /* get name of the property */ buf++; /* skip ' ' */ offs = 0; @@ -174,18 +178,22 @@ static void readkmsg(void) { int i; - uchar pRcv[8096+1]; + uchar pRcv[8192+1]; char errmsg[2048]; for (;;) { dbgprintf("imkmsg waiting for kernel log line\n"); /* every read() from the opened device node receives one record of the printk buffer */ - i = read(fklog, pRcv, 8096); + i = read(fklog, pRcv, 8192); if (i > 0) { /* successful read of message of nonzero length */ pRcv[i] = '\0'; + } else if (i == -EPIPE) { + imkmsgLogIntMsg(LOG_WARNING, + "imkmsg: some messages in circular buffer got overwritten"); + continue; } else { /* something went wrong - error or zero length message */ if (i < 0 && errno != EINTR && errno != EAGAIN) { diff --git a/plugins/imrelp/imrelp.c b/plugins/imrelp/imrelp.c index dc67f4fe..5e0ae552 100644 --- a/plugins/imrelp/imrelp.c +++ b/plugins/imrelp/imrelp.c @@ -208,6 +208,7 @@ addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) if(pRelpEngine == NULL) { CHKiRet(relpEngineConstruct(&pRelpEngine)); CHKiRet(relpEngineSetDbgprint(pRelpEngine, dbgprintf)); + CHKiRet(relpEngineSetFamily(pRelpEngine, glbl.GetDefPFFamily())); CHKiRet(relpEngineSetEnableCmd(pRelpEngine, (uchar*) "syslog", eRelpCmdState_Required)); CHKiRet(relpEngineSetSyslogRcv(pRelpEngine, onSyslogRcv)); if (!glbl.GetDisableDNS()) { diff --git a/plugins/imudp/imudp.c b/plugins/imudp/imudp.c index dde8f105..7bf1473a 100644 --- a/plugins/imudp/imudp.c +++ b/plugins/imudp/imudp.c @@ -75,6 +75,7 @@ static struct lstn_s { struct lstn_s *next; int sock; /* socket */ ruleset_t *pRuleset; /* bound ruleset */ + prop_t *pInputName; statsobj_t *stats; /* listener stats */ ratelimit_t *ratelimiter; STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) @@ -91,7 +92,6 @@ static uchar *pRcvBuf = NULL; /* receive buffer (for a single packet). We use a * it so that we can check available memory in willRun() and request * termination if we can not get it. -- rgerhards, 2007-12-27 */ -static prop_t *pInputName = NULL; /* our inputName currently is always "imudp", and this will hold it */ #define TIME_REQUERY_DFLT 2 #define SCHED_PRIO_UNSET -12345678 /* a value that indicates that the scheduling priority has not been set */ @@ -108,10 +108,12 @@ struct instanceConf_s { uchar *pszBindAddr; /* IP to bind socket to */ uchar *pszBindPort; /* Port to bind socket to */ uchar *pszBindRuleset; /* name of ruleset to bind to */ + uchar *inputname; ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ int ratelimitInterval; int ratelimitBurst; struct instanceConf_s *next; + sbool bAppendPortToInpname; }; struct modConfData_s { @@ -141,6 +143,8 @@ static struct cnfparamblk modpblk = /* input instance parameters */ static struct cnfparamdescr inppdescr[] = { { "port", eCmdHdlrArray, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */ + { "inputname", eCmdHdlrGetWord, 0 }, + { "inputname.appendport", eCmdHdlrBinary, 0 }, { "address", eCmdHdlrString, 0 }, { "ruleset", eCmdHdlrString, 0 }, { "ratelimit.interval", eCmdHdlrInt, 0 }, @@ -169,6 +173,8 @@ createInstance(instanceConf_t **pinst) inst->pszBindPort = NULL; inst->pszBindAddr = NULL; inst->pszBindRuleset = NULL; + inst->inputname = NULL; + inst->bAppendPortToInpname = 0; inst->ratelimitBurst = 10000; /* arbitrary high limit */ inst->ratelimitInterval = 0; /* off */ @@ -229,7 +235,8 @@ addListner(instanceConf_t *inst) struct lstn_s *newlcnfinfo; uchar *bindName; uchar *port; - uchar dispname[64]; + uchar dispname[64], inpnameBuf[128]; + uchar *inputname; /* check which address to bind to. We could do this more compact, but have not * done so in order to make the code more readable. -- rgerhards, 2007-12-27 @@ -257,6 +264,21 @@ addListner(instanceConf_t *inst) snprintf((char*)dispname, sizeof(dispname), "imudp(%s:%s)", bindName, port); dispname[sizeof(dispname)-1] = '\0'; /* just to be on the save side... */ CHKiRet(ratelimitNew(&newlcnfinfo->ratelimiter, (char*)dispname, NULL)); + if(inst->inputname == NULL) { + inputname = (uchar*)"imudp"; + } else { + inputname = inst->inputname; + } + if(inst->bAppendPortToInpname) { + snprintf((char*)inpnameBuf, sizeof(inpnameBuf), "%s%s", + inputname, port); + inpnameBuf[sizeof(inpnameBuf)-1] = '\0'; + inputname = inpnameBuf; + } + CHKiRet(prop.Construct(&newlcnfinfo->pInputName)); + CHKiRet(prop.SetString(newlcnfinfo->pInputName, + inputname, ustrlen(inputname))); + CHKiRet(prop.ConstructFinalize(newlcnfinfo->pInputName)); ratelimitSetLinuxLike(newlcnfinfo->ratelimiter, inst->ratelimitInterval, inst->ratelimitBurst); /* support statistics gathering */ @@ -390,7 +412,7 @@ processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *f /* we now create our own message object and submit it to the queue */ CHKiRet(msgConstructWithTime(&pMsg, &stTime, ttGenTime)); MsgSetRawMsg(pMsg, (char*)pRcvBuf, lenRcvBuf); - MsgSetInputName(pMsg, pInputName); + MsgSetInputName(pMsg, lstn->pInputName); MsgSetRuleset(pMsg, lstn->pRuleset); MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY); pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME | NEEDS_DNSRESOL; @@ -695,6 +717,10 @@ createListner(es_str_t *port, struct cnfparamvals *pvals) continue; if(!strcmp(inppblk.descr[i].name, "port")) { continue; /* array, handled by caller */ + } else if(!strcmp(inppblk.descr[i].name, "inputname")) { + inst->inputname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "inputname.appendport")) { + inst->bAppendPortToInpname = (int) pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "address")) { inst->pszBindAddr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { @@ -853,7 +879,6 @@ CODESTARTactivateCnfPrePrivDrop ABORT_FINALIZE(RS_RET_NO_RUN); } - setSchedParams(pModConf); finalize_it: ENDactivateCnfPrePrivDrop @@ -874,6 +899,7 @@ CODESTARTfreeCnf free(inst->pszBindPort); free(inst->pszBindAddr); free(inst->pBindRuleset); + free(inst->inputname); del = inst; inst = inst->next; free(del); @@ -886,6 +912,15 @@ ENDfreeCnf */ BEGINrunInput CODESTARTrunInput + /* Note well: the setting of scheduling parameters will not work + * when we dropped privileges (if the user is not sufficently + * privileged, of course). Howerver, we can't change the + * scheduling params in PrePrivDrop(), as at that point our thread + * is not yet created. So at least as an interim solution, we do + * NOT support both setting sched parameters and dropping + * privileges within the same instance. + */ + setSchedParams(runModConf); iRet = rcvMainLoop(pThrd); ENDrunInput @@ -907,6 +942,7 @@ CODESTARTafterRun statsobj.Destruct(&(lstn->stats)); ratelimitDestruct(lstn->ratelimiter); close(lstn->sock); + prop.Destruct(&lstn->pInputName); lstnDel = lstn; lstn = lstn->next; free(lstnDel); @@ -921,9 +957,6 @@ ENDafterRun BEGINmodExit CODESTARTmodExit - if(pInputName != NULL) - prop.Destruct(&pInputName); - /* release what we no longer need */ objRelease(errmsg, CORE_COMPONENT); objRelease(glbl, CORE_COMPONENT); @@ -978,11 +1011,6 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(ruleset, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&pInputName)); - CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imudp"), sizeof("imudp") - 1)); - CHKiRet(prop.ConstructFinalize(pInputName)); - /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputudpserverbindruleset", 0, eCmdHdlrGetWord, NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index a5360bee..c503852c 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -146,6 +146,7 @@ typedef struct lstn_s { sbool bWritePid; /* write original PID into tag */ sbool bDiscardOwnMsgs; /* discard messages that originated from ourselves */ sbool bUseSysTimeStamp; /* use timestamp from system (instead of from message) */ + sbool bUnlink; /* unlink&re-create socket at start and end of processing */ } lstn_t; static lstn_t listeners[MAXFUNIX]; @@ -201,6 +202,7 @@ struct instanceConf_s { int bAnnotate; /* annotate trusted properties */ int bParseTrusted; /* parse trusted properties */ sbool bDiscardOwnMsgs; /* discard messages that originated from our own pid? */ + sbool bUnlink; struct instanceConf_s *next; }; @@ -220,6 +222,7 @@ struct modConfData_s { sbool bUseSysTimeStamp; sbool bDiscardOwnMsgs; sbool configSetViaV2Method; + sbool bUnlink; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ @@ -228,11 +231,13 @@ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current lo static struct cnfparamdescr modpdescr[] = { { "syssock.use", eCmdHdlrBinary, 0 }, { "syssock.name", eCmdHdlrGetWord, 0 }, + { "syssock.unlink", eCmdHdlrBinary, 0 }, { "syssock.ignoretimestamp", eCmdHdlrBinary, 0 }, { "syssock.ignoreownmessages", eCmdHdlrBinary, 0 }, { "syssock.flowcontrol", eCmdHdlrBinary, 0 }, { "syssock.usesystimestamp", eCmdHdlrBinary, 0 }, { "syssock.annotate", eCmdHdlrBinary, 0 }, + { "syssock.parsetrusted", eCmdHdlrBinary, 0 }, { "syssock.usepidfromsystem", eCmdHdlrBinary, 0 }, { "syssock.ratelimit.interval", eCmdHdlrInt, 0 }, { "syssock.ratelimit.burst", eCmdHdlrInt, 0 }, @@ -247,6 +252,7 @@ static struct cnfparamblk modpblk = /* input instance parameters */ static struct cnfparamdescr inppdescr[] = { { "socket", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: addunixlistensocket */ + { "unlink", eCmdHdlrBinary, 0 }, { "createpath", eCmdHdlrBinary, 0 }, { "parsetrusted", eCmdHdlrBinary, 0 }, { "ignoreownmessages", eCmdHdlrBinary, 0 }, @@ -284,7 +290,7 @@ createInstance(instanceConf_t **pinst) inst->sockName = NULL; inst->pLogHostName = NULL; inst->ratelimitInterval = DFLT_ratelimitInterval; - inst->ratelimitBurst = DFLT_ratelimitSeverity; + inst->ratelimitBurst = DFLT_ratelimitBurst; inst->ratelimitSeverity = DFLT_ratelimitSeverity; inst->bUseFlowCtl = 0; inst->bIgnoreTimestamp = 1; @@ -294,6 +300,7 @@ createInstance(instanceConf_t **pinst) inst->bAnnotate = 0; inst->bParseTrusted = 0; inst->bDiscardOwnMsgs = 1; + inst->bUnlink = 1; inst->next = NULL; /* node created, let's add to config */ @@ -398,12 +405,15 @@ addListner(instanceConf_t *inst) listeners[nfd].bAnnotate = inst->bAnnotate; listeners[nfd].bParseTrusted = inst->bParseTrusted; listeners[nfd].bDiscardOwnMsgs = inst->bDiscardOwnMsgs; + listeners[nfd].bUnlink = inst->bUnlink; listeners[nfd].bWritePid = inst->bWritePid; listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp; CHKiRet(ratelimitNew(&listeners[nfd].dflt_ratelimiter, "imuxsock", NULL)); ratelimitSetLinuxLike(listeners[nfd].dflt_ratelimiter, listeners[nfd].ratelimitInterval, listeners[nfd].ratelimitBurst); + ratelimitSetSeverity(listeners[nfd].dflt_ratelimiter, + listeners[nfd].ratelimitSev); nfd++; } else { errmsg.LogError(0, NO_ERRCODE, "Out of unix socket name descriptors, ignoring %s\n", @@ -448,7 +458,8 @@ createLogSocket(lstn_t *pLstn) struct sockaddr_un sunx; DEFiRet; - unlink((char*)pLstn->sockName); + if(pLstn->bUnlink) + unlink((char*)pLstn->sockName); memset(&sunx, 0, sizeof(sunx)); sunx.sun_family = AF_UNIX; if(pLstn->bCreatePath) { @@ -561,6 +572,10 @@ findRatelimiter(lstn_t *pLstn, struct ucred *cred, ratelimit_t **prl) FINALIZE; } #endif + if(pLstn->ht == NULL) { + *prl = NULL; + FINALIZE; + } rl = hashtable_search(pLstn->ht, &cred->pid); if(rl == NULL) { @@ -573,6 +588,7 @@ findRatelimiter(lstn_t *pLstn, struct ucred *cred, ratelimit_t **prl) pidbuf[sizeof(pidbuf)-1] = '\0'; /* to be on safe side */ CHKiRet(ratelimitNew(&rl, "imuxsock", pidbuf)); ratelimitSetLinuxLike(rl, pLstn->ratelimitInterval, pLstn->ratelimitBurst); + ratelimitSetSeverity(rl, pLstn->ratelimitSev); CHKmalloc(keybuf = malloc(sizeof(pid_t))); *keybuf = cred->pid; r = hashtable_insert(pLstn->ht, keybuf, rl); @@ -762,10 +778,7 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim facil = LOG_FAC(pri); sever = LOG_PRI(pri); - if(sever >= pLstn->ratelimitSev) { - /* note: if cred == NULL, then ratelimiter == NULL as well! */ - findRatelimiter(pLstn, cred, &ratelimiter); /* ignore error, better so than others... */ - } + findRatelimiter(pLstn, cred, &ratelimiter); /* ignore error, better so than others... */ if(ts == NULL) { datetime.getCurrTime(&st, &tt); @@ -1054,6 +1067,7 @@ activateListeners() listeners[0].bAnnotate = runModConf->bAnnotateSysSock; listeners[0].bParseTrusted = runModConf->bParseTrusted; listeners[0].bDiscardOwnMsgs = runModConf->bDiscardOwnMsgs; + listeners[0].bUnlink = runModConf->bUnlink; listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp; listeners[0].flags = runModConf->bIgnoreTimestamp ? IGNDATE : NOFLAG; listeners[0].flowCtl = runModConf->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; @@ -1061,6 +1075,7 @@ activateListeners() ratelimitSetLinuxLike(listeners[0].dflt_ratelimiter, listeners[0].ratelimitInterval, listeners[0].ratelimitBurst); + ratelimitSetSeverity(listeners[0].dflt_ratelimiter,listeners[0].ratelimitSev); sd_fds = sd_listen_fds(0); if(sd_fds < 0) { @@ -1103,6 +1118,7 @@ CODESTARTbeginCnfLoad pModConf->bAnnotateSysSock = 0; pModConf->bParseTrusted = 0; pModConf->bDiscardOwnMsgs = 1; + pModConf->bUnlink = 1; pModConf->ratelimitIntervalSysSock = DFLT_ratelimitInterval; pModConf->ratelimitBurstSysSock = DFLT_ratelimitBurst; pModConf->ratelimitSeveritySysSock = DFLT_ratelimitSeverity; @@ -1139,12 +1155,16 @@ CODESTARTsetModCnf loadModConf->bIgnoreTimestamp = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.ignoreownmessages")) { loadModConf->bDiscardOwnMsgs = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.unlink")) { + loadModConf->bUnlink = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.flowcontrol")) { loadModConf->bUseFlowCtl = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.usesystimestamp")) { loadModConf->bUseSysTimeStamp = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.annotate")) { loadModConf->bAnnotateSysSock = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "syssock.parsetrusted")) { + loadModConf->bParseTrusted = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.usepidfromsystem")) { loadModConf->bWritePidSysSock = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "syssock.ratelimit.interval")) { @@ -1201,6 +1221,8 @@ CODESTARTnewInpInst inst->bParseTrusted = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "ignoreownmessages")) { inst->bDiscardOwnMsgs = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "unlink")) { + inst->bUnlink = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "hostname")) { inst->pLogHostName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(modpblk.descr[i].name, "ignoretimestamp")) { @@ -1240,6 +1262,9 @@ CODESTARTendCnfLoad loadModConf->bUseFlowCtl = cs.bUseFlowCtlSysSock; loadModConf->bAnnotateSysSock = cs.bAnnotateSysSock; loadModConf->bParseTrusted = cs.bParseTrusted; + loadModConf->ratelimitIntervalSysSock = cs.ratelimitIntervalSysSock; + loadModConf->ratelimitBurstSysSock = cs.ratelimitBurstSysSock; + loadModConf->ratelimitSeveritySysSock = cs.ratelimitSeveritySysSock; } loadModConf = NULL; /* done loading */ @@ -1377,8 +1402,10 @@ CODESTARTafterRun listeners[i].fd < SD_LISTEN_FDS_START + sd_fds) continue; - DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName); - unlink((char*) listeners[i].sockName); + if(listeners[i].bUnlink) { + DBGPRINTF("imuxsock: unlinking unix socket file[%d] %s\n", i, listeners[i].sockName); + unlink((char*) listeners[i].sockName); + } } discardLogSockets(); @@ -1491,6 +1518,7 @@ CODEmodInit_QueryRegCFSLineHdlr listeners[0].bAnnotate = 0; listeners[0].bParseTrusted = 0; listeners[0].bDiscardOwnMsgs = 1; + listeners[0].bUnlink = 1; listeners[0].bCreatePath = 0; listeners[0].bUseSysTimeStamp = 1; if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, diff --git a/plugins/mmanon/mmanon.c b/plugins/mmanon/mmanon.c index fc0c8a03..a1c99d09 100644 --- a/plugins/mmanon/mmanon.c +++ b/plugins/mmanon/mmanon.c @@ -29,6 +29,7 @@ #include <signal.h> #include <errno.h> #include <unistd.h> +#include <stdint.h> #include "conf.h" #include "syslogd-types.h" #include "srUtils.h" diff --git a/plugins/mmnormalize/mmnormalize.c b/plugins/mmnormalize/mmnormalize.c index fd2004a3..fcadc328 100644 --- a/plugins/mmnormalize/mmnormalize.c +++ b/plugins/mmnormalize/mmnormalize.c @@ -397,7 +397,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizerulebase", 0, eCmdHdlrGetWord, setRuleBase, NULL, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizeuserawmsg", 0, eCmdHdlrInt, + CHKiRet(omsdRegCFSLineHdlr((uchar *)"mmnormalizeuserawmsg", 0, eCmdHdlrBinary, NULL, &cs.bUseRawMsg, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); diff --git a/plugins/omlibdbi/omlibdbi.c b/plugins/omlibdbi/omlibdbi.c index 390e59d5..6e27ad22 100644 --- a/plugins/omlibdbi/omlibdbi.c +++ b/plugins/omlibdbi/omlibdbi.c @@ -70,6 +70,7 @@ typedef struct _instanceData { uchar *dbName; /* database to use */ unsigned uLastDBErrno; /* last errno returned by libdbi or 0 if all is well */ uchar *tplName; /* format template to use */ + int txSupport; /* transaction support */ } instanceData; typedef struct configSettings_s { @@ -261,7 +262,7 @@ static rsRetVal initConn(instanceData *pData, int bSilent) # endif if(pData->conn == NULL) { errmsg.LogError(0, RS_RET_SUSPENDED, "can not initialize libdbi connection"); - iRet = RS_RET_SUSPENDED; + ABORT_FINALIZE(RS_RET_SUSPENDED); } else { /* we could get the handle, now on with work... */ /* Connect to database */ dbi_conn_set_option(pData->conn, "host", (char*) pData->host); @@ -272,8 +273,9 @@ static rsRetVal initConn(instanceData *pData, int bSilent) if(dbi_conn_connect(pData->conn) < 0) { reportDBError(pData, bSilent); closeConn(pData); /* ignore any error we may get */ - iRet = RS_RET_SUSPENDED; + ABORT_FINALIZE(RS_RET_SUSPENDED); } + pData->txSupport = dbi_conn_cap_get(pData->conn, "transaction_support"); } finalize_it: @@ -329,12 +331,46 @@ CODESTARTtryResume } ENDtryResume +/* transaction support 2013-03 */ +BEGINbeginTransaction +CODESTARTbeginTransaction + if(pData->conn == NULL) { + CHKiRet(initConn(pData, 0)); + } +# if HAVE_DBI_TXSUPP + if (pData->txSupport == 1) { + if (dbi_conn_transaction_begin(pData->conn) != 0) { + dbgprintf("libdbi server error: begin transaction not successful\n"); + iRet = RS_RET_SUSPENDED; + } + } +# endif +finalize_it: +ENDbeginTransaction +/* end transaction */ + BEGINdoAction CODESTARTdoAction - dbgprintf("\n"); - iRet = writeDB(ppString[0], pData); + CHKiRet(writeDB(ppString[0], pData)); +# if HAVE_DBI_TXSUPP + if (pData->txSupport == 1) { + iRet = RS_RET_DEFER_COMMIT; + } +# endif +finalize_it: ENDdoAction +/* transaction support 2013-03 */ +BEGINendTransaction +CODESTARTendTransaction +# if HAVE_DBI_TXSUPP + if (dbi_conn_transaction_commit(pData->conn) != 0) { + dbgprintf("libdbi server error: transaction not committed\n"); + iRet = RS_RET_SUSPENDED; + } +# endif +ENDendTransaction +/* end transaction */ BEGINbeginCnfLoad CODESTARTbeginCnfLoad @@ -427,7 +463,6 @@ CODESTARTnewActInst CHKiRet(createInstance(&pData)); setInstParamDefaults(pData); - CODE_STD_STRING_REQUESTnewActInst(1) for(i = 0 ; i < actpblk.nParams ; ++i) { if(!pvals[i].bUsed) @@ -468,7 +503,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* ok, if we reach this point, we have something for us */ CHKiRet(createInstance(&pData)); - /* no create the instance based on what we currently have */ if(cs.drvrName == NULL) { errmsg.LogError(0, RS_RET_NO_DRIVERNAME, "omlibdbi: no db driver name given - action can not be created"); @@ -513,6 +547,7 @@ CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt @@ -542,6 +577,10 @@ CODESTARTmodInit INITLegCnfVars *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr +# ifndef HAVE_DBI_TXSUPP + DBGPRINTF("omlibdbi: no transaction support in libdbi\n"); +# warning libdbi too old - transactions are not enabled (use 0.9 or later) +# endif CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(regCfSysLineHdlr2((uchar *)"actionlibdbidriverdirectory", 0, eCmdHdlrGetWord, NULL, &cs.dbiDrvrDir, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"actionlibdbidriver", 0, eCmdHdlrGetWord, NULL, &cs.drvrName, STD_LOADABLE_MODULE_ID)); diff --git a/plugins/omrelp/omrelp.c b/plugins/omrelp/omrelp.c index e55836c5..c9e32444 100644 --- a/plugins/omrelp/omrelp.c +++ b/plugins/omrelp/omrelp.c @@ -7,7 +7,7 @@ * * File begun on 2008-03-13 by RGerhards * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -43,6 +43,7 @@ #include "glbl.h" #include "errmsg.h" #include "debug.h" +#include "unicode-helper.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP @@ -57,12 +58,14 @@ DEFobjCurrIf(glbl) static relpEngine_t *pRelpEngine; /* our relp engine */ typedef struct _instanceData { - char *f_hname; + uchar *target; int compressionLevel; /* 0 - no compression, else level for zlib */ - char *port; + uchar *port; int bInitialConnect; /* is this the initial connection request of our module? (0-no, 1-yes) */ int bIsConnected; /* currently connected to server? 0 - no, 1 - yes */ - relpClt_t *pRelpClt; /* relp client for this instance */ + unsigned timeout; + relpClt_t *pRelpClt; /* relp client for this instance */ + uchar *tplName; } instanceData; typedef struct configSettings_s { @@ -70,30 +73,114 @@ typedef struct configSettings_s { } configSettings_t; static configSettings_t __attribute__((unused)) cs; + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "target", eCmdHdlrGetWord, 1 }, + { "port", eCmdHdlrGetWord, 0 }, + { "timeout", eCmdHdlrInt, 0 }, + { "template", eCmdHdlrGetWord, 1 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars ENDinitConfVars -/* get the syslog forward port from selector_t. The passed in - * struct must be one that is setup for forwarding. - * rgerhards, 2007-06-28 - * We may change the implementation to try to lookup the port - * if it is unspecified. So far, we use the IANA default auf 514. +/* We may change the implementation to try to lookup the port + * if it is unspecified. So far, we use 514 as default (what probably + * is not a really bright idea, but kept for backward compatibility). */ -static char *getRelpPt(instanceData *pData) +static uchar *getRelpPt(instanceData *pData) { assert(pData != NULL); if(pData->port == NULL) - return("514"); + return((uchar*)"514"); else return(pData->port); } +static inline rsRetVal +doCreateRelpClient(instanceData *pData) +{ + DEFiRet; + if(relpEngineCltConstruct(pRelpEngine, &pData->pRelpClt) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpCltSetTimeout(pData->pRelpClt, pData->timeout) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); +finalize_it: + RETiRet; +} + + BEGINcreateInstance CODESTARTcreateInstance pData->bInitialConnect = 1; ENDcreateInstance +BEGINfreeInstance +CODESTARTfreeInstance + if(pData->pRelpClt != NULL) + relpEngineCltDestruct(pRelpEngine, &pData->pRelpClt); + free(pData->target); + free(pData->port); + free(pData->tplName); +ENDfreeInstance + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->target = NULL; + pData->port = NULL; + pData->tplName = NULL; + pData->timeout = 90; +} + + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "target")) { + pData->target = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "port")) { + pData->port = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "timeout")) { + pData->timeout = (unsigned) pvals[i].val.d.n; + } else { + dbgprintf("omrelp: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + CODE_STD_STRING_REQUESTnewActInst(1) + + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup((pData->tplName == NULL) ? + "RSYSLOG_ForwardFormat" : (char*)pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + + CHKiRet(doCreateRelpClient(pData)); + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -101,25 +188,16 @@ CODESTARTisCompatibleWithFeature iRet = RS_RET_OK; ENDisCompatibleWithFeature - -BEGINfreeInstance -CODESTARTfreeInstance - if(pData->port != NULL) - free(pData->port); - - /* final cleanup */ - if(pData->pRelpClt != NULL) - relpEngineCltDestruct(pRelpEngine, &pData->pRelpClt); - - if(pData->f_hname != NULL) - free(pData->f_hname); - -ENDfreeInstance +BEGINSetShutdownImmdtPtr +CODESTARTSetShutdownImmdtPtr + relpEngineSetShutdownImmdtPtr(pRelpEngine, pPtr); + DBGPRINTF("omrelp: shutdownImmediate ptr now is %p\n", pPtr); +ENDSetShutdownImmdtPtr BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo - printf("RELP/%s", pData->f_hname); + dbgprintf("RELP/%s", pData->target); ENDdbgPrintInstInfo @@ -131,7 +209,7 @@ static rsRetVal doConnect(instanceData *pData) DEFiRet; if(pData->bInitialConnect) { - iRet = relpCltConnect(pData->pRelpClt, glbl.GetDefPFFamily(), (uchar*) pData->port, (uchar*) pData->f_hname); + iRet = relpCltConnect(pData->pRelpClt, glbl.GetDefPFFamily(), pData->port, pData->target); if(iRet == RELP_RET_OK) pData->bInitialConnect = 0; } else { @@ -160,7 +238,7 @@ BEGINdoAction size_t lenMsg; relpRetVal ret; CODESTARTdoAction - dbgprintf(" %s:%s/RELP\n", pData->f_hname, getRelpPt(pData)); + dbgprintf(" %s:%s/RELP\n", pData->target, getRelpPt(pData)); if(!pData->bIsConnected) { CHKiRet(doConnect(pData)); @@ -309,21 +387,17 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* TODO: make this if go away! */ if(*p == ';') { *p = '\0'; /* trick to obtain hostname (later)! */ - CHKmalloc(pData->f_hname = strdup((char*) q)); + CHKmalloc(pData->target = ustrdup(q)); *p = ';'; } else { - CHKmalloc(pData->f_hname = strdup((char*) q)); + CHKmalloc(pData->target = ustrdup(q)); } /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*) "RSYSLOG_ForwardFormat")); - /* create our relp client */ - CHKiRet(relpEngineCltConstruct(pRelpEngine, &pData->pRelpClt)); /* we use CHKiRet as librelp has a similar return value range */ + CHKiRet(doCreateRelpClient(pData)); - /* TODO: do we need to call freeInstance if we failed - this is a general question for - * all output modules. I'll address it later as the interface evolves. rgerhards, 2007-07-25 - */ CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -342,6 +416,8 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_SetShutdownImmdtPtr ENDqueryEtryPt diff --git a/plugins/omstdout/omstdout.c b/plugins/omstdout/omstdout.c index 59f9c8bb..a84a7593 100644 --- a/plugins/omstdout/omstdout.c +++ b/plugins/omstdout/omstdout.c @@ -105,6 +105,7 @@ BEGINdoAction int iBuf; char szBuf[65564]; size_t len; + int r; CODESTARTdoAction if(pData->bUseArrayInterface) { /* if we use array passing, we need to put together a string @@ -140,9 +141,15 @@ CODESTARTdoAction * actually intends to use this module in production (why???), this code * needs to be more solid. -- rgerhards, 2012-11-28 */ - if(write(1, toWrite, len)) {}; /* 1 is stdout! */ + if((r = write(1, toWrite, len)) != (int) len) { /* 1 is stdout! */ + DBGPRINTF("omstdout: error %d writing to stdout[%d]: %s\n", + r, len, toWrite); + } if(pData->bEnsureLFEnding && toWrite[len-1] != '\n') { - if(write(1, "\n", 1)) {}; /* write missing LF */ + if((r = write(1, "\n", 1)) != 1) { /* write missing LF */ + DBGPRINTF("omstdout: error %d writing \\n to stdout\n", + r); + } } ENDdoAction diff --git a/plugins/omudpspoof/omudpspoof.c b/plugins/omudpspoof/omudpspoof.c index 4f37fd8c..9c4c80ba 100644 --- a/plugins/omudpspoof/omudpspoof.c +++ b/plugins/omudpspoof/omudpspoof.c @@ -481,7 +481,7 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) 0, /* checksum */ source_ip.sin_addr.s_addr, tempaddr->sin_addr.s_addr, - (u_int8_t*)(msg+msgOffs), /* payload */ + (uint8_t*)(msg+msgOffs), /* payload */ pktLen, /* payload size */ pData->libnet_handle, /* libnet handle */ ip); /* libnet id */ diff --git a/runtime/Makefile.am b/runtime/Makefile.am index fbc92d9c..ee5a3ef2 100644 --- a/runtime/Makefile.am +++ b/runtime/Makefile.am @@ -17,6 +17,7 @@ librsyslog_la_SOURCES = \ module-template.h \ im-helper.h \ obj-types.h \ + sigprov.h \ nsd.h \ glbl.h \ glbl.c \ @@ -92,14 +93,15 @@ librsyslog_la_SOURCES = \ ../template.h # the files with ../ we need to work on - so that they either become part of the # runtime or will no longer be needed. -- rgerhards, 2008-06-13 +# if WITH_MODDIRS -librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) -I\$(top_srcdir)/tools +librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) $(LIBGCRYPT_CFLAGS) -I\$(top_srcdir)/tools else -librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I$(top_srcdir) $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) -I\$(top_srcdir)/tools -I\$(top_srcdir)/grammar +librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I$(top_srcdir) $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) $(LIBGCRYPT_CFLAGS) -I\$(top_srcdir)/tools -I\$(top_srcdir)/grammar endif #librsyslog_la_LDFLAGS = -module -avoid-version -librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBEE_LIBS) +librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBGCRYPT_LIBS) $(LIBEE_LIBS) # # regular expression support @@ -172,6 +174,35 @@ lmnsd_gtls_la_LDFLAGS = -module -avoid-version lmnsd_gtls_la_LIBADD = $(GNUTLS_LIBS) endif +# +# support library for libgcrypt +# +if ENABLE_LIBGCRYPT + noinst_LTLIBRARIES += libgcry.la + libgcry_la_SOURCES = libgcry.c libgcry_common.c libgcry.h + libgcry_la_CPPFLAGS = $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) + pkglib_LTLIBRARIES += lmcry_gcry.la + lmcry_gcry_la_SOURCES = lmcry_gcry.c lmcry_gcry.h + lmcry_gcry_la_CPPFLAGS = $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) + lmcry_gcry_la_LDFLAGS = -module -avoid-version `libgcrypt-config --libs` + lmcry_gcry_la_LIBADD = libgcry.la $(LIBGCRYPT_LIBS) +endif + + +# +# support library for guardtime +# +if ENABLE_GUARDTIME + noinst_LTLIBRARIES += librsgt.la + librsgt_la_SOURCES = librsgt.c librsgt_read.c librsgt.h + pkglib_LTLIBRARIES += lmsig_gt.la + lmsig_gt_la_SOURCES = lmsig_gt.c lmsig_gt.h + lmsig_gt_la_CPPFLAGS = $(RSRT_CFLAGS) $(GUARDTIME_CFLAGS) + lmsig_gt_la_LDFLAGS = -module -avoid-version + lmsig_gt_la_LIBADD = librsgt.la $(GUARDTIME_LIBS) +endif + + update-systemd: - curl http://cgit.freedesktop.org/systemd/plain/src/sd-daemon.c > sd-daemon.c - curl http://cgit.freedesktop.org/systemd/plain/src/sd-daemon.h > sd-daemon.h + curl http://cgit.freedesktop.org/systemd/systemd/plain/src/libsystemd-daemon/sd-daemon.c > sd-daemon.c + curl http://cgit.freedesktop.org/systemd/systemd/plain/src/systemd/sd-daemon.h > sd-daemon.h diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c index 6b06d427..a437b7f8 100644 --- a/runtime/cfsysline.c +++ b/runtime/cfsysline.c @@ -350,8 +350,9 @@ static rsRetVal doGetGID(uchar **pp, rsRetVal (*pSetHdlr)(void*, uid_t), void *p struct group gBuf; DEFiRet; uchar szName[256]; - int bufSize = 2048; + int bufSize = 1024; char * stringBuf = NULL; + int err; assert(pp != NULL); assert(*pp != NULL); @@ -361,20 +362,21 @@ static rsRetVal doGetGID(uchar **pp, rsRetVal (*pSetHdlr)(void*, uid_t), void *p ABORT_FINALIZE(RS_RET_NOT_FOUND); } - - CHKmalloc(stringBuf = malloc(bufSize)); - while(pgBuf == NULL) { - errno = 0; - getgrnam_r((char*)szName, &gBuf, stringBuf, bufSize, &pgBuf); - if((pgBuf == NULL) && (errno == ERANGE)) { - /* Increase bufsize and try again.*/ - bufSize *= 2; - CHKmalloc(stringBuf = realloc(stringBuf, bufSize)); - } - } + do { + /* Increase bufsize and try again.*/ + bufSize *= 2; + CHKmalloc(stringBuf = realloc(stringBuf, bufSize)); + err = getgrnam_r((char*)szName, &gBuf, stringBuf, bufSize, &pgBuf); + } while((pgBuf == NULL) && (err == ERANGE)); if(pgBuf == NULL) { - errmsg.LogError(0, RS_RET_NOT_FOUND, "ID for group '%s' could not be found or error", (char*)szName); + if (err != 0) { + rs_strerror_r(err, stringBuf, bufSize); + errmsg.LogError(0, RS_RET_NOT_FOUND, "Query for group '%s' resulted in an error: %s\n", + (char*)szName, stringBuf); + } else { + errmsg.LogError(0, RS_RET_NOT_FOUND, "ID for group '%s' could not be found", (char*)szName); + } iRet = RS_RET_NOT_FOUND; } else { if(pSetHdlr == NULL) { diff --git a/runtime/cryprov.h b/runtime/cryprov.h new file mode 100644 index 00000000..8496b745 --- /dev/null +++ b/runtime/cryprov.h @@ -0,0 +1,39 @@ +/* The interface definition for (file) crypto providers. + * + * This is just an abstract driver interface, which needs to be + * implemented by concrete classes. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_CRYPROV_H +#define INCLUDED_CRYPROV_H + +#include <gcrypt.h> + +/* interface */ +BEGINinterface(cryprov) /* name must also be changed in ENDinterface macro! */ + rsRetVal (*Construct)(void *ppThis); + rsRetVal (*SetCnfParam)(void *ppThis, struct nvlst *lst); + rsRetVal (*Destruct)(void *ppThis); + rsRetVal (*OnFileOpen)(void *pThis, uchar *fn, void *pFileInstData); + rsRetVal (*Encrypt)(void *pFileInstData, uchar *buf, size_t *lenBuf); + rsRetVal (*OnFileClose)(void *pFileInstData, off64_t offsLogfile); +ENDinterface(cryprov) +#define cryprovCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#endif /* #ifndef INCLUDED_CRYPROV_H */ diff --git a/runtime/debug.c b/runtime/debug.c index 1f22d34e..876f61d0 100644 --- a/runtime/debug.c +++ b/runtime/debug.c @@ -303,7 +303,7 @@ static inline void dbgFuncDBRemoveMutexLock(dbgFuncDB_t *pFuncDB, pthread_mutex_ void dbgOutputTID(char* name) { -# ifdef HAVE_SYSCALL +# if defined(HAVE_SYSCALL) && defined(HAVE_SYS_gettid) if(bOutputTidToStderr) fprintf(stderr, "thread tid %u, name '%s'\n", (unsigned)syscall(SYS_gettid), name); @@ -1315,6 +1315,15 @@ dbgmalloc(size_t size) } +/* report fd used for debug log. This is needed in case of + * auto-backgrounding, where the debug log shall not be closed. + */ +int +dbgGetDbglogFd(void) +{ + return altdbg; +} + /* read in the runtime options * rgerhards, 2008-02-28 */ diff --git a/runtime/debug.h b/runtime/debug.h index d8585fee..f3226098 100644 --- a/runtime/debug.h +++ b/runtime/debug.h @@ -107,6 +107,7 @@ void dbgSetThrdName(uchar *pszName); void dbgPrintAllDebugInfo(void); void *dbgmalloc(size_t size); void dbgOutputTID(char* name); +int dbgGetDbglogFd(void); /* macros */ #ifdef DEBUGLESS diff --git a/runtime/libgcry.c b/runtime/libgcry.c new file mode 100644 index 00000000..e57ee8bc --- /dev/null +++ b/runtime/libgcry.c @@ -0,0 +1,426 @@ +/* gcry.c - rsyslog's libgcrypt based crypto provider + * + * Copyright 2013 Adiscon GmbH. + * + * We need to store some additional information in support of encryption. + * For this, we create a side-file, which is named like the actual log + * file, but with the suffix ".encinfo" appended. It contains the following + * records: + * IV:<hex> The initial vector used at block start. Also indicates start + * start of block. + * END:<int> The end offset of the block, as uint64_t in decimal notation. + * This is used during encryption to know when the current + * encryption block ends. + * For the current implementation, there must always be an IV record + * followed by an END record. Each records is LF-terminated. Record + * types can simply be extended in the future by specifying new + * types (like "IV") before the colon. + * To identify a file as rsyslog encryption info file, it must start with + * the line "FILETYPE:rsyslog-enrcyption-info" + * There are some size constraints: the recordtype must be 31 bytes at + * most and the actual value (between : and LF) must be 1023 bytes at most. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#if HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdio.h> +#include <gcrypt.h> +#include <sys/stat.h> +#include <sys/uio.h> +#include <sys/types.h> +#include <unistd.h> +#include <fcntl.h> +#include <errno.h> + +#include "rsyslog.h" +#include "libgcry.h" + + +static rsRetVal +eiWriteRec(gcryfile gf, char *recHdr, size_t lenRecHdr, char *buf, size_t lenBuf) +{ + struct iovec iov[3]; + ssize_t nwritten, towrite; + DEFiRet; + + iov[0].iov_base = recHdr; + iov[0].iov_len = lenRecHdr; + iov[1].iov_base = buf; + iov[1].iov_len = lenBuf; + iov[2].iov_base = "\n"; + iov[2].iov_len = 1; + towrite = iov[0].iov_len + iov[1].iov_len + iov[2].iov_len; + nwritten = writev(gf->fd, iov, sizeof(iov)/sizeof(struct iovec)); + if(nwritten != towrite) { + DBGPRINTF("eiWrite%s: error writing file, towrite %d, " + "nwritten %d\n", recHdr, (int) towrite, (int) nwritten); + ABORT_FINALIZE(RS_RET_EI_WR_ERR); + } + DBGPRINTF("encryption info file %s: written %s, len %d\n", + recHdr, gf->eiName, (int) nwritten); +finalize_it: + RETiRet; +} + +static rsRetVal +eiOpenRead(gcryfile gf) +{ + DEFiRet; + gf->fd = open((char*)gf->eiName, O_RDONLY|O_NOCTTY|O_CLOEXEC); + if(gf->fd == -1) { + ABORT_FINALIZE(errno == ENOENT ? RS_RET_EI_NO_EXISTS : RS_RET_EI_OPN_ERR); + } +finalize_it: + RETiRet; +} + + +static rsRetVal +eiCheckFiletype(gcryfile gf) +{ + char hdrBuf[128]; + size_t toRead, didRead; + DEFiRet; + + CHKiRet(eiOpenRead(gf)); + if(Debug) memset(hdrBuf, 0, sizeof(hdrBuf)); /* for dbgprintf below! */ + toRead = sizeof("FILETYPE:")-1 + sizeof(RSGCRY_FILETYPE_NAME)-1 + 1; + didRead = read(gf->fd, hdrBuf, toRead); + close(gf->fd); + DBGPRINTF("eiCheckFiletype read %d bytes: '%s'\n", didRead, hdrBuf); + if( didRead != toRead + || strncmp(hdrBuf, "FILETYPE:" RSGCRY_FILETYPE_NAME "\n", toRead)) + iRet = RS_RET_EI_INVLD_FILE; +finalize_it: + RETiRet; +} + +static rsRetVal +eiOpenAppend(gcryfile gf) +{ + rsRetVal localRet; + DEFiRet; + localRet = eiCheckFiletype(gf); + if(localRet == RS_RET_OK) { + gf->fd = open((char*)gf->eiName, + O_WRONLY|O_APPEND|O_NOCTTY|O_CLOEXEC, 0600); + if(gf->fd == -1) { + ABORT_FINALIZE(RS_RET_EI_OPN_ERR); + } + } else if(localRet == RS_RET_EI_NO_EXISTS) { + /* looks like we need to create a new file */ + gf->fd = open((char*)gf->eiName, + O_WRONLY|O_CREAT|O_NOCTTY|O_CLOEXEC, 0600); + if(gf->fd == -1) { + ABORT_FINALIZE(RS_RET_EI_OPN_ERR); + } + CHKiRet(eiWriteRec(gf, "FILETYPE:", 9, RSGCRY_FILETYPE_NAME, + sizeof(RSGCRY_FILETYPE_NAME)-1)); + } else { + gf->fd = -1; + ABORT_FINALIZE(localRet); + } + DBGPRINTF("encryption info file %s: opened as #%d\n", + gf->eiName, gf->fd); +finalize_it: + RETiRet; +} + +static rsRetVal +eiWriteIV(gcryfile gf, uchar *iv) +{ + static const char hexchars[16] = + {'0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f'}; + unsigned iSrc, iDst; + char hex[4096]; + DEFiRet; + + if(gf->blkLength > sizeof(hex)/2) { + DBGPRINTF("eiWriteIV: crypto block len way too large, aborting " + "write"); + ABORT_FINALIZE(RS_RET_ERR); + } + + for(iSrc = iDst = 0 ; iSrc < gf->blkLength ; ++iSrc) { + hex[iDst++] = hexchars[iv[iSrc]>>4]; + hex[iDst++] = hexchars[iv[iSrc]&0x0f]; + } + + iRet = eiWriteRec(gf, "IV:", 3, hex, gf->blkLength*2); +finalize_it: + RETiRet; +} + +/* we do not return an error state, as we MUST close the file, + * no matter what happens. + */ +static void +eiClose(gcryfile gf, off64_t offsLogfile) +{ + char offs[21]; + size_t len; + if(gf->fd == -1) + return; + /* 2^64 is 20 digits, so the snprintf buffer is large enough */ + len = snprintf(offs, sizeof(offs), "%lld", offsLogfile); + eiWriteRec(gf, "END:", 4, offs, len); + close(gf->fd); + DBGPRINTF("encryption info file %s: closed\n", gf->eiName); +} + +static rsRetVal +gcryfileConstruct(gcryctx ctx, gcryfile *pgf, uchar *logfn) +{ + char fn[MAXFNAME+1]; + gcryfile gf; + DEFiRet; + + CHKmalloc(gf = calloc(1, sizeof(struct gcryfile_s))); + gf->ctx = ctx; + snprintf(fn, sizeof(fn), "%s%s", logfn, ENCINFO_SUFFIX); + fn[MAXFNAME] = '\0'; /* be on save side */ + gf->eiName = (uchar*) strdup(fn); + *pgf = gf; +finalize_it: + RETiRet; +} + + +gcryctx +gcryCtxNew(void) +{ + gcryctx ctx; + ctx = calloc(1, sizeof(struct gcryctx_s)); + ctx->algo = GCRY_CIPHER_AES128; + ctx->mode = GCRY_CIPHER_MODE_CBC; + return ctx; +} + +int +gcryfileDestruct(gcryfile gf, off64_t offsLogfile) +{ + int r = 0; + if(gf == NULL) + goto done; + + eiClose(gf, offsLogfile); + free(gf->eiName); + free(gf); +done: return r; +} +void +rsgcryCtxDel(gcryctx ctx) +{ + if(ctx != NULL) { + free(ctx); + } +} + +static inline void +addPadding(gcryfile pF, uchar *buf, size_t *plen) +{ + unsigned i; + size_t nPad; + nPad = (pF->blkLength - *plen % pF->blkLength) % pF->blkLength; + dbgprintf("DDDD: addPadding %d chars, blkLength %d, mod %d, pad %d\n", + *plen, pF->blkLength, *plen % pF->blkLength, nPad); + for(i = 0 ; i < nPad ; ++i) + buf[(*plen)+i] = 0x00; + (*plen)+= nPad; +} + +static inline void +removePadding(char *buf, size_t *plen) +{ + unsigned len = (unsigned) *plen; + unsigned iSrc, iDst; + char *frstNUL; + + frstNUL = strchr(buf, 0x00); + if(frstNUL == NULL) + goto done; + iDst = iSrc = frstNUL - buf; + + while(iSrc < len) { + if(buf[iSrc] != 0x00) + buf[iDst++] = buf[iSrc]; + ++iSrc; + } + + *plen = iDst; +done: return; +} + +/* returns 0 on succes, positive if key length does not match and key + * of return value size is required. + */ +int +rsgcrySetKey(gcryctx ctx, unsigned char *key, uint16_t keyLen) +{ + uint16_t reqKeyLen; + int r; + + reqKeyLen = gcry_cipher_get_algo_keylen(ctx->algo); + if(keyLen != reqKeyLen) { + r = reqKeyLen; + goto done; + } + ctx->keyLen = keyLen; + ctx->key = malloc(keyLen); + memcpy(ctx->key, key, keyLen); + r = 0; +done: return r; +} + +rsRetVal +rsgcrySetMode(gcryctx ctx, uchar *modename) +{ + int mode; + DEFiRet; + + mode = rsgcryModename2Mode((char *)modename); + if(mode == GCRY_CIPHER_MODE_NONE) { + ABORT_FINALIZE(RS_RET_CRY_INVLD_MODE); + } + ctx->mode = mode; +finalize_it: + RETiRet; +} + +rsRetVal +rsgcrySetAlgo(gcryctx ctx, uchar *algoname) +{ + int algo; + DEFiRet; + + algo = rsgcryAlgoname2Algo((char *)algoname); + if(algo == GCRY_CIPHER_NONE) { + ABORT_FINALIZE(RS_RET_CRY_INVLD_ALGO); + } + ctx->algo = algo; +finalize_it: + RETiRet; +} + +/* As of some Linux and security expert I spoke to, /dev/urandom + * provides very strong random numbers, even if it runs out of + * entropy. As far as he knew, this is save for all applications + * (and he had good proof that I currently am not permitted to + * reproduce). -- rgerhards, 2013-03-04 + */ +void +seedIV(gcryfile gf, uchar **iv) +{ + int fd; + + *iv = malloc(gf->blkLength); /* do NOT zero-out! */ + /* if we cannot obtain data from /dev/urandom, we use whatever + * is present at the current memory location as random data. Of + * course, this is very weak and we should consider a different + * option, especially when not running under Linux (for Linux, + * unavailability of /dev/urandom is just a theoretic thing, it + * will always work...). -- TODO -- rgerhards, 2013-03-06 + */ + if((fd = open("/dev/urandom", O_RDONLY)) > 0) { + if(read(fd, *iv, gf->blkLength)) {}; /* keep compiler happy */ + close(fd); + } +} + +rsRetVal +rsgcryInitCrypt(gcryctx ctx, gcryfile *pgf, uchar *fname) +{ + gcry_error_t gcryError; + gcryfile gf = NULL; + uchar *iv = NULL; + DEFiRet; + + CHKiRet(gcryfileConstruct(ctx, &gf, fname)); + + gf->blkLength = gcry_cipher_get_algo_blklen(ctx->algo); + + gcryError = gcry_cipher_open(&gf->chd, ctx->algo, ctx->mode, 0); + if (gcryError) { + dbgprintf("gcry_cipher_open failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + ABORT_FINALIZE(RS_RET_ERR); + } + + gcryError = gcry_cipher_setkey(gf->chd, gf->ctx->key, gf->ctx->keyLen); + if (gcryError) { + dbgprintf("gcry_cipher_setkey failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + ABORT_FINALIZE(RS_RET_ERR); + } + + seedIV(gf, &iv); + gcryError = gcry_cipher_setiv(gf->chd, iv, gf->blkLength); + if (gcryError) { + dbgprintf("gcry_cipher_setiv failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + ABORT_FINALIZE(RS_RET_ERR); + } + CHKiRet(eiOpenAppend(gf)); + CHKiRet(eiWriteIV(gf, iv)); + *pgf = gf; +finalize_it: + free(iv); + if(iRet != RS_RET_OK && gf != NULL) + gcryfileDestruct(gf, -1); + RETiRet; +} + +int +rsgcryEncrypt(gcryfile pF, uchar *buf, size_t *len) +{ + int gcryError; + DEFiRet; + + if(*len == 0) + FINALIZE; + + addPadding(pF, buf, len); + gcryError = gcry_cipher_encrypt(pF->chd, buf, *len, NULL, 0); + if(gcryError) { + dbgprintf("gcry_cipher_encrypt failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + ABORT_FINALIZE(RS_RET_ERR); + } +finalize_it: + RETiRet; +} + + +/* module-init dummy for potential later use */ +int +rsgcryInit(void) +{ + return 0; +} + +/* module-deinit dummy for potential later use */ +void +rsgcryExit(void) +{ + return; +} diff --git a/runtime/libgcry.h b/runtime/libgcry.h new file mode 100644 index 00000000..5dde1576 --- /dev/null +++ b/runtime/libgcry.h @@ -0,0 +1,99 @@ +/* libgcry.h - rsyslog's guardtime support library + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_LIBGCRY_H +#define INCLUDED_LIBGCRY_H +#include <gt_base.h> + + +struct gcryctx_s { + uchar *key; + size_t keyLen; + int algo; + int mode; +}; +typedef struct gcryctx_s *gcryctx; +typedef struct gcryfile_s *gcryfile; + +/* this describes a file, as far as libgcry is concerned */ +struct gcryfile_s { + gcry_cipher_hd_t chd; /* cypher handle */ + size_t blkLength; /* size of low-level crypto block */ + uchar *eiName; /* name of .encinfo file */ + int fd; /* descriptor of .encinfo file (-1 if not open) */ + gcryctx ctx; +}; + +int gcryGetKeyFromFile(char *fn, char **key, unsigned *keylen); +int rsgcryInit(void); +void rsgcryExit(void); +int rsgcrySetKey(gcryctx ctx, unsigned char *key, uint16_t keyLen); +rsRetVal rsgcrySetMode(gcryctx ctx, uchar *algoname); +rsRetVal rsgcrySetAlgo(gcryctx ctx, uchar *modename); +gcryctx gcryCtxNew(void); +void rsgcryCtxDel(gcryctx ctx); +int gcryfileDestruct(gcryfile gf, off64_t offsLogfile); +rsRetVal rsgcryInitCrypt(gcryctx ctx, gcryfile *pgf, uchar *fname); +int rsgcryEncrypt(gcryfile pF, uchar *buf, size_t *len); + +/* error states */ +#define RSGCRYE_EI_OPEN 1 /* error opening .encinfo file */ +#define RSGCRYE_OOM 4 /* ran out of memory */ + +#define EIF_MAX_RECTYPE_LEN 31 /* max length of record types */ +#define EIF_MAX_VALUE_LEN 1023 /* max length of value types */ +#define RSGCRY_FILETYPE_NAME "rsyslog-enrcyption-info" +#define ENCINFO_SUFFIX ".encinfo" + +static inline int +rsgcryAlgoname2Algo(char *algoname) { + if(!strcmp((char*)algoname, "3DES")) return GCRY_CIPHER_3DES; + if(!strcmp((char*)algoname, "CAST5")) return GCRY_CIPHER_CAST5; + if(!strcmp((char*)algoname, "BLOWFISH")) return GCRY_CIPHER_BLOWFISH; + if(!strcmp((char*)algoname, "AES128")) return GCRY_CIPHER_AES128; + if(!strcmp((char*)algoname, "AES192")) return GCRY_CIPHER_AES192; + if(!strcmp((char*)algoname, "AES256")) return GCRY_CIPHER_AES256; + if(!strcmp((char*)algoname, "TWOFISH")) return GCRY_CIPHER_TWOFISH; + if(!strcmp((char*)algoname, "TWOFISH128")) return GCRY_CIPHER_TWOFISH128; + if(!strcmp((char*)algoname, "ARCFOUR")) return GCRY_CIPHER_ARCFOUR; + if(!strcmp((char*)algoname, "DES")) return GCRY_CIPHER_DES; + if(!strcmp((char*)algoname, "SERPENT128")) return GCRY_CIPHER_SERPENT128; + if(!strcmp((char*)algoname, "SERPENT192")) return GCRY_CIPHER_SERPENT192; + if(!strcmp((char*)algoname, "SERPENT256")) return GCRY_CIPHER_SERPENT256; + if(!strcmp((char*)algoname, "RFC2268_40")) return GCRY_CIPHER_RFC2268_40; + if(!strcmp((char*)algoname, "SEED")) return GCRY_CIPHER_SEED; + if(!strcmp((char*)algoname, "CAMELLIA128")) return GCRY_CIPHER_CAMELLIA128; + if(!strcmp((char*)algoname, "CAMELLIA192")) return GCRY_CIPHER_CAMELLIA192; + if(!strcmp((char*)algoname, "CAMELLIA256")) return GCRY_CIPHER_CAMELLIA256; + return GCRY_CIPHER_NONE; +} + +static inline int +rsgcryModename2Mode(char *modename) { + if(!strcmp((char*)modename, "ECB")) return GCRY_CIPHER_MODE_ECB; + if(!strcmp((char*)modename, "CFB")) return GCRY_CIPHER_MODE_CFB; + if(!strcmp((char*)modename, "CBC")) return GCRY_CIPHER_MODE_CBC; + if(!strcmp((char*)modename, "STREAM")) return GCRY_CIPHER_MODE_STREAM; + if(!strcmp((char*)modename, "OFB")) return GCRY_CIPHER_MODE_OFB; + if(!strcmp((char*)modename, "CTR")) return GCRY_CIPHER_MODE_CTR; + if(!strcmp((char*)modename, "AESWRAP")) return GCRY_CIPHER_MODE_AESWRAP; + return GCRY_CIPHER_MODE_NONE; +} +#endif /* #ifndef INCLUDED_LIBGCRY_H */ diff --git a/runtime/libgcry_common.c b/runtime/libgcry_common.c new file mode 100644 index 00000000..63b5e5d5 --- /dev/null +++ b/runtime/libgcry_common.c @@ -0,0 +1,206 @@ +/* libgcry_common.c + * This file hosts functions both being used by the rsyslog runtime as + * well as tools who do not use the runtime (so we can maintain the + * code at a single place). + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#if HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdio.h> +#include <gcrypt.h> +#include <sys/stat.h> +#include <sys/uio.h> +#include <sys/types.h> +#include <unistd.h> +#include <fcntl.h> +#include <errno.h> + +#include "rsyslog.h" /* we need data typedefs */ +#include "libgcry.h" + + +/* read a key from a key file + * @param[out] key - key buffer, must be freed by caller + * @param[out] keylen - length of buffer + * @returns 0 if OK, something else otherwise (we do not use + * iRet as this is also called from non-rsyslog w/o runtime) + * The key length is limited to 64KiB to prevent DoS. + * Note well: key is a blob, not a C string (NUL may be present!) + */ +int +gcryGetKeyFromFile(char *fn, char **key, unsigned *keylen) +{ + struct stat sb; + int fd; + int r; + + if(stat(fn, &sb) == -1) { + r = 1; goto done; + } + if((sb.st_mode & S_IFMT) != S_IFREG) { + r = 2; goto done; + } + if(sb.st_size > 64*1024) { + r = 3; goto done; + } + if((*key = malloc(sb.st_size)) == NULL) { + r = -1; goto done; + } + if((fd = open(fn, O_RDONLY)) < 0) { + r = 4; goto done; + } + if(read(fd, *key, sb.st_size) != sb.st_size) { + r = 5; goto done; + } + *keylen = sb.st_size; + close(fd); + r = 0; +done: return r; +} + + +/* execute the child process (must be called in child context + * after fork). + */ + +static void +execKeyScript(char *cmd, int pipefd[]) +{ + char *newargv[] = { NULL }; + char *newenviron[] = { NULL }; + + dup2(pipefd[0], STDIN_FILENO); + dup2(pipefd[1], STDOUT_FILENO); + + /* finally exec child */ +fprintf(stderr, "pre execve: %s\n", cmd); + execve(cmd, newargv, newenviron); + /* switch to? + execlp((char*)program, (char*) program, (char*)arg, NULL); + */ + + /* we should never reach this point, but if we do, we terminate */ +done: return; +} + + +static int +openPipe(char *cmd, int *fd) +{ + int pipefd[2]; + pid_t cpid; + int r; + + if(pipe(pipefd) == -1) { + r = 1; goto done; + } + + cpid = fork(); + if(cpid == -1) { + r = 1; goto done; + } + + if(cpid == 0) { + /* we are the child */ + execKeyScript(cmd, pipefd); + exit(1); + } + + close(pipefd[1]); + *fd = pipefd[0]; + r = 0; +done: return r; +} + + +/* Read a character from the program's output. */ +// TODO: highly unoptimized version, should be used in buffered +// mode +static int +readProgChar(int fd, char *c) +{ + int r; + if(read(fd, c, 1) != 1) { + r = 1; goto done; + } + r = 0; +done: return r; +} + +/* Read a line from the script. Line is terminated by LF, which + * is NOT put into the buffer. + * buf must be 64KiB + */ +static int +readProgLine(int fd, char *buf) +{ + char c; + int r; + unsigned i; + + for(i = 0 ; i < 64*1024 ; ++i) { + if((r = readProgChar(fd, &c)) != 0) goto done; + if(c == '\n') + break; + buf[i] = c; + }; + if(i >= 64*1024) { + r = 1; goto done; + } + buf[i] = '\0'; + r = 0; +done: return r; +} +static int +readProgKey(int fd, char *buf, unsigned keylen) +{ + char c; + int r; + unsigned i; + + for(i = 0 ; i < keylen ; ++i) { + if((r = readProgChar(fd, &c)) != 0) goto done; + buf[i] = c; + }; + r = 0; +done: return r; +} + +int +gcryGetKeyFromProg(char *cmd, char **key, unsigned *keylen) +{ + int r; + int fd; + char rcvBuf[64*1024]; + + if((r = openPipe(cmd, &fd)) != 0) goto done; + if((r = readProgLine(fd, rcvBuf)) != 0) goto done; + if(strcmp(rcvBuf, "RSYSLOG-KEY-PROVIDER:0")) { + r = 2; goto done; + } + if((r = readProgLine(fd, rcvBuf)) != 0) goto done; + *keylen = atoi(rcvBuf); + if((*key = malloc(*keylen)) == NULL) { + r = -1; goto done; + } + if((r = readProgKey(fd, *key, *keylen)) != 0) goto done; +done: return r; +} diff --git a/runtime/librsgt.c b/runtime/librsgt.c new file mode 100644 index 00000000..afafe2f2 --- /dev/null +++ b/runtime/librsgt.c @@ -0,0 +1,844 @@ +/* librsgt.c - rsyslog's guardtime support library + * + * Regarding the online algorithm for Merkle tree signing. Expected + * calling sequence is: + * + * sigblkConstruct + * for each signature block: + * sigblkInit + * for each record: + * sigblkAddRecord + * sigblkFinish + * sigblkDestruct + * + * Obviously, the next call after sigblkFinsh must either be to + * sigblkInit or sigblkDestruct (if no more signature blocks are + * to be emitted, e.g. on file close). sigblkDestruct saves state + * information (most importantly last block hash) and sigblkConstruct + * reads (or initilizes if not present) it. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <stdint.h> +#include <assert.h> +#include <unistd.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#define MAXFNAME 1024 + +#include <gt_http.h> + +#include "librsgt.h" + +typedef unsigned char uchar; +#ifndef VERSION +#define VERSION "no-version" +#endif + + +static void +reportErr(gtctx ctx, char *errmsg) +{ + if(ctx->errFunc == NULL) + goto done; + ctx->errFunc(ctx->usrptr, (uchar*)errmsg); +done: return; +} + +static void +reportGTAPIErr(gtctx ctx, gtfile gf, char *apiname, int ecode) +{ + char errbuf[4096]; + snprintf(errbuf, sizeof(errbuf), "%s[%s:%d]: %s", + (gf == NULL) ? (uchar*)"" : gf->sigfilename, + apiname, ecode, GT_getErrorString(ecode)); + errbuf[sizeof(errbuf)-1] = '\0'; + reportErr(ctx, errbuf); +} + +void +rsgtsetErrFunc(gtctx ctx, void (*func)(void*, uchar *), void *usrptr) +{ + ctx->usrptr = usrptr; + ctx->errFunc = func; +} + +imprint_t * +rsgtImprintFromGTDataHash(GTDataHash *hash) +{ + imprint_t *imp; + + if((imp = calloc(1, sizeof(imprint_t))) == NULL) { + goto done; + } + imp->hashID = hashIdentifier(hash->algorithm), + imp->len = hash->digest_length; + if((imp->data = (uint8_t*)malloc(imp->len)) == NULL) { + free(imp); imp = NULL; goto done; + } + memcpy(imp->data, hash->digest, imp->len); +done: return imp; +} + +void +rsgtimprintDel(imprint_t *imp) +{ + if(imp != NULL) { + free(imp->data), + free(imp); + } +} + +int +rsgtInit(char *usragent) +{ + int r = 0; + int ret = GT_OK; + + ret = GT_init(); + if(ret != GT_OK) { + r = 1; + goto done; + } + ret = GTHTTP_init(usragent, 1); + if(ret != GT_OK) { + r = 1; + goto done; + } +done: return r; +} + +void +rsgtExit(void) +{ + GTHTTP_finalize(); + GT_finalize(); +} + + +static inline gtfile +rsgtfileConstruct(gtctx ctx) +{ + gtfile gf; + if((gf = calloc(1, sizeof(struct gtfile_s))) == NULL) + goto done; + gf->ctx = ctx; + gf->hashAlg = ctx->hashAlg; + gf->bKeepRecordHashes = ctx->bKeepRecordHashes; + gf->bKeepTreeHashes = ctx->bKeepTreeHashes; + gf->x_prev = NULL; + +done: return gf; +} + +static inline int +tlvbufPhysWrite(gtfile gf) +{ + ssize_t lenBuf; + ssize_t iTotalWritten; + ssize_t iWritten; + char *pWriteBuf; + int r = 0; + + lenBuf = gf->tlvIdx; + pWriteBuf = gf->tlvBuf; + iTotalWritten = 0; + do { + iWritten = write(gf->fd, pWriteBuf, lenBuf); + if(iWritten < 0) { + iWritten = 0; /* we have written NO bytes! */ + if(errno == EINTR) { + /*NO ERROR, just continue */; + } else { + reportErr(gf->ctx, "signature file write error"); + r = RSGTE_IO; + goto finalize_it; + } + } + /* advance buffer to next write position */ + iTotalWritten += iWritten; + lenBuf -= iWritten; + pWriteBuf += iWritten; + } while(lenBuf > 0); /* Warning: do..while()! */ + +finalize_it: + gf->tlvIdx = 0; + return r; +} + +static inline int +tlvbufChkWrite(gtfile gf) +{ + if(gf->tlvIdx == sizeof(gf->tlvBuf)) { + return tlvbufPhysWrite(gf); + } + return 0; +} + + +/* write to TLV file buffer. If buffer is full, an actual call occurs. Else + * output is written only on flush or close. + */ +static inline int +tlvbufAddOctet(gtfile gf, int8_t octet) +{ + int r; + r = tlvbufChkWrite(gf); + if(r != 0) goto done; + gf->tlvBuf[gf->tlvIdx++] = octet; +done: return r; +} +static inline int +tlvbufAddOctetString(gtfile gf, uint8_t *octet, int size) +{ + int i, r = 0; + for(i = 0 ; i < size ; ++i) { + r = tlvbufAddOctet(gf, octet[i]); + if(r != 0) goto done; + } +done: return r; +} +/* return the actual length in to-be-written octets of an integer */ +static inline uint8_t +tlvbufGetInt64OctetSize(uint64_t val) +{ + if(val >> 56) + return 8; + if((val >> 48) & 0xff) + return 7; + if((val >> 40) & 0xff) + return 6; + if((val >> 32) & 0xff) + return 5; + if((val >> 24) & 0xff) + return 4; + if((val >> 16) & 0xff) + return 3; + if((val >> 8) & 0xff) + return 2; + return 1; +} +static inline int +tlvbufAddInt64(gtfile gf, uint64_t val) +{ + uint8_t doWrite = 0; + int r; + if(val >> 56) { + r = tlvbufAddOctet(gf, (val >> 56) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + if(doWrite || ((val >> 48) & 0xff)) { + r = tlvbufAddOctet(gf, (val >> 48) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + if(doWrite || ((val >> 40) & 0xff)) { + r = tlvbufAddOctet(gf, (val >> 40) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + if(doWrite || ((val >> 32) & 0xff)) { + r = tlvbufAddOctet(gf, (val >> 32) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + if(doWrite || ((val >> 24) & 0xff)) { + r = tlvbufAddOctet(gf, (val >> 24) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + if(doWrite || ((val >> 16) & 0xff)) { + r = tlvbufAddOctet(gf, (val >> 16) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + if(doWrite || ((val >> 8) & 0xff)) { + r = tlvbufAddOctet(gf, (val >> 8) & 0xff), doWrite = 1; + if(r != 0) goto done; + } + r = tlvbufAddOctet(gf, val & 0xff); +done: return r; +} + + +int +tlv8Write(gtfile gf, int flags, int tlvtype, int len) +{ + int r; + r = tlvbufAddOctet(gf, (flags << 5)|tlvtype); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, len & 0xff); +done: return r; +} + +int +tlv16Write(gtfile gf, int flags, int tlvtype, uint16_t len) +{ + uint16_t typ; + int r; + typ = ((flags|1) << 13)|tlvtype; + r = tlvbufAddOctet(gf, typ >> 8); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, typ & 0xff); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, (len >> 8) & 0xff); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, len & 0xff); +done: return r; +} + +int +tlvFlush(gtfile gf) +{ + return (gf->tlvIdx == 0) ? 0 : tlvbufPhysWrite(gf); +} + +int +tlvWriteHash(gtfile gf, uint16_t tlvtype, GTDataHash *rec) +{ + unsigned tlvlen; + int r; + tlvlen = 1 + rec->digest_length; + r = tlv16Write(gf, 0x00, tlvtype, tlvlen); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, hashIdentifier(gf->hashAlg)); + if(r != 0) goto done; + r = tlvbufAddOctetString(gf, rec->digest, rec->digest_length); +done: return r; +} + +int +tlvWriteBlockSig(gtfile gf, uchar *der, uint16_t lenDer) +{ + unsigned tlvlen; + uint8_t tlvlenRecords; + int r; + + tlvlenRecords = tlvbufGetInt64OctetSize(gf->nRecords); + tlvlen = 2 + 1 /* hash algo TLV */ + + 2 + hashOutputLengthOctets(gf->hashAlg) /* iv */ + + 2 + 1 + gf->lenBlkStrtHash /* last hash */ + + 2 + tlvlenRecords /* rec-count */ + + 4 + lenDer /* rfc-3161 */; + /* write top-level TLV object (block-sig */ + r = tlv16Write(gf, 0x00, 0x0902, tlvlen); + if(r != 0) goto done; + /* and now write the children */ + //FIXME: flags??? + /* hash-algo */ + r = tlv8Write(gf, 0x00, 0x00, 1); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, hashIdentifier(gf->hashAlg)); + if(r != 0) goto done; + /* block-iv */ + r = tlv8Write(gf, 0x00, 0x01, hashOutputLengthOctets(gf->hashAlg)); + if(r != 0) goto done; + r = tlvbufAddOctetString(gf, gf->IV, hashOutputLengthOctets(gf->hashAlg)); + if(r != 0) goto done; + /* last-hash */ + r = tlv8Write(gf, 0x00, 0x02, gf->lenBlkStrtHash+1); + if(r != 0) goto done; + r = tlvbufAddOctet(gf, hashIdentifier(gf->hashAlg)); + if(r != 0) goto done; + r = tlvbufAddOctetString(gf, gf->blkStrtHash, gf->lenBlkStrtHash); + if(r != 0) goto done; + /* rec-count */ + r = tlv8Write(gf, 0x00, 0x03, tlvlenRecords); + if(r != 0) goto done; + r = tlvbufAddInt64(gf, gf->nRecords); + if(r != 0) goto done; + /* rfc-3161 */ + r = tlv16Write(gf, 0x00, 0x906, lenDer); + if(r != 0) goto done; + r = tlvbufAddOctetString(gf, der, lenDer); +done: return r; +} + +/* support for old platforms - graceful degrade */ +#ifndef O_CLOEXEC +#define O_CLOEXEC 0 +#endif +/* read rsyslog log state file; if we cannot access it or the + * contents looks invalid, we flag it as non-present (and thus + * begin a new hash chain). + * The context is initialized accordingly. + */ +static void +readStateFile(gtfile gf) +{ + int fd; + struct rsgtstatefile sf; + + fd = open((char*)gf->statefilename, O_RDONLY|O_NOCTTY|O_CLOEXEC, 0600); + if(fd == -1) goto err; + + if(read(fd, &sf, sizeof(sf)) != sizeof(sf)) goto err; + if(strncmp(sf.hdr, "GTSTAT10", 8)) goto err; + + gf->lenBlkStrtHash = sf.lenHash; + gf->blkStrtHash = calloc(1, gf->lenBlkStrtHash); + if(read(fd, gf->blkStrtHash, gf->lenBlkStrtHash) + != gf->lenBlkStrtHash) { + free(gf->blkStrtHash); + goto err; + } +return; + +err: + gf->lenBlkStrtHash = hashOutputLengthOctets(gf->hashAlg); + gf->blkStrtHash = calloc(1, gf->lenBlkStrtHash); +} + +/* persist all information that we need to re-open and append + * to a log signature file. + */ +static void +writeStateFile(gtfile gf) +{ + int fd; + struct rsgtstatefile sf; + + fd = open((char*)gf->statefilename, + O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, 0600); + if(fd == -1) + goto done; + + memcpy(sf.hdr, "GTSTAT10", 8); + sf.hashID = hashIdentifier(gf->hashAlg); + sf.lenHash = gf->x_prev->len; + /* if the write fails, we cannot do anything against that. We check + * the condition just to keep the compiler happy. + */ + if(write(fd, &sf, sizeof(sf))){}; + if(write(fd, gf->x_prev->data, gf->x_prev->len)){}; + close(fd); +done: return; +} + + +int +tlvClose(gtfile gf) +{ + int r; + r = tlvFlush(gf); + close(gf->fd); + gf->fd = -1; + writeStateFile(gf); + return r; +} + + +/* note: if file exists, the last hash for chaining must + * be read from file. + */ +int +tlvOpen(gtfile gf, char *hdr, unsigned lenHdr) +{ + int r = 0; + gf->fd = open((char*)gf->sigfilename, + O_WRONLY|O_APPEND|O_NOCTTY|O_CLOEXEC, 0600); + if(gf->fd == -1) { + /* looks like we need to create a new file */ + gf->fd = open((char*)gf->sigfilename, + O_WRONLY|O_CREAT|O_NOCTTY|O_CLOEXEC, 0600); + if(gf->fd == -1) { + r = RSGTE_IO; + goto done; + } + memcpy(gf->tlvBuf, hdr, lenHdr); + gf->tlvIdx = lenHdr; + } else { + gf->tlvIdx = 0; /* header already present! */ + } + /* we now need to obtain the last previous hash, so that + * we can continue the hash chain. We do not check for error + * as a state file error can be recovered by graceful degredation. + */ + readStateFile(gf); +done: return r; +} + +/* + * As of some Linux and security expert I spoke to, /dev/urandom + * provides very strong random numbers, even if it runs out of + * entropy. As far as he knew, this is save for all applications + * (and he had good proof that I currently am not permitted to + * reproduce). -- rgerhards, 2013-03-04 + */ +void +seedIV(gtfile gf) +{ + int hashlen; + int fd; + + hashlen = hashOutputLengthOctets(gf->hashAlg); + gf->IV = malloc(hashlen); /* do NOT zero-out! */ + /* if we cannot obtain data from /dev/urandom, we use whatever + * is present at the current memory location as random data. Of + * course, this is very weak and we should consider a different + * option, especially when not running under Linux (for Linux, + * unavailability of /dev/urandom is just a theoretic thing, it + * will always work...). -- TODO -- rgerhards, 2013-03-06 + */ + if((fd = open("/dev/urandom", O_RDONLY)) > 0) { + if(read(fd, gf->IV, hashlen)) {}; /* keep compiler happy */ + close(fd); + } +} + +gtctx +rsgtCtxNew(void) +{ + gtctx ctx; + ctx = calloc(1, sizeof(struct gtctx_s)); + ctx->hashAlg = GT_HASHALG_SHA256; + ctx->errFunc = NULL; + ctx->usrptr = NULL; + ctx->timestamper = strdup( + "http://stamper.guardtime.net/gt-signingservice"); + return ctx; +} + +/* either returns gtfile object or NULL if something went wrong */ +gtfile +rsgtCtxOpenFile(gtctx ctx, unsigned char *logfn) +{ + gtfile gf; + char fn[MAXFNAME+1]; + + if((gf = rsgtfileConstruct(ctx)) == NULL) + goto done; + + snprintf(fn, sizeof(fn), "%s.gtsig", logfn); + fn[MAXFNAME] = '\0'; /* be on save side */ + gf->sigfilename = (uchar*) strdup(fn); + snprintf(fn, sizeof(fn), "%s.gtstate", logfn); + fn[MAXFNAME] = '\0'; /* be on save side */ + gf->statefilename = (uchar*) strdup(fn); + if(tlvOpen(gf, LOGSIGHDR, sizeof(LOGSIGHDR)-1) != 0) { + reportErr(ctx, "signature file open failed"); + gf = NULL; + } +done: return gf; +} + + +/* returns 0 on succes, 1 if algo is unknown */ +int +rsgtSetHashFunction(gtctx ctx, char *algName) +{ + int r = 0; + if(!strcmp(algName, "SHA2-256")) + ctx->hashAlg = GT_HASHALG_SHA256; + else if(!strcmp(algName, "SHA2-384")) + ctx->hashAlg = GT_HASHALG_SHA384; + else if(!strcmp(algName, "SHA2-512")) + ctx->hashAlg = GT_HASHALG_SHA512; + else if(!strcmp(algName, "SHA1")) + ctx->hashAlg = GT_HASHALG_SHA1; + else if(!strcmp(algName, "RIPEMD-160")) + ctx->hashAlg = GT_HASHALG_RIPEMD160; + else if(!strcmp(algName, "SHA2-224")) + ctx->hashAlg = GT_HASHALG_SHA224; + else + r = 1; + return r; +} + +int +rsgtfileDestruct(gtfile gf) +{ + int r = 0; + if(gf == NULL) + goto done; + + if(!gf->disabled && gf->bInBlk) { + r = sigblkFinish(gf); + if(r != 0) gf->disabled = 1; + } + if(!gf->disabled) + r = tlvClose(gf); + free(gf->sigfilename); + free(gf->statefilename); + free(gf->IV); + free(gf->blkStrtHash); + rsgtimprintDel(gf->x_prev); + free(gf); +done: return r; +} + +void +rsgtCtxDel(gtctx ctx) +{ + if(ctx != NULL) { + free(ctx->timestamper); + free(ctx); + } +} + +/* new sigblk is initialized, but maybe in existing ctx */ +void +sigblkInit(gtfile gf) +{ + if(gf == NULL) goto done; + seedIV(gf); + memset(gf->roots_valid, 0, sizeof(gf->roots_valid)/sizeof(char)); + gf->nRoots = 0; + gf->nRecords = 0; + gf->bInBlk = 1; +done: return; +} + + +/* concat: add IV to buffer */ +static inline void +bufAddIV(gtfile gf, uchar *buf, size_t *len) +{ + memcpy(buf+*len, gf->IV, hashOutputLengthOctets(gf->hashAlg)); + *len += sizeof(gf->IV); +} + + +/* concat: add imprint to buffer */ +static inline void +bufAddImprint(gtfile gf, uchar *buf, size_t *len, imprint_t *imp) +{ + if(imp == NULL) { + /* TODO: how to get the REAL HASH ID? --> add field? */ + buf[*len] = hashIdentifier(gf->hashAlg); + ++(*len); + memcpy(buf+*len, gf->blkStrtHash, gf->lenBlkStrtHash); + *len += gf->lenBlkStrtHash; + } else { + buf[*len] = imp->hashID; + ++(*len); + memcpy(buf+*len, imp->data, imp->len); + *len += imp->len; + } +} +/* concat: add hash to buffer */ +static inline void +bufAddHash(gtfile gf, uchar *buf, size_t *len, GTDataHash *hash) +{ + buf[*len] = hashIdentifier(gf->hashAlg); + ++(*len); + memcpy(buf+*len, hash->digest, hash->digest_length); + *len += hash->digest_length; +} +/* concat: add tree level to buffer */ +static inline void +bufAddLevel(uchar *buf, size_t *len, uint8_t level) +{ + memcpy(buf+*len, &level, sizeof(level)); + *len += sizeof(level); +} + + +int +hash_m(gtfile gf, GTDataHash **m) +{ + int rgt; + uchar concatBuf[16*1024]; + size_t len = 0; + int r = 0; + + bufAddImprint(gf, concatBuf, &len, gf->x_prev); + bufAddIV(gf, concatBuf, &len); + rgt = GTDataHash_create(gf->hashAlg, concatBuf, len, m); + if(rgt != GT_OK) { + reportGTAPIErr(gf->ctx, gf, "GTDataHash_create", rgt); + r = RSGTE_HASH_CREATE; + goto done; + } +done: return r; +} + +int +hash_r(gtfile gf, GTDataHash **r, const uchar *rec, const size_t len) +{ + int ret = 0, rgt; + rgt = GTDataHash_create(gf->hashAlg, rec, len, r); + if(rgt != GT_OK) { + reportGTAPIErr(gf->ctx, gf, "GTDataHash_create", rgt); + ret = RSGTE_HASH_CREATE; + goto done; + } +done: return ret; +} + + +int +hash_node(gtfile gf, GTDataHash **node, GTDataHash *m, GTDataHash *rec, + uint8_t level) +{ + int r = 0, rgt; + uchar concatBuf[16*1024]; + size_t len = 0; + + bufAddHash(gf, concatBuf, &len, m); + bufAddHash(gf, concatBuf, &len, rec); + bufAddLevel(concatBuf, &len, level); + rgt = GTDataHash_create(gf->hashAlg, concatBuf, len, node); + if(rgt != GT_OK) { + reportGTAPIErr(gf->ctx, gf, "GTDataHash_create", rgt); + r = RSGTE_HASH_CREATE; + goto done; + } +done: return r; +} + + +int +sigblkAddRecord(gtfile gf, const uchar *rec, const size_t len) +{ + GTDataHash *x; /* current hash */ + GTDataHash *m, *r, *t, *t_del; + uint8_t j; + int ret = 0; + + if(gf == NULL || gf->disabled) goto done; + if((ret = hash_m(gf, &m)) != 0) goto done; + if((ret = hash_r(gf, &r, rec, len)) != 0) goto done; + if(gf->bKeepRecordHashes) + tlvWriteHash(gf, 0x0900, r); + if((ret = hash_node(gf, &x, m, r, 1)) != 0) goto done; /* hash leaf */ + /* persists x here if Merkle tree needs to be persisted! */ + if(gf->bKeepTreeHashes) + tlvWriteHash(gf, 0x0901, x); + rsgtimprintDel(gf->x_prev); + gf->x_prev = rsgtImprintFromGTDataHash(x); + /* add x to the forest as new leaf, update roots list */ + t = x; + for(j = 0 ; j < gf->nRoots ; ++j) { + if(gf->roots_valid[j] == 0) { + gf->roots_hash[j] = t; + gf->roots_valid[j] = 1; + t = NULL; + break; + } else if(t != NULL) { + /* hash interim node */ + t_del = t; + ret = hash_node(gf, &t, gf->roots_hash[j], t_del, j+2); + gf->roots_valid[j] = 0; + GTDataHash_free(gf->roots_hash[j]); + GTDataHash_free(t_del); + if(ret != 0) goto done; + if(gf->bKeepTreeHashes) + tlvWriteHash(gf, 0x0901, t); + } + } + if(t != NULL) { + /* new level, append "at the top" */ + gf->roots_hash[gf->nRoots] = t; + gf->roots_valid[gf->nRoots] = 1; + ++gf->nRoots; + assert(gf->nRoots < MAX_ROOTS); + t = NULL; + } + ++gf->nRecords; + + /* cleanup (x is cleared as part of the roots array) */ + GTDataHash_free(m); + GTDataHash_free(r); + + if(gf->nRecords == gf->blockSizeLimit) { + ret = sigblkFinish(gf); + if(ret != 0) goto done; + sigblkInit(gf); + } +done: + if(ret != 0) { + gf->disabled = 1; + } + return ret; +} + +static int +timestampIt(gtfile gf, GTDataHash *hash) +{ + unsigned char *der; + size_t lenDer; + int r = GT_OK; + int ret = 0; + GTTimestamp *timestamp = NULL; + + /* Get the timestamp. */ + r = GTHTTP_createTimestampHash(hash, gf->ctx->timestamper, ×tamp); + + if(r != GT_OK) { + reportGTAPIErr(gf->ctx, gf, "GTHTTP_createTimestampHash", r); + ret = 1; + goto done; + } + + /* Encode timestamp. */ + r = GTTimestamp_getDEREncoded(timestamp, &der, &lenDer); + if(r != GT_OK) { + reportGTAPIErr(gf->ctx, gf, "GTTimestamp_getDEREncoded", r); + ret = 1; + goto done; + } + + tlvWriteBlockSig(gf, der, lenDer); + +done: + GT_free(der); + GTTimestamp_free(timestamp); + return ret; +} + + +int +sigblkFinish(gtfile gf) +{ + GTDataHash *root, *rootDel; + int8_t j; + int ret = 0; + + if(gf->nRecords == 0) + goto done; + + root = NULL; + for(j = 0 ; j < gf->nRoots ; ++j) { + if(root == NULL) { + root = gf->roots_valid[j] ? gf->roots_hash[j] : NULL; + gf->roots_valid[j] = 0; + } else if(gf->roots_valid[j]) { + rootDel = root; + ret = hash_node(gf, &root, gf->roots_hash[j], rootDel, j+2); + gf->roots_valid[j] = 0; + GTDataHash_free(gf->roots_hash[j]); + GTDataHash_free(rootDel); + if(ret != 0) goto done; /* checks hash_node() result! */ + } + } + if((ret = timestampIt(gf, root)) != 0) goto done; + + GTDataHash_free(root); + free(gf->blkStrtHash); + gf->lenBlkStrtHash = gf->x_prev->len; + gf->blkStrtHash = malloc(gf->lenBlkStrtHash); + memcpy(gf->blkStrtHash, gf->x_prev->data, gf->x_prev->len); +done: + gf->bInBlk = 0; + return ret; +} diff --git a/runtime/librsgt.h b/runtime/librsgt.h new file mode 100644 index 00000000..bfcc4628 --- /dev/null +++ b/runtime/librsgt.h @@ -0,0 +1,388 @@ +/* librsgt.h - rsyslog's guardtime support library + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_LIBRSGT_H +#define INCLUDED_LIBRSGT_H +#include <gt_base.h> + +/* Max number of roots inside the forest. This permits blocks of up to + * 2^MAX_ROOTS records. We assume that 64 is sufficient for all use + * cases ;) [and 64 is not really a waste of memory, so we do not even + * try to work with reallocs and such...] + */ +#define MAX_ROOTS 64 +#define LOGSIGHDR "LOGSIG10" + +/* context for gt calls. This primarily serves as a container for the + * config settings. The actual file-specific data is kept in gtfile. + */ +struct gtctx_s { + enum GTHashAlgorithm hashAlg; + uint8_t bKeepRecordHashes; + uint8_t bKeepTreeHashes; + uint64_t blockSizeLimit; + char *timestamper; + void (*errFunc)(void *, unsigned char*); + void *usrptr; /* for error function */ +}; +typedef struct gtctx_s *gtctx; +typedef struct gtfile_s *gtfile; +typedef struct gterrctx_s gterrctx_t; +typedef struct imprint_s imprint_t; +typedef struct block_sig_s block_sig_t; +typedef struct tlvrecord_s tlvrecord_t; + +/* this describes a file, as far as librsgt is concerned */ +struct gtfile_s { + /* the following data items are mirrored from gtctx to + * increase cache hit ratio (they are frequently accesed). + */ + enum GTHashAlgorithm hashAlg; + uint8_t bKeepRecordHashes; + uint8_t bKeepTreeHashes; + /* end mirrored properties */ + uint8_t disabled; /* permits to disable this file --> set to 1 */ + uint64_t blockSizeLimit; + uint8_t *IV; /* initial value for blinding masks */ + imprint_t *x_prev; /* last leaf hash (maybe of previous block) --> preserve on term */ + unsigned char *sigfilename; + unsigned char *statefilename; + int fd; + unsigned char *blkStrtHash; /* last hash from previous block */ + uint16_t lenBlkStrtHash; + uint64_t nRecords; /* current number of records in current block */ + uint64_t bInBlk; /* are we currently inside a blk --> need to finish on close */ + int8_t nRoots; + /* algo engineering: roots structure is split into two arrays + * in order to improve cache hits. + */ + int8_t roots_valid[MAX_ROOTS]; + GTDataHash *roots_hash[MAX_ROOTS]; + /* data members for the associated TLV file */ + char tlvBuf[4096]; + int tlvIdx; /* current index into tlvBuf */ + gtctx ctx; +}; + +struct tlvrecord_s { + uint16_t tlvtype; + uint16_t tlvlen; + uint8_t hdr[4]; /* the raw header (as persisted to file) */ + uint8_t lenHdr; /* length of raw header */ + uint8_t data[64*1024]; /* the actual data part (of length tlvlen) */ +}; + +/* The following structure describes the "error context" to be used + * for verification and similiar reader functions. While verifying, + * we need some information (like filenames or block numbers) that + * is not readily available from the other objects (or not even known + * to librsgt). In order to provide meaningful error messages, this + * information must be passed in from the external callers. In order + * to centralize information (and make it more manageable), we use + * ths error context here, which contains everything needed to + * generate good error messages. Members of this structure are + * maintained both by library users (the callers) as well as + * the library itself. Who does what simply depends on who has + * the relevant information. + */ +struct gterrctx_s { + FILE *fp; /**< file for error messages */ + char *filename; + uint8_t verbose; + uint64_t recNumInFile; + uint64_t recNum; + uint64_t blkNum; + uint8_t treeLevel; + GTDataHash *computedHash; + GTDataHash *lefthash, *righthash; /* hashes to display if tree hash fails */ + imprint_t *fileHash; + int gtstate; /* status from last relevant GT.*() function call */ + char *errRec; + char *frstRecInBlk; /* This holds the first message seen inside the current block */ +}; + +struct imprint_s { + uint8_t hashID; + int len; + uint8_t *data; +}; + +#define SIGID_RFC3161 0 +struct block_sig_s { + uint8_t hashID; + uint8_t sigID; /* what type of *signature*? */ + uint8_t *iv; + imprint_t lastHash; + uint64_t recCount; + struct { + struct { + uint8_t *data; + size_t len; /* must be size_t due to GT API! */ + } der; + } sig; +}; + + +/* the following defines the gtstate file record. Currently, this record + * is fixed, we may change that over time. + */ +struct rsgtstatefile { + char hdr[8]; /* must be "GTSTAT10" */ + uint8_t hashID; + uint8_t lenHash; + /* after that, the hash value is contained within the file */ +}; + +/* Flags and record types for TLV handling */ +#define RSGT_FLAG_TLV16 0x20 + +/* error states */ +#define RSGTE_IO 1 /* any kind of io error */ +#define RSGTE_FMT 2 /* data fromat error */ +#define RSGTE_INVLTYP 3 /* invalid TLV type record (unexcpected at this point) */ +#define RSGTE_OOM 4 /* ran out of memory */ +#define RSGTE_LEN 5 /* error related to length records */ +#define RSGTE_TS_EXTEND 6/* error extending timestamp */ +#define RSGTE_INVLD_RECCNT 7/* mismatch between actual records and records + given in block-sig record */ +#define RSGTE_INVLHDR 8/* invalid file header */ +#define RSGTE_EOF 9 /* specific EOF */ +#define RSGTE_MISS_REC_HASH 10 /* record hash missing when expected */ +#define RSGTE_MISS_TREE_HASH 11 /* tree hash missing when expected */ +#define RSGTE_INVLD_REC_HASH 12 /* invalid record hash (failed verification) */ +#define RSGTE_INVLD_TREE_HASH 13 /* invalid tree hash (failed verification) */ +#define RSGTE_INVLD_REC_HASHID 14 /* invalid record hash ID (failed verification) */ +#define RSGTE_INVLD_TREE_HASHID 15 /* invalid tree hash ID (failed verification) */ +#define RSGTE_MISS_BLOCKSIG 16 /* block signature record missing when expected */ +#define RSGTE_INVLD_TIMESTAMP 17 /* RFC3161 timestamp is invalid */ +#define RSGTE_TS_DERDECODE 18 /* error DER-Decoding a timestamp */ +#define RSGTE_TS_DERENCODE 19 /* error DER-Encoding a timestamp */ +#define RSGTE_HASH_CREATE 20 /* error creating a hash */ + +/* the following function maps RSGTE_* state to a string - must be updated + * whenever a new state is added. + * Note: it is thread-safe to call this function, as it returns a pointer + * into constant memory pool. + */ +static inline char * +RSGTE2String(int err) +{ + switch(err) { + case 0: + return "success"; + case RSGTE_IO: + return "i/o error"; + case RSGTE_FMT: + return "data format error"; + case RSGTE_INVLTYP: + return "invalid/unexpected tlv record type"; + case RSGTE_OOM: + return "out of memory"; + case RSGTE_LEN: + return "length record problem"; + case RSGTE_TS_EXTEND: + return "error extending timestamp"; + case RSGTE_INVLD_RECCNT: + return "mismatch between actual record count and number in block signature record"; + case RSGTE_INVLHDR: + return "invalid file header"; + case RSGTE_EOF: + return "EOF"; + case RSGTE_MISS_REC_HASH: + return "record hash missing"; + case RSGTE_MISS_TREE_HASH: + return "tree hash missing"; + case RSGTE_INVLD_REC_HASH: + return "record hash mismatch"; + case RSGTE_INVLD_TREE_HASH: + return "tree hash mismatch"; + case RSGTE_INVLD_REC_HASHID: + return "invalid record hash ID"; + case RSGTE_INVLD_TREE_HASHID: + return "invalid tree hash ID"; + case RSGTE_MISS_BLOCKSIG: + return "missing block signature record"; + case RSGTE_INVLD_TIMESTAMP: + return "RFC3161 timestamp invalid"; + case RSGTE_TS_DERDECODE: + return "error DER-decoding RFC3161 timestamp"; + case RSGTE_TS_DERENCODE: + return "error DER-encoding RFC3161 timestamp"; + case RSGTE_HASH_CREATE: + return "error creating hash"; + default: + return "unknown error"; + } +} + + +static inline uint16_t +hashOutputLengthOctets(uint8_t hashID) +{ + switch(hashID) { + case GT_HASHALG_SHA1: /* paper: SHA1 */ + return 20; + case GT_HASHALG_RIPEMD160: /* paper: RIPEMD-160 */ + return 20; + case GT_HASHALG_SHA224: /* paper: SHA2-224 */ + return 28; + case GT_HASHALG_SHA256: /* paper: SHA2-256 */ + return 32; + case GT_HASHALG_SHA384: /* paper: SHA2-384 */ + return 48; + case GT_HASHALG_SHA512: /* paper: SHA2-512 */ + return 64; + default:return 32; + } +} + +static inline uint8_t +hashIdentifier(enum GTHashAlgorithm hashID) +{ + switch(hashID) { + case GT_HASHALG_SHA1: /* paper: SHA1 */ + return 0x00; + case GT_HASHALG_RIPEMD160: /* paper: RIPEMD-160 */ + return 0x02; + case GT_HASHALG_SHA224: /* paper: SHA2-224 */ + return 0x03; + case GT_HASHALG_SHA256: /* paper: SHA2-256 */ + return 0x01; + case GT_HASHALG_SHA384: /* paper: SHA2-384 */ + return 0x04; + case GT_HASHALG_SHA512: /* paper: SHA2-512 */ + return 0x05; + default:return 0xff; + } +} +static inline char * +hashAlgName(uint8_t hashID) +{ + switch(hashID) { + case GT_HASHALG_SHA1: + return "SHA1"; + case GT_HASHALG_RIPEMD160: + return "RIPEMD-160"; + case GT_HASHALG_SHA224: + return "SHA2-224"; + case GT_HASHALG_SHA256: + return "SHA2-256"; + case GT_HASHALG_SHA384: + return "SHA2-384"; + case GT_HASHALG_SHA512: + return "SHA2-512"; + default:return "[unknown]"; + } +} +static inline enum GTHashAlgorithm +hashID2Alg(uint8_t hashID) +{ + switch(hashID) { + case 0x00: + return GT_HASHALG_SHA1; + case 0x02: + return GT_HASHALG_RIPEMD160; + case 0x03: + return GT_HASHALG_SHA224; + case 0x01: + return GT_HASHALG_SHA256; + case 0x04: + return GT_HASHALG_SHA384; + case 0x05: + return GT_HASHALG_SHA512; + default: + return 0xff; + } +} +static inline char * +sigTypeName(uint8_t sigID) +{ + switch(sigID) { + case SIGID_RFC3161: + return "RFC3161"; + default:return "[unknown]"; + } +} +static inline uint16_t +getIVLen(block_sig_t *bs) +{ + return hashOutputLengthOctets(bs->hashID); +} +static inline void +rsgtSetTimestamper(gtctx ctx, char *timestamper) +{ + free(ctx->timestamper); + ctx->timestamper = strdup(timestamper); +} +static inline void +rsgtSetBlockSizeLimit(gtctx ctx, uint64_t limit) +{ + ctx->blockSizeLimit = limit; +} +static inline void +rsgtSetKeepRecordHashes(gtctx ctx, int val) +{ + ctx->bKeepRecordHashes = val; +} +static inline void +rsgtSetKeepTreeHashes(gtctx ctx, int val) +{ + ctx->bKeepTreeHashes = val; +} + +int rsgtSetHashFunction(gtctx ctx, char *algName); +int rsgtInit(char *usragent); +void rsgtExit(void); +gtctx rsgtCtxNew(void); +void rsgtsetErrFunc(gtctx ctx, void (*func)(void*, unsigned char *), void *usrptr); +gtfile rsgtCtxOpenFile(gtctx ctx, unsigned char *logfn); +int rsgtfileDestruct(gtfile gf); +void rsgtCtxDel(gtctx ctx); +void sigblkInit(gtfile gf); +int sigblkAddRecord(gtfile gf, const unsigned char *rec, const size_t len); +int sigblkFinish(gtfile gf); +imprint_t * rsgtImprintFromGTDataHash(GTDataHash *hash); +void rsgtimprintDel(imprint_t *imp); +/* reader functions */ +int rsgt_tlvrdHeader(FILE *fp, unsigned char *hdr); +int rsgt_tlvrd(FILE *fp, tlvrecord_t *rec, void *obj); +void rsgt_tlvprint(FILE *fp, uint16_t tlvtype, void *obj, uint8_t verbose); +void rsgt_printBLOCK_SIG(FILE *fp, block_sig_t *bs, uint8_t verbose); +int rsgt_getBlockParams(FILE *fp, uint8_t bRewind, block_sig_t **bs, uint8_t *bHasRecHashes, uint8_t *bHasIntermedHashes); +int rsgt_chkFileHdr(FILE *fp, char *expect); +gtfile rsgt_vrfyConstruct_gf(void); +void rsgt_vrfyBlkInit(gtfile gf, block_sig_t *bs, uint8_t bHasRecHashes, uint8_t bHasIntermedHashes); +int rsgt_vrfy_nextRec(block_sig_t *bs, gtfile gf, FILE *sigfp, FILE *nsigfp, unsigned char *rec, size_t len, gterrctx_t *ectx); +int verifyBLOCK_SIG(block_sig_t *bs, gtfile gf, FILE *sigfp, FILE *nsigfp, uint8_t bExtend, gterrctx_t *ectx); +void rsgt_errctxInit(gterrctx_t *ectx); +void rsgt_errctxExit(gterrctx_t *ectx); +void rsgt_errctxSetErrRec(gterrctx_t *ectx, char *rec); +void rsgt_errctxFrstRecInBlk(gterrctx_t *ectx, char *rec); +void rsgt_objfree(uint16_t tlvtype, void *obj); + + +/* TODO: replace these? */ +int hash_m(gtfile gf, GTDataHash **m); +int hash_r(gtfile gf, GTDataHash **r, const unsigned char *rec, const size_t len); +int hash_node(gtfile gf, GTDataHash **node, GTDataHash *m, GTDataHash *r, uint8_t level); +extern char *rsgt_read_puburl; /**< url of publication server */ +extern uint8_t rsgt_read_showVerified; + +#endif /* #ifndef INCLUDED_LIBRSGT_H */ diff --git a/runtime/librsgt_read.c b/runtime/librsgt_read.c new file mode 100644 index 00000000..25c0db4d --- /dev/null +++ b/runtime/librsgt_read.c @@ -0,0 +1,1092 @@ +/* librsgt_read.c - rsyslog's guardtime support library + * This includes functions used for reading signature (and + * other related) files. Well, actually it also contains + * some writing functionality, but only as far as rsyslog + * itself is not concerned, but "just" the utility programs. + * + * This part of the library uses C stdio and expects that the + * caller will open and close the file to be read itself. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdio.h> +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <stdint.h> +#include <assert.h> +#include <unistd.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <gt_http.h> + +#include "librsgt.h" + +typedef unsigned char uchar; +#ifndef VERSION +#define VERSION "no-version" +#endif +#define MAXFNAME 1024 + +static int rsgt_read_debug = 0; +char *rsgt_read_puburl = "http://verify.guardtime.com/gt-controlpublications.bin"; +char *rsgt_extend_puburl = "http://verifier.guardtime.net/gt-extendingservice"; +uint8_t rsgt_read_showVerified = 0; + +/* macro to obtain next char from file including error tracking */ +#define NEXTC if((c = fgetc(fp)) == EOF) { \ + r = feof(fp) ? RSGTE_EOF : RSGTE_IO; \ + goto done; \ + } + +/* check return state of operation and abort, if non-OK */ +#define CHKr(code) if((r = code) != 0) goto done + + +/* if verbose==0, only the first and last two octets are shown, + * otherwise everything. + */ +static void +outputHexBlob(FILE *fp, uint8_t *blob, uint16_t len, uint8_t verbose) +{ + unsigned i; + if(verbose || len <= 8) { + for(i = 0 ; i < len ; ++i) + fprintf(fp, "%2.2x", blob[i]); + } else { + fprintf(fp, "%2.2x%2.2x%2.2x[...]%2.2x%2.2x%2.2x", + blob[0], blob[1], blob[2], + blob[len-3], blob[len-2], blob[len-1]); + } +} + +static inline void +outputHash(FILE *fp, char *hdr, uint8_t *data, uint16_t len, uint8_t verbose) +{ + fprintf(fp, "%s", hdr); + outputHexBlob(fp, data, len, verbose); + fputc('\n', fp); +} + +void +rsgt_errctxInit(gterrctx_t *ectx) +{ + ectx->fp = NULL; + ectx->filename = NULL; + ectx->recNum = 0; + ectx->gtstate = 0; + ectx->recNumInFile = 0; + ectx->blkNum = 0; + ectx->verbose = 0; + ectx->errRec = NULL; + ectx->frstRecInBlk = NULL; + ectx->fileHash = NULL; + ectx->lefthash = ectx->righthash = ectx->computedHash = NULL; +} +void +rsgt_errctxExit(gterrctx_t *ectx) +{ + free(ectx->filename); + free(ectx->frstRecInBlk); +} + +/* note: we do not copy the record, so the caller MUST not destruct + * it before processing of the record is completed. To remove the + * current record without setting a new one, call this function + * with rec==NULL. + */ +void +rsgt_errctxSetErrRec(gterrctx_t *ectx, char *rec) +{ + ectx->errRec = strdup(rec); +} +/* This stores the block's first record. Here we copy the data, + * as the caller will usually not preserve it long enough. + */ +void +rsgt_errctxFrstRecInBlk(gterrctx_t *ectx, char *rec) +{ + free(ectx->frstRecInBlk); + ectx->frstRecInBlk = strdup(rec); +} + +static void +reportError(int errcode, gterrctx_t *ectx) +{ + if(ectx->fp != NULL) { + fprintf(ectx->fp, "%s[%llu:%llu:%llu]: error[%u]: %s\n", + ectx->filename, + (long long unsigned) ectx->blkNum, (long long unsigned) ectx->recNum, + (long long unsigned) ectx->recNumInFile, + errcode, RSGTE2String(errcode)); + if(ectx->frstRecInBlk != NULL) + fprintf(ectx->fp, "\tBlock Start Record.: '%s'\n", ectx->frstRecInBlk); + if(ectx->errRec != NULL) + fprintf(ectx->fp, "\tRecord in Question.: '%s'\n", ectx->errRec); + if(ectx->computedHash != NULL) { + outputHash(ectx->fp, "\tComputed Hash......: ", ectx->computedHash->digest, + ectx->computedHash->digest_length, ectx->verbose); + } + if(ectx->fileHash != NULL) { + outputHash(ectx->fp, "\tSignature File Hash: ", ectx->fileHash->data, + ectx->fileHash->len, ectx->verbose); + } + if(errcode == RSGTE_INVLD_TREE_HASH || + errcode == RSGTE_INVLD_TREE_HASHID) { + fprintf(ectx->fp, "\tTree Level.........: %d\n", (int) ectx->treeLevel); + outputHash(ectx->fp, "\tTree Left Hash.....: ", ectx->lefthash->digest, + ectx->lefthash->digest_length, ectx->verbose); + outputHash(ectx->fp, "\tTree Right Hash....: ", ectx->righthash->digest, + ectx->righthash->digest_length, ectx->verbose); + } + if(errcode == RSGTE_INVLD_TIMESTAMP || + errcode == RSGTE_TS_DERDECODE) { + fprintf(ectx->fp, "\tPublication Server.: %s\n", rsgt_read_puburl); + fprintf(ectx->fp, "\tGT Verify Timestamp: [%u]%s\n", + ectx->gtstate, GTHTTP_getErrorString(ectx->gtstate)); + } + if(errcode == RSGTE_TS_EXTEND || + errcode == RSGTE_TS_DERDECODE) { + fprintf(ectx->fp, "\tExtending Server...: %s\n", rsgt_extend_puburl); + fprintf(ectx->fp, "\tGT Extend Timestamp: [%u]%s\n", + ectx->gtstate, GTHTTP_getErrorString(ectx->gtstate)); + } + if(errcode == RSGTE_TS_DERENCODE) { + fprintf(ectx->fp, "\tAPI return state...: [%u]%s\n", + ectx->gtstate, GTHTTP_getErrorString(ectx->gtstate)); + } + } +} + +/* obviously, this is not an error-reporting function. We still use + * ectx, as it has most information we need. + */ +static void +reportVerifySuccess(gterrctx_t *ectx, GTVerificationInfo *vrfyInf) +{ + if(ectx->fp != NULL) { + fprintf(ectx->fp, "%s[%llu:%llu:%llu]: block signature successfully verified\n", + ectx->filename, + (long long unsigned) ectx->blkNum, (long long unsigned) ectx->recNum, + (long long unsigned) ectx->recNumInFile); + if(ectx->frstRecInBlk != NULL) + fprintf(ectx->fp, "\tBlock Start Record.: '%s'\n", ectx->frstRecInBlk); + if(ectx->errRec != NULL) + fprintf(ectx->fp, "\tBlock End Record...: '%s'\n", ectx->errRec); + fprintf(ectx->fp, "\tGT Verify Timestamp: [%u]%s\n", + ectx->gtstate, GTHTTP_getErrorString(ectx->gtstate)); + GTVerificationInfo_print(ectx->fp, 0, vrfyInf); + } +} + +/** + * Write the provided record to the current file position. + * + * @param[in] fp file pointer for writing + * @param[out] rec tlvrecord to write + * + * @returns 0 if ok, something else otherwise + */ +static int +rsgt_tlvwrite(FILE *fp, tlvrecord_t *rec) +{ + int r = RSGTE_IO; + if(fwrite(rec->hdr, (size_t) rec->lenHdr, 1, fp) != 1) goto done; + if(fwrite(rec->data, (size_t) rec->tlvlen, 1, fp) != 1) goto done; + r = 0; +done: return r; +} + +/** + * Read a header from a binary file. + * @param[in] fp file pointer for processing + * @param[in] hdr buffer for the header. Must be 9 bytes + * (8 for header + NUL byte) + * @returns 0 if ok, something else otherwise + */ +int +rsgt_tlvrdHeader(FILE *fp, uchar *hdr) +{ + int r; + if(fread(hdr, 8, 1, fp) != 1) { + r = RSGTE_IO; + goto done; + } + hdr[8] = '\0'; + r = 0; +done: return r; +} + +/* read type a complete tlv record + */ +static int +rsgt_tlvRecRead(FILE *fp, tlvrecord_t *rec) +{ + int r = 1; + int c; + + NEXTC; + rec->hdr[0] = c; + rec->tlvtype = c & 0x1f; + if(c & 0x20) { /* tlv16? */ + rec->lenHdr = 4; + NEXTC; + rec->hdr[1] = c; + rec->tlvtype = (rec->tlvtype << 8) | c; + NEXTC; + rec->hdr[2] = c; + rec->tlvlen = c << 8; + NEXTC; + rec->hdr[3] = c; + rec->tlvlen |= c; + } else { + NEXTC; + rec->lenHdr = 2; + rec->hdr[1] = c; + rec->tlvlen = c; + } + if(fread(rec->data, (size_t) rec->tlvlen, 1, fp) != 1) { + r = RSGTE_IO; + goto done; + } + + if(rsgt_read_debug) + printf("read tlvtype %4.4x, len %u\n", (unsigned) rec->tlvtype, + (unsigned) rec->tlvlen); + r = 0; +done: return r; +} + +/* decode a sub-tlv record from an existing record's memory buffer + */ +static int +rsgt_tlvDecodeSUBREC(tlvrecord_t *rec, uint16_t *stridx, tlvrecord_t *newrec) +{ + int r = 1; + int c; + + if(rec->tlvlen == *stridx) {r=RSGTE_LEN; goto done;} + c = rec->data[(*stridx)++]; + newrec->hdr[0] = c; + newrec->tlvtype = c & 0x1f; + if(c & 0x20) { /* tlv16? */ + newrec->lenHdr = 4; + if(rec->tlvlen == *stridx) {r=RSGTE_LEN; goto done;} + c = rec->data[(*stridx)++]; + newrec->hdr[1] = c; + newrec->tlvtype = (newrec->tlvtype << 8) | c; + if(rec->tlvlen == *stridx) {r=RSGTE_LEN; goto done;} + c = rec->data[(*stridx)++]; + newrec->hdr[2] = c; + newrec->tlvlen = c << 8; + if(rec->tlvlen == *stridx) {r=RSGTE_LEN; goto done;} + c = rec->data[(*stridx)++]; + newrec->hdr[3] = c; + newrec->tlvlen |= c; + } else { + if(rec->tlvlen == *stridx) {r=RSGTE_LEN; goto done;} + c = rec->data[(*stridx)++]; + newrec->lenHdr = 2; + newrec->hdr[1] = c; + newrec->tlvlen = c; + } + if(rec->tlvlen < *stridx + newrec->tlvlen) {r=RSGTE_LEN; goto done;} + memcpy(newrec->data, (rec->data)+(*stridx), newrec->tlvlen); + *stridx += newrec->tlvlen; + + if(rsgt_read_debug) + printf("read sub-tlv: tlvtype %4.4x, len %u\n", + (unsigned) newrec->tlvtype, + (unsigned) newrec->tlvlen); + r = 0; +done: return r; +} + + +static int +rsgt_tlvDecodeIMPRINT(tlvrecord_t *rec, imprint_t **imprint) +{ + int r = 1; + imprint_t *imp; + + if((imp = calloc(1, sizeof(imprint_t))) == NULL) { + r = RSGTE_OOM; + goto done; + } + + imp->hashID = rec->data[0]; + if(rec->tlvlen != 1 + hashOutputLengthOctets(imp->hashID)) { + r = RSGTE_LEN; + goto done; + } + imp->len = rec->tlvlen - 1; + if((imp->data = (uint8_t*)malloc(imp->len)) == NULL) {r=RSGTE_OOM;goto done;} + memcpy(imp->data, rec->data+1, imp->len); + *imprint = imp; + r = 0; +done: return r; +} + +static int +rsgt_tlvDecodeHASH_ALGO(tlvrecord_t *rec, uint16_t *strtidx, uint8_t *hashAlg) +{ + int r = 1; + tlvrecord_t subrec; + + CHKr(rsgt_tlvDecodeSUBREC(rec, strtidx, &subrec)); + if(!(subrec.tlvtype == 0x00 && subrec.tlvlen == 1)) { + r = RSGTE_FMT; + goto done; + } + *hashAlg = subrec.data[0]; + r = 0; +done: return r; +} +static int +rsgt_tlvDecodeBLOCK_IV(tlvrecord_t *rec, uint16_t *strtidx, uint8_t **iv) +{ + int r = 1; + tlvrecord_t subrec; + + CHKr(rsgt_tlvDecodeSUBREC(rec, strtidx, &subrec)); + if(!(subrec.tlvtype == 0x01)) { + r = RSGTE_INVLTYP; + goto done; + } + if((*iv = (uint8_t*)malloc(subrec.tlvlen)) == NULL) {r=RSGTE_OOM;goto done;} + memcpy(*iv, subrec.data, subrec.tlvlen); + r = 0; +done: return r; +} +static int +rsgt_tlvDecodeLAST_HASH(tlvrecord_t *rec, uint16_t *strtidx, imprint_t *imp) +{ + int r = 1; + tlvrecord_t subrec; + + CHKr(rsgt_tlvDecodeSUBREC(rec, strtidx, &subrec)); + if(!(subrec.tlvtype == 0x02)) { r = RSGTE_INVLTYP; goto done; } + imp->hashID = subrec.data[0]; + if(subrec.tlvlen != 1 + hashOutputLengthOctets(imp->hashID)) { + r = RSGTE_LEN; + goto done; + } + imp->len = subrec.tlvlen - 1; + if((imp->data = (uint8_t*)malloc(imp->len)) == NULL) {r=RSGTE_OOM;goto done;} + memcpy(imp->data, subrec.data+1, subrec.tlvlen-1); + r = 0; +done: return r; +} +static int +rsgt_tlvDecodeREC_COUNT(tlvrecord_t *rec, uint16_t *strtidx, uint64_t *cnt) +{ + int r = 1; + int i; + uint64_t val; + tlvrecord_t subrec; + + CHKr(rsgt_tlvDecodeSUBREC(rec, strtidx, &subrec)); + if(!(subrec.tlvtype == 0x03 && subrec.tlvlen <= 8)) { r = RSGTE_INVLTYP; goto done; } + val = 0; + for(i = 0 ; i < subrec.tlvlen ; ++i) { + val = (val << 8) + subrec.data[i]; + } + *cnt = val; + r = 0; +done: return r; +} +static int +rsgt_tlvDecodeSIG(tlvrecord_t *rec, uint16_t *strtidx, block_sig_t *bs) +{ + int r = 1; + tlvrecord_t subrec; + + CHKr(rsgt_tlvDecodeSUBREC(rec, strtidx, &subrec)); + if(!(subrec.tlvtype == 0x0906)) { r = RSGTE_INVLTYP; goto done; } + bs->sig.der.len = subrec.tlvlen; + bs->sigID = SIGID_RFC3161; + if((bs->sig.der.data = (uint8_t*)malloc(bs->sig.der.len)) == NULL) {r=RSGTE_OOM;goto done;} + memcpy(bs->sig.der.data, subrec.data, bs->sig.der.len); + r = 0; +done: return r; +} + +static int +rsgt_tlvDecodeBLOCK_SIG(tlvrecord_t *rec, block_sig_t **blocksig) +{ + int r = 1; + uint16_t strtidx = 0; + block_sig_t *bs; + if((bs = calloc(1, sizeof(block_sig_t))) == NULL) { + r = RSGTE_OOM; + goto done; + } + CHKr(rsgt_tlvDecodeHASH_ALGO(rec, &strtidx, &(bs->hashID))); + CHKr(rsgt_tlvDecodeBLOCK_IV(rec, &strtidx, &(bs->iv))); + CHKr(rsgt_tlvDecodeLAST_HASH(rec, &strtidx, &(bs->lastHash))); + CHKr(rsgt_tlvDecodeREC_COUNT(rec, &strtidx, &(bs->recCount))); + CHKr(rsgt_tlvDecodeSIG(rec, &strtidx, bs)); + if(strtidx != rec->tlvlen) { + r = RSGTE_LEN; + goto done; + } + *blocksig = bs; + r = 0; +done: return r; +} +static int +rsgt_tlvRecDecode(tlvrecord_t *rec, void *obj) +{ + int r = 1; + switch(rec->tlvtype) { + case 0x0900: + case 0x0901: + r = rsgt_tlvDecodeIMPRINT(rec, obj); + if(r != 0) goto done; + break; + case 0x0902: + r = rsgt_tlvDecodeBLOCK_SIG(rec, obj); + if(r != 0) goto done; + break; + } +done: + return r; +} + +static int +rsgt_tlvrdRecHash(FILE *fp, FILE *outfp, imprint_t **imp) +{ + int r; + tlvrecord_t rec; + + if((r = rsgt_tlvrd(fp, &rec, imp)) != 0) goto done; + if(rec.tlvtype != 0x0900) { + r = RSGTE_MISS_REC_HASH; + rsgt_objfree(rec.tlvtype, *imp); + goto done; + } + if(outfp != NULL) + if((r = rsgt_tlvwrite(outfp, &rec)) != 0) goto done; + r = 0; +done: return r; +} + +static int +rsgt_tlvrdTreeHash(FILE *fp, FILE *outfp, imprint_t **imp) +{ + int r; + tlvrecord_t rec; + + if((r = rsgt_tlvrd(fp, &rec, imp)) != 0) goto done; + if(rec.tlvtype != 0x0901) { + r = RSGTE_MISS_TREE_HASH; + rsgt_objfree(rec.tlvtype, *imp); + goto done; + } + if(outfp != NULL) + if((r = rsgt_tlvwrite(outfp, &rec)) != 0) goto done; + r = 0; +done: return r; +} + +/* read BLOCK_SIG during verification phase */ +static int +rsgt_tlvrdVrfyBlockSig(FILE *fp, block_sig_t **bs, tlvrecord_t *rec) +{ + int r; + + if((r = rsgt_tlvrd(fp, rec, bs)) != 0) goto done; + if(rec->tlvtype != 0x0902) { + r = RSGTE_MISS_BLOCKSIG; + rsgt_objfree(rec->tlvtype, *bs); + goto done; + } + r = 0; +done: return r; +} + +/** + * Read the next "object" from file. This usually is + * a single TLV, but may be something larger, for + * example in case of a block-sig TLV record. + * Unknown type records are ignored (or run aborted + * if we are not permitted to skip). + * + * @param[in] fp file pointer for processing + * @param[out] tlvtype type of tlv record (top-level for + * structured objects. + * @param[out] tlvlen length of the tlv record value + * @param[out] obj pointer to object; This is a proper + * tlv record structure, which must be casted + * by the caller according to the reported type. + * The object must be freed by the caller (TODO: better way?) + * + * @returns 0 if ok, something else otherwise + */ +int +rsgt_tlvrd(FILE *fp, tlvrecord_t *rec, void *obj) +{ + int r; + if((r = rsgt_tlvRecRead(fp, rec)) != 0) goto done; + r = rsgt_tlvRecDecode(rec, obj); +done: return r; +} + + +/* return if a blob is all zero */ +static inline int +blobIsZero(uint8_t *blob, uint16_t len) +{ + int i; + for(i = 0 ; i < len ; ++i) + if(blob[i] != 0) + return 0; + return 1; +} + +static void +rsgt_printIMPRINT(FILE *fp, char *name, imprint_t *imp, uint8_t verbose) +{ + fprintf(fp, "%s", name); + outputHexBlob(fp, imp->data, imp->len, verbose); + fputc('\n', fp); +} + +static void +rsgt_printREC_HASH(FILE *fp, imprint_t *imp, uint8_t verbose) +{ + rsgt_printIMPRINT(fp, "[0x0900]Record hash: ", + imp, verbose); +} + +static void +rsgt_printINT_HASH(FILE *fp, imprint_t *imp, uint8_t verbose) +{ + rsgt_printIMPRINT(fp, "[0x0901]Tree hash..: ", + imp, verbose); +} + +/** + * Output a human-readable representation of a block_sig_t + * to proviced file pointer. This function is mainly inteded for + * debugging purposes or dumping tlv files. + * + * @param[in] fp file pointer to send output to + * @param[in] bsig ponter to block_sig_t to output + * @param[in] verbose if 0, abbreviate blob hexdump, else complete + */ +void +rsgt_printBLOCK_SIG(FILE *fp, block_sig_t *bs, uint8_t verbose) +{ + fprintf(fp, "[0x0902]Block Signature Record:\n"); + fprintf(fp, "\tPrevious Block Hash:\n"); + fprintf(fp, "\t Algorithm..: %s\n", hashAlgName(bs->lastHash.hashID)); + fprintf(fp, "\t Hash.......: "); + outputHexBlob(fp, bs->lastHash.data, bs->lastHash.len, verbose); + fputc('\n', fp); + if(blobIsZero(bs->lastHash.data, bs->lastHash.len)) + fprintf(fp, "\t NOTE: New Hash Chain Start!\n"); + fprintf(fp, "\tHash Algorithm: %s\n", hashAlgName(bs->hashID)); + fprintf(fp, "\tIV............: "); + outputHexBlob(fp, bs->iv, getIVLen(bs), verbose); + fputc('\n', fp); + fprintf(fp, "\tRecord Count..: %llu\n", bs->recCount); + fprintf(fp, "\tSignature Type: %s\n", sigTypeName(bs->sigID)); + fprintf(fp, "\tSignature Len.: %u\n", bs->sig.der.len); + fprintf(fp, "\tSignature.....: "); + outputHexBlob(fp, bs->sig.der.data, bs->sig.der.len, verbose); + fputc('\n', fp); +} + + +/** + * Output a human-readable representation of a tlv object. + * + * @param[in] fp file pointer to send output to + * @param[in] tlvtype type of tlv object (record) + * @param[in] verbose if 0, abbreviate blob hexdump, else complete + */ +void +rsgt_tlvprint(FILE *fp, uint16_t tlvtype, void *obj, uint8_t verbose) +{ + switch(tlvtype) { + case 0x0900: + rsgt_printREC_HASH(fp, obj, verbose); + break; + case 0x0901: + rsgt_printINT_HASH(fp, obj, verbose); + break; + case 0x0902: + rsgt_printBLOCK_SIG(fp, obj, verbose); + break; + default:fprintf(fp, "unknown tlv record %4.4x\n", tlvtype); + break; + } +} + +/** + * Free the provided object. + * + * @param[in] tlvtype type of tlv object (record) + * @param[in] obj the object to be destructed + */ +void +rsgt_objfree(uint16_t tlvtype, void *obj) +{ + switch(tlvtype) { + case 0x0900: + case 0x0901: + free(((imprint_t*)obj)->data); + break; + case 0x0902: + free(((block_sig_t*)obj)->iv); + free(((block_sig_t*)obj)->lastHash.data); + free(((block_sig_t*)obj)->sig.der.data); + break; + default:fprintf(stderr, "rsgt_objfree: unknown tlv record %4.4x\n", + tlvtype); + break; + } + free(obj); +} + +/** + * Read block parameters. This detects if the block contains the + * individual log hashes, the intermediate hashes and the overall + * block paramters (from the signature block). As we do not have any + * begin of block record, we do not know e.g. the hash algorithm or IV + * until reading the block signature record. And because the file is + * purely sequential and variable size, we need to read all records up to + * the next signature record. + * If a caller intends to verify a log file based on the parameters, + * he must re-read the file from the begining (we could keep things + * in memory, but this is impractical for large blocks). In order + * to facitate this, the function permits to rewind to the original + * read location when it is done. + * + * @param[in] fp file pointer of tlv file + * @param[in] bRewind 0 - do not rewind at end of procesing, 1 - do so + * @param[out] bs block signature record + * @param[out] bHasRecHashes 0 if record hashes are present, 1 otherwise + * @param[out] bHasIntermedHashes 0 if intermediate hashes are present, + * 1 otherwise + * + * @returns 0 if ok, something else otherwise + */ +int +rsgt_getBlockParams(FILE *fp, uint8_t bRewind, block_sig_t **bs, + uint8_t *bHasRecHashes, uint8_t *bHasIntermedHashes) +{ + int r; + uint64_t nRecs = 0; + uint8_t bDone = 0; + off_t rewindPos = 0; + void *obj; + tlvrecord_t rec; + + if(bRewind) + rewindPos = ftello(fp); + *bHasRecHashes = 0; + *bHasIntermedHashes = 0; + *bs = NULL; + + while(!bDone) { /* we will err out on EOF */ + if((r = rsgt_tlvrd(fp, &rec, &obj)) != 0) goto done; + switch(rec.tlvtype) { + case 0x0900: + ++nRecs; + *bHasRecHashes = 1; + break; + case 0x0901: + *bHasIntermedHashes = 1; + break; + case 0x0902: + *bs = (block_sig_t*) obj; + bDone = 1; + break; + default:fprintf(fp, "unknown tlv record %4.4x\n", rec.tlvtype); + break; + } + if(!bDone) + rsgt_objfree(rec.tlvtype, obj); + } + + if(*bHasRecHashes && (nRecs != (*bs)->recCount)) { + r = RSGTE_INVLD_RECCNT; + goto done; + } + + if(bRewind) { + if(fseeko(fp, rewindPos, SEEK_SET) != 0) { + r = RSGTE_IO; + goto done; + } + } +done: + return r; +} + + +/** + * Read the file header and compare it to the expected value. + * The file pointer is placed right after the header. + * @param[in] fp file pointer of tlv file + * @param[in] excpect expected header (e.g. "LOGSIG10") + * @returns 0 if ok, something else otherwise + */ +int +rsgt_chkFileHdr(FILE *fp, char *expect) +{ + int r; + char hdr[9]; + + if((r = rsgt_tlvrdHeader(fp, (uchar*)hdr)) != 0) goto done; + if(strcmp(hdr, expect)) + r = RSGTE_INVLHDR; + else + r = 0; +done: + return r; +} + +gtfile +rsgt_vrfyConstruct_gf(void) +{ + gtfile gf; + if((gf = calloc(1, sizeof(struct gtfile_s))) == NULL) + goto done; + gf->x_prev = NULL; + +done: return gf; +} + +void +rsgt_vrfyBlkInit(gtfile gf, block_sig_t *bs, uint8_t bHasRecHashes, uint8_t bHasIntermedHashes) +{ + gf->hashAlg = hashID2Alg(bs->hashID); + gf->bKeepRecordHashes = bHasRecHashes; + gf->bKeepTreeHashes = bHasIntermedHashes; + free(gf->IV); + gf->IV = malloc(getIVLen(bs)); + memcpy(gf->IV, bs->iv, getIVLen(bs)); + free(gf->blkStrtHash); + gf->lenBlkStrtHash = bs->lastHash.len; + gf->blkStrtHash = malloc(gf->lenBlkStrtHash); + memcpy(gf->blkStrtHash, bs->lastHash.data, gf->lenBlkStrtHash); +} + +static int +rsgt_vrfy_chkRecHash(gtfile gf, FILE *sigfp, FILE *nsigfp, + GTDataHash *recHash, gterrctx_t *ectx) +{ + int r = 0; + imprint_t *imp = NULL; + + if((r = rsgt_tlvrdRecHash(sigfp, nsigfp, &imp)) != 0) + reportError(r, ectx); + goto done; + if(imp->hashID != hashIdentifier(gf->hashAlg)) { + reportError(r, ectx); + r = RSGTE_INVLD_REC_HASHID; + goto done; + } + if(memcmp(imp->data, recHash->digest, + hashOutputLengthOctets(imp->hashID))) { + r = RSGTE_INVLD_REC_HASH; + ectx->computedHash = recHash; + ectx->fileHash = imp; + reportError(r, ectx); + ectx->computedHash = NULL, ectx->fileHash = NULL; + goto done; + } + r = 0; +done: + if(imp != NULL) + rsgt_objfree(0x0900, imp); + return r; +} + +static int +rsgt_vrfy_chkTreeHash(gtfile gf, FILE *sigfp, FILE *nsigfp, + GTDataHash *hash, gterrctx_t *ectx) +{ + int r = 0; + imprint_t *imp = NULL; + + if((r = rsgt_tlvrdTreeHash(sigfp, nsigfp, &imp)) != 0) { + reportError(r, ectx); + goto done; + } + if(imp->hashID != hashIdentifier(gf->hashAlg)) { + reportError(r, ectx); + r = RSGTE_INVLD_TREE_HASHID; + goto done; + } + if(memcmp(imp->data, hash->digest, + hashOutputLengthOctets(imp->hashID))) { + r = RSGTE_INVLD_TREE_HASH; + ectx->computedHash = hash; + ectx->fileHash = imp; + reportError(r, ectx); + ectx->computedHash = NULL, ectx->fileHash = NULL; + goto done; + } + r = 0; +done: + if(imp != NULL) + rsgt_objfree(0x0901, imp); + return r; +} + +int +rsgt_vrfy_nextRec(block_sig_t *bs, gtfile gf, FILE *sigfp, FILE *nsigfp, + unsigned char *rec, size_t len, gterrctx_t *ectx) +{ + int r = 0; + GTDataHash *x; /* current hash */ + GTDataHash *m, *recHash = NULL, *t, *t_del; + uint8_t j; + + hash_m(gf, &m); + hash_r(gf, &recHash, rec, len); + if(gf->bKeepRecordHashes) { + r = rsgt_vrfy_chkRecHash(gf, sigfp, nsigfp, recHash, ectx); + if(r != 0) goto done; + } + hash_node(gf, &x, m, recHash, 1); /* hash leaf */ + if(gf->bKeepTreeHashes) { + ectx->treeLevel = 0; + ectx->lefthash = m; + ectx->righthash = recHash; + r = rsgt_vrfy_chkTreeHash(gf, sigfp, nsigfp, x, ectx); + if(r != 0) goto done; + } + rsgtimprintDel(gf->x_prev); + gf->x_prev = rsgtImprintFromGTDataHash(x); + /* add x to the forest as new leaf, update roots list */ + t = x; + for(j = 0 ; j < gf->nRoots ; ++j) { + if(gf->roots_valid[j] == 0) { + gf->roots_hash[j] = t; + gf->roots_valid[j] = 1; + t = NULL; + break; + } else if(t != NULL) { + /* hash interim node */ + ectx->treeLevel = j+1; + ectx->righthash = t; + t_del = t; + hash_node(gf, &t, gf->roots_hash[j], t_del, j+2); + gf->roots_valid[j] = 0; + if(gf->bKeepTreeHashes) { + ectx->lefthash = gf->roots_hash[j]; + r = rsgt_vrfy_chkTreeHash(gf, sigfp, nsigfp, t, ectx); + if(r != 0) goto done; /* mem leak ok, we terminate! */ + } + GTDataHash_free(gf->roots_hash[j]); + GTDataHash_free(t_del); + } + } + if(t != NULL) { + /* new level, append "at the top" */ + gf->roots_hash[gf->nRoots] = t; + gf->roots_valid[gf->nRoots] = 1; + ++gf->nRoots; + assert(gf->nRoots < MAX_ROOTS); + t = NULL; + } + ++gf->nRecords; + + /* cleanup */ + GTDataHash_free(m); +done: + if(recHash != NULL) + GTDataHash_free(recHash); + return r; +} + + +/* TODO: think about merging this with the writer. The + * same applies to the other computation algos. + */ +static int +verifySigblkFinish(gtfile gf, GTDataHash **pRoot) +{ + GTDataHash *root, *rootDel; + int8_t j; + int r; + + if(gf->nRecords == 0) + goto done; + + root = NULL; + for(j = 0 ; j < gf->nRoots ; ++j) { + if(root == NULL) { + root = gf->roots_valid[j] ? gf->roots_hash[j] : NULL; + gf->roots_valid[j] = 0; /* guess this is redundant with init, maybe del */ + } else if(gf->roots_valid[j]) { + rootDel = root; + hash_node(gf, &root, gf->roots_hash[j], root, j+2); + gf->roots_valid[j] = 0; /* guess this is redundant with init, maybe del */ + GTDataHash_free(rootDel); + } + } + + free(gf->blkStrtHash); + gf->blkStrtHash = NULL; + *pRoot = root; + r = 0; +done: + gf->bInBlk = 0; + return r; +} + + +/* helper for rsgt_extendSig: */ +#define COPY_SUBREC_TO_NEWREC \ + memcpy(newrec.data+iWr, subrec.hdr, subrec.lenHdr); \ + iWr += subrec.lenHdr; \ + memcpy(newrec.data+iWr, subrec.data, subrec.tlvlen); \ + iWr += subrec.tlvlen; +static inline int +rsgt_extendSig(GTTimestamp *timestamp, tlvrecord_t *rec, gterrctx_t *ectx) +{ + GTTimestamp *out_timestamp; + uint8_t *der; + size_t lenDer; + int r, rgt; + tlvrecord_t newrec, subrec; + uint16_t iRd, iWr; + + rgt = GTHTTP_extendTimestamp(timestamp, rsgt_extend_puburl, &out_timestamp); + if(rgt != GT_OK) { + ectx->gtstate = rgt; + r = RSGTE_TS_EXTEND; + goto done; + } + r = GTTimestamp_getDEREncoded(out_timestamp, &der, &lenDer); + if(r != GT_OK) { + r = RSGTE_TS_DERENCODE; + ectx->gtstate = rgt; + goto done; + } + /* update block_sig tlv record with new extended timestamp */ + /* we now need to copy all tlv records before the actual der + * encoded part. + */ + iRd = iWr = 0; + // TODO; check tlvtypes at comment places below! + if ((r = rsgt_tlvDecodeSUBREC(rec, &iRd, &subrec)) != 0) goto done; + /* HASH_ALGO */ + COPY_SUBREC_TO_NEWREC + if ((r = rsgt_tlvDecodeSUBREC(rec, &iRd, &subrec)) != 0) goto done; + /* BLOCK_IV */ + COPY_SUBREC_TO_NEWREC + if ((r = rsgt_tlvDecodeSUBREC(rec, &iRd, &subrec)) != 0) goto done; + /* LAST_HASH */ + COPY_SUBREC_TO_NEWREC + if ((r = rsgt_tlvDecodeSUBREC(rec, &iRd, &subrec)) != 0) goto done; + /* REC_COUNT */ + COPY_SUBREC_TO_NEWREC + if ((r = rsgt_tlvDecodeSUBREC(rec, &iRd, &subrec)) != 0) goto done; + /* actual sig! */ + newrec.data[iWr++] = 0x09 | RSGT_FLAG_TLV16; + newrec.data[iWr++] = 0x06; + newrec.data[iWr++] = (lenDer >> 8) & 0xff; + newrec.data[iWr++] = lenDer & 0xff; + /* now we know how large the new main record is */ + newrec.tlvlen = (uint16_t) iWr+lenDer; + newrec.tlvtype = rec->tlvtype; + newrec.hdr[0] = rec->hdr[0]; + newrec.hdr[1] = rec->hdr[1]; + newrec.hdr[2] = (newrec.tlvlen >> 8) & 0xff; + newrec.hdr[3] = newrec.tlvlen & 0xff; + newrec.lenHdr = 4; + memcpy(newrec.data+iWr, der, lenDer); + /* and finally copy back new record to existing one */ + memcpy(rec, &newrec, sizeof(newrec)-sizeof(newrec.data)+newrec.tlvlen+4); + r = 0; +done: + return r; +} + + +/* verify the root hash. This also means we need to compute the + * Merkle tree root for the current block. + */ +int +verifyBLOCK_SIG(block_sig_t *bs, gtfile gf, FILE *sigfp, FILE *nsigfp, + uint8_t bExtend, gterrctx_t *ectx) +{ + int r; + int gtstate; + block_sig_t *file_bs = NULL; + GTTimestamp *timestamp = NULL; + GTVerificationInfo *vrfyInf; + GTDataHash *root = NULL; + tlvrecord_t rec; + + if((r = verifySigblkFinish(gf, &root)) != 0) + goto done; + if((r = rsgt_tlvrdVrfyBlockSig(sigfp, &file_bs, &rec)) != 0) + goto done; + if(ectx->recNum != bs->recCount) { + r = RSGTE_INVLD_RECCNT; + goto done; + } + + gtstate = GTTimestamp_DERDecode(file_bs->sig.der.data, + file_bs->sig.der.len, ×tamp); + if(gtstate != GT_OK) { + r = RSGTE_TS_DERDECODE; + ectx->gtstate = gtstate; + goto done; + } + + gtstate = GTHTTP_verifyTimestampHash(timestamp, root, NULL, + NULL, NULL, rsgt_read_puburl, 0, &vrfyInf); + if(! (gtstate == GT_OK + && vrfyInf->verification_errors == GT_NO_FAILURES) ) { + r = RSGTE_INVLD_TIMESTAMP; + ectx->gtstate = gtstate; + goto done; + } + + if(rsgt_read_showVerified) + reportVerifySuccess(ectx, vrfyInf); + if(bExtend) + if((r = rsgt_extendSig(timestamp, &rec, ectx)) != 0) goto done; + + if(nsigfp != NULL) + if((r = rsgt_tlvwrite(nsigfp, &rec)) != 0) goto done; + r = 0; +done: + if(file_bs != NULL) + rsgt_objfree(0x0902, file_bs); + if(r != 0) + reportError(r, ectx); + if(timestamp != NULL) + GTTimestamp_free(timestamp); + return r; +} diff --git a/runtime/lmcry_gcry.c b/runtime/lmcry_gcry.c new file mode 100644 index 00000000..2e4cfff3 --- /dev/null +++ b/runtime/lmcry_gcry.c @@ -0,0 +1,290 @@ +/* lmcry_gcry.c + * + * An implementation of the cryprov interface for libgcrypt. + * + * Copyright 2013 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" + +#include "rsyslog.h" +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +#include "module-template.h" +#include "glbl.h" +#include "errmsg.h" +#include "cryprov.h" +#include "libgcry.h" +#include "lmcry_gcry.h" + +MODULE_TYPE_LIB +MODULE_TYPE_NOKEEP + +/* static data */ +DEFobjStaticHelpers +DEFobjCurrIf(errmsg) +DEFobjCurrIf(glbl) + +/* tables for interfacing with the v6 config system */ +static struct cnfparamdescr cnfpdescr[] = { + { "cry.key", eCmdHdlrGetWord, 0 }, + { "cry.keyfile", eCmdHdlrGetWord, 0 }, + { "cry.keyprogram", eCmdHdlrGetWord, 0 }, + { "cry.mode", eCmdHdlrGetWord, 0 }, /* CBC, ECB, etc */ + { "cry.algo", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk pblk = + { CNFPARAMBLK_VERSION, + sizeof(cnfpdescr)/sizeof(struct cnfparamdescr), + cnfpdescr + }; + + +#if 0 +static void +errfunc(__attribute__((unused)) void *usrptr, uchar *emsg) +{ + errmsg.LogError(0, RS_RET_CRYPROV_ERR, "Crypto Provider" + "Error: %s - disabling encryption", emsg); +} +#endif + +/* Standard-Constructor + */ +BEGINobjConstruct(lmcry_gcry) + dbgprintf("DDDD: lmcry_gcry: called construct\n"); + pThis->ctx = gcryCtxNew(); +ENDobjConstruct(lmcry_gcry) + + +/* destructor for the lmcry_gcry object */ +BEGINobjDestruct(lmcry_gcry) /* be sure to specify the object type also in END and CODESTART macros! */ +CODESTARTobjDestruct(lmcry_gcry) + dbgprintf("DDDD: lmcry_gcry: called destruct\n"); + rsgcryCtxDel(pThis->ctx); +ENDobjDestruct(lmcry_gcry) + + +/* apply all params from param block to us. This must be called + * after construction, but before the OnFileOpen() entry point. + * Defaults are expected to have been set during construction. + */ +static rsRetVal +SetCnfParam(void *pT, struct nvlst *lst) +{ + lmcry_gcry_t *pThis = (lmcry_gcry_t*) pT; + int i, r; + unsigned keylen; + uchar *key = NULL; + uchar *keyfile = NULL; + uchar *keyprogram = NULL; + uchar *algo = NULL; + uchar *mode = NULL; + int nKeys; /* number of keys (actually methods) specified */ + struct cnfparamvals *pvals; + DEFiRet; + + nKeys = 0; + pvals = nvlstGetParams(lst, &pblk, NULL); + if(Debug) { + dbgprintf("param blk in lmcry_gcry:\n"); + cnfparamsPrint(&pblk, pvals); + } + + for(i = 0 ; i < pblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblk.descr[i].name, "cry.key")) { + key = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + ++nKeys; + } else if(!strcmp(pblk.descr[i].name, "cry.keyfile")) { + keyfile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + ++nKeys; + } else if(!strcmp(pblk.descr[i].name, "cry.keyprogram")) { + keyprogram = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + ++nKeys; + } else if(!strcmp(pblk.descr[i].name, "cry.mode")) { + mode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblk.descr[i].name, "cry.algo")) { + algo = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + DBGPRINTF("lmcry_gcry: program error, non-handled " + "param '%s'\n", pblk.descr[i].name); + } + } + if(algo != NULL) { + iRet = rsgcrySetAlgo(pThis->ctx, algo); + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "cry.algo '%s' is not know/supported", algo); + FINALIZE; + } + } + if(mode != NULL) { + iRet = rsgcrySetMode(pThis->ctx, mode); + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "cry.mode '%s' is not know/supported", mode); + FINALIZE; + } + } + /* note: key must be set AFTER algo/mode is set (as it depends on them) */ + if(nKeys != 1) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "excactly one of the following " + "parameters can be specified: cry.key, cry.keyfile, cry.keyprogram\n"); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + if(key != NULL) { + errmsg.LogError(0, RS_RET_ERR, "Note: specifying an actual key directly from the " + "config file is highly insecure - DO NOT USE FOR PRODUCTION"); + keylen = strlen((char*)key); + } + if(keyfile != NULL) { + r = gcryGetKeyFromFile((char*)keyfile, (char**)&key, &keylen); + if(r != 0) { + errmsg.LogError(0, RS_RET_ERR, "error %d reading keyfile %s\n", + r, keyfile); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + } + if(keyprogram != NULL) { + r = gcryGetKeyFromProg((char*)keyprogram, (char**)&key, &keylen); + if(r != 0) { + errmsg.LogError(0, RS_RET_ERR, "error %d obtaining key from program %s\n", + r, keyprogram); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + } + + /* if we reach this point, we have a valid key */ + r = rsgcrySetKey(pThis->ctx, key, keylen); + if(r > 0) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "Key length %d expected, but " + "key of length %d given", r, keylen); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + + cnfparamvalsDestruct(pvals, &pblk); + if(key != NULL) { + memset(key, 0, strlen((char*)key)); + free(key); + } + free(keyfile); + free(algo); + free(mode); +finalize_it: + RETiRet; +} + + +static rsRetVal +OnFileOpen(void *pT, uchar *fn, void *pGF) +{ + lmcry_gcry_t *pThis = (lmcry_gcry_t*) pT; + gcryfile *pgf = (gcryfile*) pGF; + DEFiRet; +dbgprintf("DDDD: cry: onFileOpen: %s\n", fn); + + CHKiRet(rsgcryInitCrypt(pThis->ctx, pgf, fn)); +finalize_it: + /* TODO: enable this error message (need to cleanup loop first ;)) + errmsg.LogError(0, iRet, "Encryption Provider" + "Error: cannot open .encinfo file - disabling log file"); + */ + RETiRet; +} + +static rsRetVal +Encrypt(void *pF, uchar *rec, size_t *lenRec) +{ + DEFiRet; +dbgprintf("DDDD: Encrypt (%u): %s\n", *lenRec-1, rec); + iRet = rsgcryEncrypt(pF, rec, lenRec); + + RETiRet; +} + +static rsRetVal +OnFileClose(void *pF, off64_t offsLogfile) +{ + DEFiRet; +dbgprintf("DDDD: onFileClose\n"); + gcryfileDestruct(pF, offsLogfile); + + RETiRet; +} + +BEGINobjQueryInterface(lmcry_gcry) +CODESTARTobjQueryInterface(lmcry_gcry) + if(pIf->ifVersion != cryprovCURR_IF_VERSION) {/* check for current version, increment on each change */ + ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); + } + pIf->Construct = (rsRetVal(*)(void*)) lmcry_gcryConstruct; + pIf->SetCnfParam = SetCnfParam; + pIf->Destruct = (rsRetVal(*)(void*)) lmcry_gcryDestruct; + pIf->OnFileOpen = OnFileOpen; + pIf->Encrypt = Encrypt; + pIf->OnFileClose = OnFileClose; +finalize_it: +ENDobjQueryInterface(lmcry_gcry) + + +BEGINObjClassExit(lmcry_gcry, OBJ_IS_LOADABLE_MODULE) /* CHANGE class also in END MACRO! */ +CODESTARTObjClassExit(lmcry_gcry) + /* release objects we no longer need */ + objRelease(errmsg, CORE_COMPONENT); + objRelease(glbl, CORE_COMPONENT); + + rsgcryExit(); +ENDObjClassExit(lmcry_gcry) + + +BEGINObjClassInit(lmcry_gcry, 1, OBJ_IS_LOADABLE_MODULE) /* class, version */ + /* request objects we use */ + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + + if(rsgcryInit() != 0) { + errmsg.LogError(0, RS_RET_CRYPROV_ERR, "error initializing " + "crypto provider - cannot encrypt"); + ABORT_FINALIZE(RS_RET_CRYPROV_ERR); + } +ENDObjClassInit(lmcry_gcry) + + +/* --------------- here now comes the plumbing that makes as a library module --------------- */ + + +BEGINmodExit +CODESTARTmodExit + lmcry_gcryClassExit(); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_LIB_QUERIES +ENDqueryEtryPt + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ + /* Initialize all classes that are in our module - this includes ourselfs */ + CHKiRet(lmcry_gcryClassInit(pModInfo)); /* must be done after tcps_sess, as we use it */ +ENDmodInit diff --git a/runtime/lmcry_gcry.h b/runtime/lmcry_gcry.h new file mode 100644 index 00000000..c0205ab9 --- /dev/null +++ b/runtime/lmcry_gcry.h @@ -0,0 +1,39 @@ +/* An implementation of the cryprov interface for libgcrypt. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_LMCRY_GCRY_H +#define INCLUDED_LMCRY_GCRY_H +#include "cryprov.h" + +/* interface is defined in cryprov.h, we just implement it! */ +#define lmcry_gcryCURR_IF_VERSION cryprovCURR_IF_VERSION +typedef cryprov_if_t lmcry_gcry_if_t; + +/* the lmcry_gcry object */ +struct lmcry_gcry_s { + BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ + gcryctx ctx; +}; +typedef struct lmcry_gcry_s lmcry_gcry_t; + +/* prototypes */ +PROTOTYPEObj(lmcry_gcry); + +#endif /* #ifndef INCLUDED_LMCRY_GCRY_H */ diff --git a/runtime/lmsig_gt.c b/runtime/lmsig_gt.c new file mode 100644 index 00000000..09691292 --- /dev/null +++ b/runtime/lmsig_gt.c @@ -0,0 +1,229 @@ +/* lmsig_gt.c + * + * An implementation of the sigprov interface for GuardTime. + * + * Copyright 2013 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" + +#include "rsyslog.h" +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +#include "module-template.h" +#include "glbl.h" +#include "errmsg.h" +#include "sigprov.h" +#include "lmsig_gt.h" + +MODULE_TYPE_LIB +MODULE_TYPE_NOKEEP + +/* static data */ +DEFobjStaticHelpers +DEFobjCurrIf(errmsg) +DEFobjCurrIf(glbl) + +/* tables for interfacing with the v6 config system */ +static struct cnfparamdescr cnfpdescr[] = { + { "sig.hashfunction", eCmdHdlrGetWord, 0 }, + { "sig.timestampservice", eCmdHdlrGetWord, 0 }, + { "sig.block.sizelimit", eCmdHdlrSize, 0 }, + { "sig.keeprecordhashes", eCmdHdlrBinary, 0 }, + { "sig.keeptreehashes", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk pblk = + { CNFPARAMBLK_VERSION, + sizeof(cnfpdescr)/sizeof(struct cnfparamdescr), + cnfpdescr + }; + + +static void +errfunc(__attribute__((unused)) void *usrptr, uchar *emsg) +{ + errmsg.LogError(0, RS_RET_SIGPROV_ERR, "Signature Provider" + "Error: %s - disabling signatures", emsg); +} + +/* Standard-Constructor + */ +BEGINobjConstruct(lmsig_gt) + dbgprintf("DDDD: lmsig_gt: called construct\n"); + pThis->ctx = rsgtCtxNew(); + rsgtsetErrFunc(pThis->ctx, errfunc, NULL); +ENDobjConstruct(lmsig_gt) + + +/* destructor for the lmsig_gt object */ +BEGINobjDestruct(lmsig_gt) /* be sure to specify the object type also in END and CODESTART macros! */ +CODESTARTobjDestruct(lmsig_gt) + dbgprintf("DDDD: lmsig_gt: called destruct\n"); + rsgtCtxDel(pThis->ctx); +ENDobjDestruct(lmsig_gt) + + +/* apply all params from param block to us. This must be called + * after construction, but before the OnFileOpen() entry point. + * Defaults are expected to have been set during construction. + */ +rsRetVal +SetCnfParam(void *pT, struct nvlst *lst) +{ + lmsig_gt_t *pThis = (lmsig_gt_t*) pT; + int i; + uchar *cstr; + struct cnfparamvals *pvals; + pvals = nvlstGetParams(lst, &pblk, NULL); + if(Debug) { + dbgprintf("sig param blk in lmsig_gt:\n"); + cnfparamsPrint(&pblk, pvals); + } + + for(i = 0 ; i < pblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(pblk.descr[i].name, "sig.hashfunction")) { + cstr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + if(rsgtSetHashFunction(pThis->ctx, (char*)cstr) != 0) { + errmsg.LogError(0, RS_RET_ERR, "Hash function " + "'%s' unknown - using default", cstr); + } + free(cstr); + } else if(!strcmp(pblk.descr[i].name, "sig.timestampservice")) { + cstr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + rsgtSetTimestamper(pThis->ctx, (char*) cstr); + free(cstr); + } else if(!strcmp(pblk.descr[i].name, "sig.block.sizelimit")) { + rsgtSetBlockSizeLimit(pThis->ctx, pvals[i].val.d.n); + } else if(!strcmp(pblk.descr[i].name, "sig.keeprecordhashes")) { + rsgtSetKeepRecordHashes(pThis->ctx, pvals[i].val.d.n); + } else if(!strcmp(pblk.descr[i].name, "sig.keeptreehashes")) { + rsgtSetKeepTreeHashes(pThis->ctx, pvals[i].val.d.n); + } else { + DBGPRINTF("lmsig_gt: program error, non-handled " + "param '%s'\n", pblk.descr[i].name); + } + } + cnfparamvalsDestruct(pvals, &pblk); + return RS_RET_OK; +} + + +static rsRetVal +OnFileOpen(void *pT, uchar *fn, void *pGF) +{ + lmsig_gt_t *pThis = (lmsig_gt_t*) pT; + gtfile *pgf = (gtfile*) pGF; + DEFiRet; +dbgprintf("DDDD: onFileOpen: %s\n", fn); + /* note: if *pgf is set to NULL, this auto-disables GT functions */ + *pgf = rsgtCtxOpenFile(pThis->ctx, fn); + sigblkInit(*pgf); + RETiRet; +} + +/* Note: we assume that the record is terminated by a \n. + * As of the GuardTime paper, \n is not part of the signed + * message, so we subtract one from the record size. This + * may cause issues with non-standard formats, but let's + * see how things evolve (the verifier will not work in + * any case when the records are not \n delimited...). + * rgerhards, 2013-03-17 + */ +static rsRetVal +OnRecordWrite(void *pF, uchar *rec, rs_size_t lenRec) +{ + DEFiRet; +dbgprintf("DDDD: onRecordWrite (%d): %s\n", lenRec-1, rec); + sigblkAddRecord(pF, rec, lenRec-1); + + RETiRet; +} + +static rsRetVal +OnFileClose(void *pF) +{ + DEFiRet; +dbgprintf("DDDD: onFileClose\n"); + rsgtfileDestruct(pF); + + RETiRet; +} + +BEGINobjQueryInterface(lmsig_gt) +CODESTARTobjQueryInterface(lmsig_gt) + if(pIf->ifVersion != sigprovCURR_IF_VERSION) {/* check for current version, increment on each change */ + ABORT_FINALIZE(RS_RET_INTERFACE_NOT_SUPPORTED); + } + pIf->Construct = (rsRetVal(*)(void*)) lmsig_gtConstruct; + pIf->SetCnfParam = SetCnfParam; + pIf->Destruct = (rsRetVal(*)(void*)) lmsig_gtDestruct; + pIf->OnFileOpen = OnFileOpen; + pIf->OnRecordWrite = OnRecordWrite; + pIf->OnFileClose = OnFileClose; +finalize_it: +ENDobjQueryInterface(lmsig_gt) + + +BEGINObjClassExit(lmsig_gt, OBJ_IS_LOADABLE_MODULE) /* CHANGE class also in END MACRO! */ +CODESTARTObjClassExit(lmsig_gt) + /* release objects we no longer need */ + objRelease(errmsg, CORE_COMPONENT); + objRelease(glbl, CORE_COMPONENT); + + rsgtExit(); +ENDObjClassExit(lmsig_gt) + + +BEGINObjClassInit(lmsig_gt, 1, OBJ_IS_LOADABLE_MODULE) /* class, version */ + /* request objects we use */ + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + + if(rsgtInit("rsyslogd " VERSION) != 0) { + errmsg.LogError(0, RS_RET_SIGPROV_ERR, "error initializing " + "signature provider - cannot sign"); + ABORT_FINALIZE(RS_RET_SIGPROV_ERR); + } +ENDObjClassInit(lmsig_gt) + + +/* --------------- here now comes the plumbing that makes as a library module --------------- */ + + +BEGINmodExit +CODESTARTmodExit + lmsig_gtClassExit(); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_LIB_QUERIES +ENDqueryEtryPt + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ + /* Initialize all classes that are in our module - this includes ourselfs */ + CHKiRet(lmsig_gtClassInit(pModInfo)); /* must be done after tcps_sess, as we use it */ +ENDmodInit diff --git a/runtime/lmsig_gt.h b/runtime/lmsig_gt.h new file mode 100644 index 00000000..665e6a8e --- /dev/null +++ b/runtime/lmsig_gt.h @@ -0,0 +1,40 @@ +/* An implementation of the sigprov interface for GuardTime. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_LMSIG_GT_H +#define INCLUDED_LMSIG_GT_H +#include "sigprov.h" +#include "librsgt.h" + +/* interface is defined in sigprov.h, we just implement it! */ +#define lmsig_gtCURR_IF_VERSION sigprovCURR_IF_VERSION +typedef sigprov_if_t lmsig_gt_if_t; + +/* the lmsig_gt object */ +struct lmsig_gt_s { + BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ + gtctx ctx; /* librsgt context - contains all we need */ +}; +typedef struct lmsig_gt_s lmsig_gt_t; + +/* prototypes */ +PROTOTYPEObj(lmsig_gt); + +#endif /* #ifndef INCLUDED_LMSIG_GT_H */ diff --git a/runtime/module-template.h b/runtime/module-template.h index fe74bac9..8a958f90 100644 --- a/runtime/module-template.h +++ b/runtime/module-template.h @@ -938,6 +938,28 @@ static rsRetVal doHUP(instanceData __attribute__((unused)) *pData)\ } +/* SetShutdownImmdtPtr() + * This function is optional. If defined by an output plugin, it is called + * each time the action is invoked to set the "ShutdownImmediate" pointer, + * which is used during termination to indicate the action should shutdown + * as quickly as possible. + */ +#define CODEqueryEtryPt_SetShutdownImmdtPtr \ + else if(!strcmp((char*) name, "SetShutdownImmdtPtr")) {\ + *pEtryPoint = SetShutdownImmdtPtr;\ + } +#define BEGINSetShutdownImmdtPtr \ +static rsRetVal SetShutdownImmdtPtr(instanceData __attribute__((unused)) *pData, int *pPtr)\ +{\ + DEFiRet; + +#define CODESTARTSetShutdownImmdtPtr + +#define ENDSetShutdownImmdtPtr \ + RETiRet;\ +} + + /* parse() - main entry point of parser modules */ #define BEGINparse \ diff --git a/runtime/modules.c b/runtime/modules.c index 9f7ff31c..e9d8d959 100644 --- a/runtime/modules.c +++ b/runtime/modules.c @@ -657,6 +657,10 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ if(localRet != RS_RET_OK && localRet != RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) ABORT_FINALIZE(localRet); + localRet = (*pNew->modQueryEtryPt)((uchar*)"SetShutdownImmdtPtr", &pNew->mod.om.SetShutdownImmdtPtr); + if(localRet != RS_RET_OK && localRet != RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) + ABORT_FINALIZE(localRet); + localRet = (*pNew->modQueryEtryPt)((uchar*)"beginTransaction", &pNew->mod.om.beginTransaction); if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) pNew->mod.om.beginTransaction = dummyBeginTransaction; diff --git a/runtime/modules.h b/runtime/modules.h index e42d19e1..64644be2 100644 --- a/runtime/modules.h +++ b/runtime/modules.h @@ -142,6 +142,7 @@ struct modInfo_s { rsRetVal (*endTransaction)(void*); rsRetVal (*parseSelectorAct)(uchar**, void**,omodStringRequest_t**); rsRetVal (*newActInst)(uchar *modName, struct nvlst *lst, void **, omodStringRequest_t **); + rsRetVal (*SetShutdownImmdtPtr)(void *pData, void *pPtr); } om; struct { /* data for library modules */ char dummy; diff --git a/runtime/msg.c b/runtime/msg.c index c302a050..a5c52810 100644 --- a/runtime/msg.c +++ b/runtime/msg.c @@ -3230,13 +3230,18 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, uchar *pSb; iFrom = pTpe->data.field.iFromPos; iTo = pTpe->data.field.iToPos; - /* need to zero-base to and from (they are 1-based!) */ - if(iFrom > 0) - --iFrom; - if(iTo > 0) - --iTo; if(bufLen == -1) bufLen = ustrlen(pRes); + if(pTpe->data.field.options.bFromPosEndRelative) { + iFrom = (bufLen < iFrom) ? 0 : bufLen - iFrom; + iTo = (bufLen < iTo)? 0 : bufLen - iTo; + } else { + /* need to zero-base to and from (they are 1-based!) */ + if(iFrom > 0) + --iFrom; + if(iTo > 0) + --iTo; + } if(iFrom == 0 && iTo >= bufLen) { /* in this case, the requested string is a superset of what we already have, * so there is no need to do any processing. This is a frequent case for size-limited diff --git a/runtime/nsd_gtls.c b/runtime/nsd_gtls.c index 0f66c766..6ef4feba 100644 --- a/runtime/nsd_gtls.c +++ b/runtime/nsd_gtls.c @@ -259,9 +259,9 @@ gtlsClientCertCallback(gnutls_session session, static rsRetVal gtlsGetCertInfo(nsd_gtls_t *pThis, cstr_t **ppStr) { - char dn[128]; - uchar lnBuf[256]; - size_t size; + uchar szBufA[1024]; + uchar *szBuf = szBufA; + size_t szBufLen = sizeof(szBufA), tmp; unsigned int algo, bits; time_t expiration_time, activation_time; const gnutls_datum *cert_list; @@ -271,8 +271,6 @@ gtlsGetCertInfo(nsd_gtls_t *pThis, cstr_t **ppStr) int gnuRet; DEFiRet; unsigned iAltName; - size_t szAltNameLen; - char szAltName[1024]; /* this is sufficient for the DNSNAME... */ assert(ppStr != NULL); ISOBJ_TYPE_assert(pThis, nsd_gtls); @@ -281,61 +279,62 @@ gtlsGetCertInfo(nsd_gtls_t *pThis, cstr_t **ppStr) return RS_RET_TLS_CERT_ERR; cert_list = gnutls_certificate_get_peers(pThis->sess, &cert_list_size); - - CHKiRet(rsCStrConstruct(&pStr)); - - snprintf((char*)lnBuf, sizeof(lnBuf), "peer provided %d certificate(s). ", cert_list_size); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); + CHKiRet(rsCStrConstructFromszStrf(&pStr, "peer provided %d certificate(s). ", cert_list_size)); if(cert_list_size > 0) { /* we only print information about the first certificate */ CHKgnutls(gnutls_x509_crt_init(&cert)); CHKgnutls(gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER)); - CHKiRet(rsCStrAppendStr(pStr, (uchar*)"Certificate 1 info: ")); - expiration_time = gnutls_x509_crt_get_expiration_time(cert); activation_time = gnutls_x509_crt_get_activation_time(cert); - ctime_r(&activation_time, dn); - dn[strlen(dn) - 1] = '\0'; /* strip linefeed */ - snprintf((char*)lnBuf, sizeof(lnBuf), "certificate valid from %s ", dn); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); - - ctime_r(&expiration_time, dn); - dn[strlen(dn) - 1] = '\0'; /* strip linefeed */ - snprintf((char*)lnBuf, sizeof(lnBuf), "to %s; ", dn); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); + ctime_r(&activation_time, szBuf); + szBuf[strlen(szBuf) - 1] = '\0'; /* strip linefeed */ + CHKiRet(rsCStrAppendStrf(pStr, (uchar*)"Certificate 1 info: " + "certificate valid from %s ", szBuf)); + ctime_r(&expiration_time, szBuf); + szBuf[strlen(szBuf) - 1] = '\0'; /* strip linefeed */ + CHKiRet(rsCStrAppendStrf(pStr, "to %s; ", szBuf)); /* Extract some of the public key algorithm's parameters */ algo = gnutls_x509_crt_get_pk_algorithm(cert, &bits); - - snprintf((char*)lnBuf, sizeof(lnBuf), "Certificate public key: %s; ", - gnutls_pk_algorithm_get_name(algo)); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); + CHKiRet(rsCStrAppendStrf(pStr, "Certificate public key: %s; ", + gnutls_pk_algorithm_get_name(algo))); /* names */ - size = sizeof(dn); - gnutls_x509_crt_get_dn(cert, dn, &size); - snprintf((char*)lnBuf, sizeof(lnBuf), "DN: %s; ", dn); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); - - size = sizeof(dn); - gnutls_x509_crt_get_issuer_dn(cert, dn, &size); - snprintf((char*)lnBuf, sizeof(lnBuf), "Issuer DN: %s; ", dn); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); + tmp = szBufLen; + if(gnutls_x509_crt_get_dn(cert, szBuf, &tmp) + == GNUTLS_E_SHORT_MEMORY_BUFFER) { + szBufLen = tmp; + szBuf = malloc(tmp); + gnutls_x509_crt_get_dn(cert, szBuf, &tmp); + } + CHKiRet(rsCStrAppendStrf(pStr, "DN: %s; ", szBuf)); + + tmp = szBufLen; + if(gnutls_x509_crt_get_issuer_dn(cert, szBuf, &tmp) + == GNUTLS_E_SHORT_MEMORY_BUFFER) { + szBufLen = tmp; + szBuf = realloc((szBuf == szBufA) ? NULL : szBuf, tmp); + gnutls_x509_crt_get_issuer_dn(cert, szBuf, &tmp); + } + CHKiRet(rsCStrAppendStrf(pStr, "Issuer DN: %s; ", szBuf)); /* dNSName alt name */ iAltName = 0; while(1) { /* loop broken below */ - szAltNameLen = sizeof(szAltName); + tmp = szBufLen; gnuRet = gnutls_x509_crt_get_subject_alt_name(cert, iAltName, - szAltName, &szAltNameLen, NULL); - if(gnuRet < 0) + szBuf, &tmp, NULL); + if(gnuRet == GNUTLS_E_SHORT_MEMORY_BUFFER) { + szBufLen = tmp; + szBuf = realloc((szBuf == szBufA) ? NULL : szBuf, tmp); + continue; + } else if(gnuRet < 0) break; else if(gnuRet == GNUTLS_SAN_DNSNAME) { /* we found it! */ - snprintf((char*)lnBuf, sizeof(lnBuf), "SAN:DNSname: %s; ", szAltName); - CHKiRet(rsCStrAppendStr(pStr, lnBuf)); + CHKiRet(rsCStrAppendStrf(pStr, "SAN:DNSname: %s; ", szBuf)); /* do NOT break, because there may be multiple dNSName's! */ } ++iAltName; @@ -352,6 +351,8 @@ finalize_it: if(pStr != NULL) rsCStrDestruct(&pStr); } + if(szBuf != szBufA) + free(szBuf); RETiRet; } diff --git a/runtime/queue.c b/runtime/queue.c index 4c8d3ac5..74090a4d 100644 --- a/runtime/queue.c +++ b/runtime/queue.c @@ -991,7 +991,7 @@ rsRetVal qqueueEnqObjDirectBatch(qqueue_t *pThis, batch_t *pBatch) * We use our knowledge about the batch_t structure below, but without that, we * pay a too-large performance toll... -- rgerhards, 2009-04-22 */ - iRet = pThis->pConsumer(pThis->pAction, pBatch, &pThis->bShutdownImmediate); + iRet = pThis->pConsumer(pThis->pAction, pBatch, NULL); RETiRet; } @@ -1150,6 +1150,7 @@ tryShutdownWorkersWithinActionTimeout(qqueue_t *pThis) DBGOPRINT((obj_t*) pThis, "trying to shutdown workers within Action Timeout"); DBGOPRINT((obj_t*) pThis, "setting EnqOnly mode\n"); pThis->bEnqOnly = 1; +dbgprintf("DDDD: setting shutdownImmediate mode, ptr %p!\n", &pThis->bShutdownImmediate); pThis->bShutdownImmediate = 1; /* now DA queue */ if(pThis->bIsDA) { @@ -1291,8 +1292,6 @@ finalize_it: RETiRet; } - - /* Constructor for the queue object * This constructs the data structure, but does not yet start the queue. That * is done by queueStart(). The reason is that we want to give the caller a chance @@ -1856,6 +1855,8 @@ ConsumerReg(qqueue_t *pThis, wti_t *pWti) /* at this spot, we may be cancelled */ pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, &iCancelStateSave); + +dbgprintf("DDDD: calling consumer with shutdownImmeditate ptr %p\n", &pThis->bShutdownImmediate); CHKiRet(pThis->pConsumer(pThis->pAction, &pWti->batch, &pThis->bShutdownImmediate)); /* we now need to check if we should deliberately delay processing a bit diff --git a/runtime/ratelimit.c b/runtime/ratelimit.c index 4b618fb5..d83da2dd 100644 --- a/runtime/ratelimit.c +++ b/runtime/ratelimit.c @@ -202,7 +202,9 @@ ratelimitMsg(ratelimit_t *ratelimit, msg_t *pMsg, msg_t **ppRepMsg) DEFiRet; *ppRepMsg = NULL; - if(ratelimit->interval) { + /* Only the messages having severity level at or below the + * treshold (the value is >=) are subject to ratelimiting. */ + if(ratelimit->interval && (pMsg->iSeverity >= ratelimit->severity)) { if(withinRatelimit(ratelimit, pMsg->ttGenTime) == 0) { msgDestruct(&pMsg); ABORT_FINALIZE(RS_RET_DISCARDMSG); @@ -284,6 +286,7 @@ ratelimitNew(ratelimit_t **ppThis, char *modname, char *dynname) namebuf[sizeof(namebuf)-1] = '\0'; /* to be on safe side */ pThis->name = strdup(namebuf); } + /* pThis->severity == 0 - all messages are ratelimited */ pThis->bReduceRepeatMsgs = loadConf->globals.bReduceRepeatMsgs; *ppThis = pThis; finalize_it: @@ -316,6 +319,15 @@ ratelimitSetThreadSafe(ratelimit_t *ratelimit) pthread_mutex_init(&ratelimit->mut, NULL); } +/* Severity level determines which messages are subject to + * ratelimiting. Default (no value set) is all messages. + */ +void +ratelimitSetSeverity(ratelimit_t *ratelimit, intTiny severity) +{ + ratelimit->severity = severity; +} + void ratelimitDestruct(ratelimit_t *ratelimit) { diff --git a/runtime/ratelimit.h b/runtime/ratelimit.h index 820817bc..a058b069 100644 --- a/runtime/ratelimit.h +++ b/runtime/ratelimit.h @@ -26,6 +26,7 @@ struct ratelimit_s { /* support for Linux kernel-type ratelimiting */ unsigned short interval; unsigned short burst; + intTiny severity; /**< ratelimit only equal or lower severity levels (eq or higher values) */ unsigned done; unsigned missed; time_t begin; @@ -41,6 +42,7 @@ struct ratelimit_s { rsRetVal ratelimitNew(ratelimit_t **ppThis, char *modname, char *dynname); void ratelimitSetThreadSafe(ratelimit_t *ratelimit); void ratelimitSetLinuxLike(ratelimit_t *ratelimit, unsigned short interval, unsigned short burst); +void ratelimitSetSeverity(ratelimit_t *ratelimit, intTiny severity); rsRetVal ratelimitMsg(ratelimit_t *ratelimit, msg_t *pMsg, msg_t **ppRep); rsRetVal ratelimitAddMsg(ratelimit_t *ratelimit, multi_submit_t *pMultiSub, msg_t *pMsg); void ratelimitDestruct(ratelimit_t *pThis); diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index e7a5dffb..9fdf2b0f 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -387,7 +387,9 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_MOD_NO_INPUT_STMT = -2224, /**< (input) module does not support input() statement */ RS_RET_NO_CEE_MSG = -2225, /**< the message being processed is NOT CEE-enhanced */ - /**** up to 2300 is reserved for v6 use ****/ + /**** up to 2290 is reserved for v6 use ****/ + RS_RET_RELP_ERR = -2291, /**<< error in RELP processing */ + /**** up to 3000 is reserved for c7 use ****/ RS_RET_JNAME_NO_ROOT = -2301, /**< root element is missing in JSON path */ RS_RET_JNAME_INVALID = -2302, /**< JSON path is invalid */ RS_RET_JSON_PARSE_ERR = -2303, /**< we had a problem parsing JSON (or extra data) */ @@ -402,6 +404,14 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_INVLD_MODE = -2311,/**< invalid mode specified in configuration */ RS_RET_INVLD_ANON_BITS = -2312,/**< mmanon: invalid number of bits to anonymize specified */ RS_RET_REPLCHAR_IGNORED = -2313,/**< mmanon: replacementChar parameter is ignored */ + RS_RET_SIGPROV_ERR = -2320,/**< error in signature provider */ + RS_RET_CRYPROV_ERR = -2321,/**< error in cryptography encryption provider */ + RS_RET_EI_OPN_ERR = -2322,/**< error opening an .encinfo file */ + RS_RET_EI_NO_EXISTS = -2323,/**< .encinfo file does not exist (status, not necessarily error!)*/ + RS_RET_EI_WR_ERR = -2324,/**< error writing an .encinfo file */ + RS_RET_EI_INVLD_FILE = -2325,/**< header indicates the file is no .encinfo file */ + RS_RET_CRY_INVLD_ALGO = -2326,/**< user specified invalid (unkonwn) crypto algorithm */ + RS_RET_CRY_INVLD_MODE = -2327,/**< user specified invalid (unkonwn) crypto mode */ /* RainerScript error messages (range 1000.. 1999) */ RS_RET_SYSVAR_NOT_FOUND = 1001, /**< system variable could not be found (maybe misspelled) */ diff --git a/runtime/ruleset.c b/runtime/ruleset.c index 66b38fc9..e3348938 100644 --- a/runtime/ruleset.c +++ b/runtime/ruleset.c @@ -11,7 +11,7 @@ * * Module begun 2009-06-10 by Rainer Gerhards * - * Copyright 2009-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -297,6 +297,7 @@ execIf(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) sbool *newAct; int i; sbool bRet; + sbool allInactive = 1; DEFiRet; newAct = newActive(pBatch); for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { @@ -306,12 +307,19 @@ execIf(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) continue; /* will be ignored in any case */ if(active == NULL || active[i]) { bRet = cnfexprEvalBool(stmt->d.s_if.expr, pBatch->pElem[i].pMsg); + allInactive = 0; } else bRet = 0; newAct[i] = bRet; DBGPRINTF("batch: item %d: expr eval: %d\n", i, bRet); } + if(allInactive) { + DBGPRINTF("execIf: all batch elements are inactive, holding execution\n"); + freeActive(newAct); + FINALIZE; + } + if(stmt->d.s_if.t_then != NULL) { scriptExec(stmt->d.s_if.t_then, pBatch, newAct); } @@ -319,7 +327,8 @@ execIf(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { if(*(pBatch->pbShutdownImmediate)) FINALIZE; - if(pBatch->eltState[i] != BATCH_STATE_DISC) + if(pBatch->eltState[i] != BATCH_STATE_DISC + && (active == NULL || active[i])) newAct[i] = !newAct[i]; } scriptExec(stmt->d.s_if.t_else, pBatch, newAct); @@ -364,7 +373,8 @@ execPRIFILT(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { if(*(pBatch->pbShutdownImmediate)) return; - if(pBatch->eltState[i] != BATCH_STATE_DISC) + if(pBatch->eltState[i] != BATCH_STATE_DISC + && (active == NULL || active[i])) newAct[i] = !newAct[i]; } scriptExec(stmt->d.s_prifilt.t_else, pBatch, newAct); @@ -504,7 +514,11 @@ scriptExec(struct cnfstmt *root, batch_t *pBatch, sbool *active) struct cnfstmt *stmt; for(stmt = root ; stmt != NULL ; stmt = stmt->next) { -dbgprintf("RRRR: scriptExec: batch of %d elements, active %p, stmt %p, nodetype %u\n", batchNumMsgs(pBatch), active, stmt, stmt->nodetype); + if(Debug) { + dbgprintf("scriptExec: batch of %d elements, active %p, active[0]:%d\n", + batchNumMsgs(pBatch), active, (active == NULL ? 1 : active[0])); + cnfstmtPrintOnly(stmt, 2, 0); + } switch(stmt->nodetype) { case S_NOP: break; @@ -521,7 +535,6 @@ dbgprintf("RRRR: scriptExec: batch of %d elements, active %p, stmt %p, nodetype execUnset(stmt, pBatch, active); break; case S_CALL: - DBGPRINTF("calling ruleset\n"); // TODO: add Name scriptExec(stmt->d.s_call.stmt, pBatch, active); break; case S_IF: diff --git a/runtime/sd-daemon.c b/runtime/sd-daemon.c index 9c23b917..79d8ca37 100644 --- a/runtime/sd-daemon.c +++ b/runtime/sd-daemon.c @@ -25,14 +25,18 @@ ***/ #ifndef _GNU_SOURCE -#define _GNU_SOURCE +# define _GNU_SOURCE #endif #include <sys/types.h> #include <sys/stat.h> #include <sys/socket.h> #include <sys/un.h> -#include <sys/fcntl.h> +#ifdef __BIONIC__ +# include <linux/fcntl.h> +#else +# include <sys/fcntl.h> +#endif #include <netinet/in.h> #include <stdlib.h> #include <errno.h> @@ -40,10 +44,28 @@ #include <string.h> #include <stdarg.h> #include <stdio.h> +#include <stddef.h> +#include <limits.h> + +#if defined(__linux__) +# include <mqueue.h> +#endif #include "sd-daemon.h" -int sd_listen_fds(int unset_environment) { +#if (__GNUC__ >= 4) +# ifdef SD_EXPORT_SYMBOLS +/* Export symbols */ +# define _sd_export_ __attribute__ ((visibility("default"))) +# else +/* Don't export the symbols */ +# define _sd_export_ __attribute__ ((visibility("hidden"))) +# endif +#else +# define _sd_export_ +#endif + +_sd_export_ int sd_listen_fds(int unset_environment) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) return 0; @@ -53,7 +75,8 @@ int sd_listen_fds(int unset_environment) { char *p = NULL; unsigned long l; - if (!(e = getenv("LISTEN_PID"))) { + e = getenv("LISTEN_PID"); + if (!e) { r = 0; goto finish; } @@ -66,7 +89,7 @@ int sd_listen_fds(int unset_environment) { goto finish; } - if (!p || *p || l <= 0) { + if (!p || p == e || *p || l <= 0) { r = -EINVAL; goto finish; } @@ -77,7 +100,8 @@ int sd_listen_fds(int unset_environment) { goto finish; } - if (!(e = getenv("LISTEN_FDS"))) { + e = getenv("LISTEN_FDS"); + if (!e) { r = 0; goto finish; } @@ -90,7 +114,7 @@ int sd_listen_fds(int unset_environment) { goto finish; } - if (!p || *p) { + if (!p || p == e || *p) { r = -EINVAL; goto finish; } @@ -98,7 +122,8 @@ int sd_listen_fds(int unset_environment) { for (fd = SD_LISTEN_FDS_START; fd < SD_LISTEN_FDS_START + (int) l; fd ++) { int flags; - if ((flags = fcntl(fd, F_GETFD)) < 0) { + flags = fcntl(fd, F_GETFD); + if (flags < 0) { r = -errno; goto finish; } @@ -124,13 +149,12 @@ finish: #endif } -int sd_is_fifo(int fd, const char *path) { +_sd_export_ int sd_is_fifo(int fd, const char *path) { struct stat st_fd; if (fd < 0) return -EINVAL; - memset(&st_fd, 0, sizeof(st_fd)); if (fstat(fd, &st_fd) < 0) return -errno; @@ -140,7 +164,6 @@ int sd_is_fifo(int fd, const char *path) { if (path) { struct stat st_path; - memset(&st_path, 0, sizeof(st_path)); if (stat(path, &st_path) < 0) { if (errno == ENOENT || errno == ENOTDIR) @@ -157,6 +180,42 @@ int sd_is_fifo(int fd, const char *path) { return 1; } +_sd_export_ int sd_is_special(int fd, const char *path) { + struct stat st_fd; + + if (fd < 0) + return -EINVAL; + + if (fstat(fd, &st_fd) < 0) + return -errno; + + if (!S_ISREG(st_fd.st_mode) && !S_ISCHR(st_fd.st_mode)) + return 0; + + if (path) { + struct stat st_path; + + if (stat(path, &st_path) < 0) { + + if (errno == ENOENT || errno == ENOTDIR) + return 0; + + return -errno; + } + + if (S_ISREG(st_fd.st_mode) && S_ISREG(st_path.st_mode)) + return + st_path.st_dev == st_fd.st_dev && + st_path.st_ino == st_fd.st_ino; + else if (S_ISCHR(st_fd.st_mode) && S_ISCHR(st_path.st_mode)) + return st_path.st_rdev == st_fd.st_rdev; + else + return 0; + } + + return 1; +} + static int sd_is_socket_internal(int fd, int type, int listening) { struct stat st_fd; @@ -208,13 +267,14 @@ union sockaddr_union { struct sockaddr_storage storage; }; -int sd_is_socket(int fd, int family, int type, int listening) { +_sd_export_ int sd_is_socket(int fd, int family, int type, int listening) { int r; if (family < 0) return -EINVAL; - if ((r = sd_is_socket_internal(fd, type, listening)) <= 0) + r = sd_is_socket_internal(fd, type, listening); + if (r <= 0) return r; if (family > 0) { @@ -236,7 +296,7 @@ int sd_is_socket(int fd, int family, int type, int listening) { return 1; } -int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port) { +_sd_export_ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port) { union sockaddr_union sockaddr; socklen_t l; int r; @@ -244,7 +304,8 @@ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port if (family != 0 && family != AF_INET && family != AF_INET6) return -EINVAL; - if ((r = sd_is_socket_internal(fd, type, listening)) <= 0) + r = sd_is_socket_internal(fd, type, listening); + if (r <= 0) return r; memset(&sockaddr, 0, sizeof(sockaddr)); @@ -281,12 +342,13 @@ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port return 1; } -int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t length) { +_sd_export_ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t length) { union sockaddr_union sockaddr; socklen_t l; int r; - if ((r = sd_is_socket_internal(fd, type, listening)) <= 0) + r = sd_is_socket_internal(fd, type, listening); + if (r <= 0) return r; memset(&sockaddr, 0, sizeof(sockaddr)); @@ -302,29 +364,66 @@ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t return 0; if (path) { - if (length <= 0) + if (length == 0) length = strlen(path); - if (length <= 0) + if (length == 0) /* Unnamed socket */ - return l == sizeof(sa_family_t); + return l == offsetof(struct sockaddr_un, sun_path); if (path[0]) /* Normal path socket */ return - (l >= sizeof(sa_family_t) + length + 1) && + (l >= offsetof(struct sockaddr_un, sun_path) + length + 1) && memcmp(path, sockaddr.un.sun_path, length+1) == 0; else /* Abstract namespace socket */ return - (l == sizeof(sa_family_t) + length) && + (l == offsetof(struct sockaddr_un, sun_path) + length) && memcmp(path, sockaddr.un.sun_path, length) == 0; } return 1; } -int sd_notify(int unset_environment, const char *state) { +_sd_export_ int sd_is_mq(int fd, const char *path) { +#if !defined(__linux__) + return 0; +#else + struct mq_attr attr; + + if (fd < 0) + return -EINVAL; + + if (mq_getattr(fd, &attr) < 0) + return -errno; + + if (path) { + char fpath[PATH_MAX]; + struct stat a, b; + + if (path[0] != '/') + return -EINVAL; + + if (fstat(fd, &a) < 0) + return -errno; + + strncpy(stpcpy(fpath, "/dev/mqueue"), path, sizeof(fpath) - 12); + fpath[sizeof(fpath)-1] = 0; + + if (stat(fpath, &b) < 0) + return -errno; + + if (a.st_dev != b.st_dev || + a.st_ino != b.st_ino) + return 0; + } + + return 1; +#endif +} + +_sd_export_ int sd_notify(int unset_environment, const char *state) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) || !defined(SOCK_CLOEXEC) return 0; #else @@ -339,7 +438,8 @@ int sd_notify(int unset_environment, const char *state) { goto finish; } - if (!(e = getenv("NOTIFY_SOCKET"))) + e = getenv("NOTIFY_SOCKET"); + if (!e) return 0; /* Must be an abstract socket, or an absolute path */ @@ -348,7 +448,8 @@ int sd_notify(int unset_environment, const char *state) { goto finish; } - if ((fd = socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0)) < 0) { + fd = socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0); + if (fd < 0) { r = -errno; goto finish; } @@ -366,7 +467,7 @@ int sd_notify(int unset_environment, const char *state) { memset(&msghdr, 0, sizeof(msghdr)); msghdr.msg_name = &sockaddr; - msghdr.msg_namelen = sizeof(sa_family_t) + strlen(e); + msghdr.msg_namelen = offsetof(struct sockaddr_un, sun_path) + strlen(e); if (msghdr.msg_namelen > sizeof(struct sockaddr_un)) msghdr.msg_namelen = sizeof(struct sockaddr_un); @@ -392,7 +493,7 @@ finish: #endif } -int sd_notifyf(int unset_environment, const char *format, ...) { +_sd_export_ int sd_notifyf(int unset_environment, const char *format, ...) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) return 0; #else @@ -414,22 +515,19 @@ int sd_notifyf(int unset_environment, const char *format, ...) { #endif } -int sd_booted(void) { +_sd_export_ int sd_booted(void) { #if defined(DISABLE_SYSTEMD) || !defined(__linux__) return 0; #else + struct stat st; - struct stat a, b; - - /* We simply test whether the systemd cgroup hierarchy is - * mounted */ - - if (lstat("/sys/fs/cgroup", &a) < 0) - return 0; + /* We test whether the runtime unit file directory has been + * created. This takes place in mount-setup.c, so is + * guaranteed to happen very early during boot. */ - if (lstat("/sys/fs/cgroup/systemd", &b) < 0) + if (lstat("/run/systemd/system/", &st) < 0) return 0; - return a.st_dev != b.st_dev; + return !!S_ISDIR(st.st_mode); #endif } diff --git a/runtime/sd-daemon.h b/runtime/sd-daemon.h index 45aac8bd..fb7456d5 100644 --- a/runtime/sd-daemon.h +++ b/runtime/sd-daemon.h @@ -58,25 +58,21 @@ extern "C" { You may find an up-to-date version of these source files online: - http://cgit.freedesktop.org/systemd/plain/src/sd-daemon.h - http://cgit.freedesktop.org/systemd/plain/src/sd-daemon.c + http://cgit.freedesktop.org/systemd/systemd/plain/src/systemd/sd-daemon.h + http://cgit.freedesktop.org/systemd/systemd/plain/src/libsystemd-daemon/sd-daemon.c This should compile on non-Linux systems, too, but with the exception of the sd_is_xxx() calls all functions will become NOPs. - See sd-daemon(7) for more information. + See sd-daemon(3) for more information. */ -#if (__GNUC__ >= 4) +#ifndef _sd_printf_attr_ +#if __GNUC__ >= 4 #define _sd_printf_attr_(a,b) __attribute__ ((format (printf, a, b))) -# if defined(SD_EXPORT_SYMBOLS) -# define _sd_hidden_ -# else -# define _sd_hidden_ __attribute__ ((visibility("hidden"))) -# endif #else #define _sd_printf_attr_(a,b) -#define _sd_hidden_ +#endif #endif /* @@ -113,7 +109,7 @@ extern "C" { See sd_listen_fds(3) for more information. */ -int sd_listen_fds(int unset_environment) _sd_hidden_; +int sd_listen_fds(int unset_environment); /* Helper call for identifying a passed file descriptor. Returns 1 if @@ -125,7 +121,19 @@ int sd_listen_fds(int unset_environment) _sd_hidden_; See sd_is_fifo(3) for more information. */ -int sd_is_fifo(int fd, const char *path) _sd_hidden_; +int sd_is_fifo(int fd, const char *path); + +/* + Helper call for identifying a passed file descriptor. Returns 1 if + the file descriptor is a special character device on the file + system stored under the specified path, 0 otherwise. + If path is NULL a path name check will not be done and the call + only verifies if the file descriptor refers to a special character. + Returns a negative errno style error code on failure. + + See sd_is_special(3) for more information. +*/ +int sd_is_special(int fd, const char *path); /* Helper call for identifying a passed file descriptor. Returns 1 if @@ -141,7 +149,7 @@ int sd_is_fifo(int fd, const char *path) _sd_hidden_; See sd_is_socket(3) for more information. */ -int sd_is_socket(int fd, int family, int type, int listening) _sd_hidden_; +int sd_is_socket(int fd, int family, int type, int listening); /* Helper call for identifying a passed file descriptor. Returns 1 if @@ -155,7 +163,7 @@ int sd_is_socket(int fd, int family, int type, int listening) _sd_hidden_; See sd_is_socket_inet(3) for more information. */ -int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port) _sd_hidden_; +int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port); /* Helper call for identifying a passed file descriptor. Returns 1 if @@ -171,7 +179,15 @@ int sd_is_socket_inet(int fd, int family, int type, int listening, uint16_t port See sd_is_socket_unix(3) for more information. */ -int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t length) _sd_hidden_; +int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t length); + +/* + Helper call for identifying a passed file descriptor. Returns 1 if + the file descriptor is a POSIX Message Queue of the specified name, + 0 otherwise. If path is NULL a message queue name check is not + done. Returns a negative errno style error code on failure. +*/ +int sd_is_mq(int fd, const char *path); /* Informs systemd about changed daemon state. This takes a number of @@ -181,7 +197,7 @@ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t READY=1 Tells systemd that daemon startup is finished (only relevant for services of Type=notify). The passed argument is a boolean "1" or "0". Since there is - little value in signalling non-readiness the only + little value in signaling non-readiness the only value daemons should send is "READY=1". STATUS=... Passes a single-line status string back to systemd @@ -201,8 +217,13 @@ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t MAINPID=... The main pid of a daemon, in case systemd did not fork off the process itself. Example: "MAINPID=4711" + WATCHDOG=1 Tells systemd to update the watchdog timestamp. + Services using this feature should do this in + regular intervals. A watchdog framework can use the + timestamps to detect failed services. + Daemons can choose to send additional variables. However, it is - recommened to prefix variable names not listed above with X_. + recommended to prefix variable names not listed above with X_. Returns a negative errno-style error code on failure. Returns > 0 if systemd could be notified, 0 if it couldn't possibly because @@ -217,7 +238,7 @@ int sd_is_socket_unix(int fd, int type, int listening, const char *path, size_t See sd_notify(3) for more information. */ -int sd_notify(int unset_environment, const char *state) _sd_hidden_; +int sd_notify(int unset_environment, const char *state); /* Similar to sd_notify() but takes a format string. @@ -239,7 +260,7 @@ int sd_notify(int unset_environment, const char *state) _sd_hidden_; See sd_notifyf(3) for more information. */ -int sd_notifyf(int unset_environment, const char *format, ...) _sd_printf_attr_(2,3) _sd_hidden_; +int sd_notifyf(int unset_environment, const char *format, ...) _sd_printf_attr_(2,3); /* Returns > 0 if the system was booted with systemd. Returns < 0 on @@ -248,11 +269,11 @@ int sd_notifyf(int unset_environment, const char *format, ...) _sd_printf_attr_( fine. You should NOT protect them with a call to this function. Also note that this function checks whether the system, not the user session is controlled by systemd. However the functions above work - for both session and system services. + for both user and system services. See sd_booted(3) for more information. */ -int sd_booted(void) _sd_hidden_; +int sd_booted(void); #ifdef __cplusplus } diff --git a/runtime/sigprov.h b/runtime/sigprov.h new file mode 100644 index 00000000..82587b7d --- /dev/null +++ b/runtime/sigprov.h @@ -0,0 +1,37 @@ +/* The interface definition for (file) signature providers. + * + * This is just an abstract driver interface, which needs to be + * implemented by concrete classes. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_SIGPROV_H +#define INCLUDED_SIGPROV_H + +/* interface */ +BEGINinterface(sigprov) /* name must also be changed in ENDinterface macro! */ + rsRetVal (*Construct)(void *ppThis); + rsRetVal (*SetCnfParam)(void *ppThis, struct nvlst *lst); + rsRetVal (*Destruct)(void *ppThis); + rsRetVal (*OnFileOpen)(void *pThis, uchar *fn, void *pFileInstData); + rsRetVal (*OnRecordWrite)(void *pFileInstData, uchar *rec, rs_size_t lenRec); + rsRetVal (*OnFileClose)(void *pFileInstData); +ENDinterface(sigprov) +#define sigprovCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#endif /* #ifndef INCLUDED_SIGPROV_H */ diff --git a/runtime/srUtils.h b/runtime/srUtils.h index 3169fd94..8626a4bb 100644 --- a/runtime/srUtils.h +++ b/runtime/srUtils.h @@ -91,6 +91,7 @@ char *rs_strerror_r(int errnum, char *buf, size_t buflen); int decodeSyslogName(uchar *name, syslogName_t *codetab); int getSubString(uchar **ppSrc, char *pDst, size_t DstSize, char cSep); rsRetVal getFileSize(uchar *pszName, off_t *pSize); +int containsGlobWildcard(char *str); /* mutex operations */ /* some useful constants */ diff --git a/runtime/srutils.c b/runtime/srutils.c index 7b485b23..6a509b4a 100644 --- a/runtime/srutils.c +++ b/runtime/srutils.c @@ -630,6 +630,28 @@ finalize_it: RETiRet; } +/* Returns 1 if the given string contains a non-escaped glob(3) + * wildcard character and 0 otherwise (or if the string is empty). + */ +int +containsGlobWildcard(char *str) +{ + char *p; + if(!str) { + return 0; + } + /* From Linux Programmer's Guide: + * "A string is a wildcard pattern if it contains one of the characters '?', '*' or '['" + * "One can remove the special meaning of '?', '*' and '[' by preceding them by a backslash" + */ + for(p = str; *p != '\0'; p++) { + if((*p == '?' || *p == '*' || *p == '[') && + (p == str || *(p-1) != '\\')) { + return 1; + } + } + return 0; +} /* vim:set ai: */ diff --git a/runtime/stream.c b/runtime/stream.c index 3e890c71..b0df8418 100644 --- a/runtime/stream.c +++ b/runtime/stream.c @@ -16,7 +16,7 @@ * it turns out to be problematic. Then, we need to quasi-refcount the number of accesses * to the object. * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -45,6 +45,7 @@ #include <pthread.h> #include <fcntl.h> #include <unistd.h> +#include <sys/types.h> #include <sys/stat.h> /* required for HP UX */ #include <errno.h> #include <pthread.h> @@ -56,6 +57,7 @@ #include "stream.h" #include "unicode-helper.h" #include "module-template.h" +#include "cryprov.h" #if HAVE_SYS_PRCTL_H # include <sys/prctl.h> #endif @@ -81,6 +83,7 @@ static void *asyncWriterThread(void *pPtr); static rsRetVal doZipWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf, int bFlush); static rsRetVal doZipFinish(strm_t *pThis); static rsRetVal strmPhysWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf); +static rsRetVal strmSeekCurrOffs(strm_t *pThis); /* methods */ @@ -197,6 +200,7 @@ static rsRetVal doPhysOpen(strm_t *pThis) { int iFlags = 0; + struct stat statOpen; DEFiRet; ISOBJ_TYPE_assert(pThis, strm); @@ -234,15 +238,76 @@ doPhysOpen(strm_t *pThis) ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); else ABORT_FINALIZE(RS_RET_IO_ERROR); + } + + if(pThis->tOperationsMode == STREAMMODE_READ) { + if(fstat(pThis->fd, &statOpen) == -1) { + DBGPRINTF("Error: cannot obtain inode# for file %s\n", pThis->pszCurrFName); + ABORT_FINALIZE(RS_RET_IO_ERROR); + } + pThis->inode = statOpen.st_ino; + } + + if(!ustrcmp(pThis->pszCurrFName, UCHAR_CONSTANT(_PATH_CONSOLE)) || isatty(pThis->fd)) { + DBGPRINTF("file %d is a tty-type file\n", pThis->fd); + pThis->bIsTTY = 1; } else { - if(!ustrcmp(pThis->pszCurrFName, UCHAR_CONSTANT(_PATH_CONSOLE)) || isatty(pThis->fd)) { - DBGPRINTF("file %d is a tty-type file\n", pThis->fd); - pThis->bIsTTY = 1; + pThis->bIsTTY = 0; + } + +dbgprintf("DDDD: cryprov %p\n", pThis->cryprov); + if(pThis->cryprov != NULL) { + CHKiRet(pThis->cryprov->OnFileOpen(pThis->cryprovData, + pThis->pszCurrFName, &pThis->cryprovFileData)); + } +finalize_it: + RETiRet; +} + + +static rsRetVal +strmSetCurrFName(strm_t *pThis) +{ + DEFiRet; + + if(pThis->sType == STREAMTYPE_FILE_CIRCULAR) { + CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, + pThis->pszFName, pThis->lenFName, pThis->iCurrFNum, pThis->iFileNumDigits)); + } else { + if(pThis->pszDir == NULL) { + if((pThis->pszCurrFName = ustrdup(pThis->pszFName)) == NULL) + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); } else { - pThis->bIsTTY = 0; + CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, + pThis->pszFName, pThis->lenFName, -1, 0)); } } +finalize_it: + RETiRet; +} + +/* This function checks if the actual file has changed and, if so, resets the + * offset. This is support for monitoring files. It should be called after + * deserializing the strm object and before doing any other operation on it + * (most importantly not an open or seek!). + */ +static rsRetVal +CheckFileChange(strm_t *pThis) +{ + struct stat statName; + DEFiRet; + CHKiRet(strmSetCurrFName(pThis)); + if(stat((char*) pThis->pszCurrFName, &statName) == -1) + ABORT_FINALIZE(RS_RET_IO_ERROR); + DBGPRINTF("stream/after deserialize checking for file change on '%s', " + "inode %u/%u, size/currOffs %llu/%llu\n", + pThis->pszCurrFName, (unsigned) pThis->inode, + (unsigned) statName.st_ino, statName.st_size, pThis->iCurrOffs); + if(pThis->inode != statName.st_ino || statName.st_size < pThis->iCurrOffs) { + DBGPRINTF("stream: file %s has changed\n", pThis->pszCurrFName); + pThis->iCurrOffs = 0; + } finalize_it: RETiRet; } @@ -265,19 +330,8 @@ static rsRetVal strmOpenFile(strm_t *pThis) if(pThis->pszFName == NULL) ABORT_FINALIZE(RS_RET_FILE_PREFIX_MISSING); - if(pThis->sType == STREAMTYPE_FILE_CIRCULAR) { - CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, - pThis->pszFName, pThis->lenFName, pThis->iCurrFNum, pThis->iFileNumDigits)); - } else { - if(pThis->pszDir == NULL) { - if((pThis->pszCurrFName = ustrdup(pThis->pszFName)) == NULL) - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - } else { - CHKiRet(genFileName(&pThis->pszCurrFName, pThis->pszDir, pThis->lenDir, - pThis->pszFName, pThis->lenFName, -1, 0)); - } - } - + CHKiRet(strmSetCurrFName(pThis)); + CHKiRet(doPhysOpen(pThis)); pThis->iCurrOffs = 0; @@ -335,6 +389,7 @@ strmWaitAsyncWriterDone(strm_t *pThis) */ static rsRetVal strmCloseFile(strm_t *pThis) { + off64_t currOffs; DEFiRet; ASSERT(pThis != NULL); @@ -355,8 +410,14 @@ static rsRetVal strmCloseFile(strm_t *pThis) * against this. -- rgerhards, 2010-03-19 */ if(pThis->fd != -1) { + currOffs = lseek64(pThis->fd, 0, SEEK_CUR); close(pThis->fd); pThis->fd = -1; + pThis->inode = 0; + if(pThis->cryprov != NULL) { + pThis->cryprov->OnFileClose(pThis->cryprovFileData, currOffs); + pThis->cryprovFileData = NULL; + } } if(pThis->fdDir != -1) { @@ -432,18 +493,15 @@ static rsRetVal strmHandleEOFMonitor(strm_t *pThis) { DEFiRet; - struct stat statOpen; struct stat statName; ISOBJ_TYPE_assert(pThis, strm); - if(fstat(pThis->fd, &statOpen) == -1) - ABORT_FINALIZE(RS_RET_IO_ERROR); if(stat((char*) pThis->pszCurrFName, &statName) == -1) ABORT_FINALIZE(RS_RET_IO_ERROR); - DBGPRINTF("stream checking for file change on '%s', inode %u/%u", - pThis->pszCurrFName, (unsigned) statOpen.st_ino, + DBGPRINTF("stream checking for file change on '%s', inode %u/%u\n", + pThis->pszCurrFName, (unsigned) pThis->inode, (unsigned) statName.st_ino); - if(statOpen.st_ino == statName.st_ino) { + if(pThis->inode == statName.st_ino) { ABORT_FINALIZE(RS_RET_EOF); } else { /* we had a file change! */ @@ -1155,10 +1213,18 @@ strmPhysWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf) DEFiRet; ISOBJ_TYPE_assert(pThis, strm); - DBGPRINTF("strmPhysWrite, stream %p, len %d\n", pThis, (int) lenBuf); + DBGPRINTF("strmPhysWrite, stream %p, len %u\n", pThis, (unsigned)lenBuf); if(pThis->fd == -1) CHKiRet(strmOpenFile(pThis)); + /* here we place our crypto interface */ +dbgprintf("DDDD: doing crypto, len %d\n", lenBuf); + if(pThis->cryprov != NULL) { + pThis->cryprov->Encrypt(pThis->cryprovFileData, pBuf, &lenBuf); + } +dbgprintf("DDDD: done crypto, len %d\n", lenBuf); + /* end crypto */ + iWritten = lenBuf; CHKiRet(doWriteCall(pThis, pBuf, &iWritten)); @@ -1343,7 +1409,11 @@ static rsRetVal strmSeek(strm_t *pThis, off64_t offs) } long long i; DBGOPRINT((obj_t*) pThis, "file %d seek, pos %llu\n", pThis->fd, (long long unsigned) offs); - i = lseek64(pThis->fd, offs, SEEK_SET); // TODO: check error! + i = lseek64(pThis->fd, offs, SEEK_SET); + if(i != offs) { + DBGPRINTF("strmSeek: error %lld seeking to offset %lld\n", i, offs); + ABORT_FINALIZE(RS_RET_IO_ERROR); + } pThis->iCurrOffs = offs; /* we are now at *this* offset */ pThis->iBufPtr = 0; /* buffer invalidated */ @@ -1551,6 +1621,8 @@ DEFpropSetMeth(strm, sIOBufSize, size_t) DEFpropSetMeth(strm, iSizeLimit, off_t) DEFpropSetMeth(strm, iFlushInterval, int) DEFpropSetMeth(strm, pszSizeLimitCmd, uchar*) +DEFpropSetMeth(strm, cryprov, cryprov_if_t*) +DEFpropSetMeth(strm, cryprovData, void*) static rsRetVal strmSetiMaxFiles(strm_t *pThis, int iNewVal) { @@ -1697,6 +1769,9 @@ static rsRetVal strmSerialize(strm_t *pThis, strm_t *pStrm) l = pThis->iCurrOffs; objSerializeSCALAR_VAR(pStrm, iCurrOffs, INT64, l); + l = pThis->inode; + objSerializeSCALAR_VAR(pStrm, inode, INT64, l); + objSerializePTR(pStrm, prevLineSegment, PSZ); CHKiRet(obj.EndSerialize(pStrm)); @@ -1796,6 +1871,8 @@ static rsRetVal strmSetProperty(strm_t *pThis, var_t *pProp) CHKiRet(strmSettOpenMode(pThis, pProp->val.num)); } else if(isProp("iCurrOffs")) { pThis->iCurrOffs = pProp->val.num; + } else if(isProp("inode")) { + pThis->inode = (ino_t) pProp->val.num; } else if(isProp("iMaxFileSize")) { CHKiRet(strmSetiMaxFileSize(pThis, pProp->val.num)); } else if(isProp("iMaxFiles")) { @@ -1865,6 +1942,7 @@ CODESTARTobjQueryInterface(strm) pIf->GetCurrOffset = strmGetCurrOffset; pIf->Dup = strmDup; pIf->SetWCntr = strmSetWCntr; + pIf->CheckFileChange = CheckFileChange; /* set methods */ pIf->SetbDeleteOnClose = strmSetbDeleteOnClose; pIf->SetiMaxFileSize = strmSetiMaxFileSize; @@ -1880,6 +1958,8 @@ CODESTARTobjQueryInterface(strm) pIf->SetiSizeLimit = strmSetiSizeLimit; pIf->SetiFlushInterval = strmSetiFlushInterval; pIf->SetpszSizeLimitCmd = strmSetpszSizeLimitCmd; + pIf->Setcryprov = strmSetcryprov; + pIf->SetcryprovData = strmSetcryprovData; finalize_it: ENDobjQueryInterface(strm) diff --git a/runtime/stream.h b/runtime/stream.h index b7e74074..61d5ede2 100644 --- a/runtime/stream.h +++ b/runtime/stream.h @@ -41,7 +41,7 @@ * deflateInit2(zstrmptr, 6, Z_DEFLATED, 31, 9, Z_DEFAULT_STRATEGY); * -------------------------------------------------------------------------- * - * Copyright 2008, 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -70,6 +70,7 @@ #include "glbl.h" #include "stream.h" #include "zlibw.h" +#include "cryprov.h" /* stream types */ typedef enum { @@ -112,6 +113,7 @@ typedef struct strm_s { int lenDir; int fd; /* the file descriptor, -1 if closed */ int fdDir; /* the directory's descriptor, in case bSync is requested (-1 if closed) */ + ino_t inode; /* current inode for files being monitored (undefined else) */ uchar *pszCurrFName; /* name of current file (if open) */ uchar *pIOBuf; /* the iobuffer currently in use to gather data */ size_t iBufPtrMax; /* current max Ptr in Buffer (if partial read!) */ @@ -133,6 +135,9 @@ typedef struct strm_s { pthread_cond_t isEmpty; unsigned short iEnq; /* this MUST be unsigned as we use module arithmetic (else invalid indexing happens!) */ unsigned short iDeq; /* this MUST be unsigned as we use module arithmetic (else invalid indexing happens!) */ + cryprov_if_t *cryprov; /* ptr to crypto provider; NULL = do not encrypt */ + void *cryprovData; /* opaque data ptr for provider use */ + void *cryprovFileData;/* opaque data ptr for file instance */ short iCnt; /* current nbr of elements in buffer */ z_stream zstrm; /* zip stream to use */ struct { @@ -187,8 +192,13 @@ BEGINinterface(strm) /* name must also be changed in ENDinterface macro! */ rsRetVal (*ReadLine)(strm_t *pThis, cstr_t **ppCStr, int mode); /* v7 added 2012-09-14 */ INTERFACEpropSetMeth(strm, bVeryReliableZip, int); + /* v8 added 2013-03-21 */ + rsRetVal (*CheckFileChange)(strm_t *pThis); + /* v9 added 2013-04-04 */ + INTERFACEpropSetMeth(strm, cryprov, cryprov_if_t*); + INTERFACEpropSetMeth(strm, cryprovData, void*); ENDinterface(strm) -#define strmCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ +#define strmCURR_IF_VERSION 9 /* increment whenever you change the interface structure! */ static inline int strmGetCurrFileNum(strm_t *pStrm) { diff --git a/runtime/stringbuf.c b/runtime/stringbuf.c index d9f80231..75d2eac4 100644 --- a/runtime/stringbuf.c +++ b/runtime/stringbuf.c @@ -32,6 +32,7 @@ #include <assert.h> #include <string.h> #include <ctype.h> +#include <stdarg.h> #include <sys/types.h> #include <libestr.h> #include "rsyslog.h" @@ -104,6 +105,56 @@ finalize_it: } +/* a helper function for rsCStr*Strf() + */ +static rsRetVal rsCStrConstructFromszStrv(cstr_t **ppThis, uchar *fmt, va_list ap) +{ + DEFiRet; + cstr_t *pThis; + va_list ap2; + uchar *sz; + int len; + + assert(ppThis != NULL); + + va_copy(ap2, ap); + len = vsnprintf(NULL, 0, fmt, ap2); + va_end(ap2); + + if(len < 0) + ABORT_FINALIZE(RS_RET_ERR); + + CHKiRet(rsCStrConstruct(&pThis)); + + pThis->iBufSize = pThis->iStrLen = len; + len++; /* account for the \0 written by vsnprintf */ + if((pThis->pBuf = (uchar*) MALLOC(sizeof(uchar) * len)) == NULL) { + RSFREEOBJ(pThis); + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } + + vsnprintf(pThis->pBuf, len, fmt, ap); + *ppThis = pThis; +finalize_it: + RETiRet; +} + + +/* construct from a printf-style formated string + */ +rsRetVal rsCStrConstructFromszStrf(cstr_t **ppThis, uchar *fmt, ...) +{ + DEFiRet; + va_list ap; + + va_start(ap, fmt); + iRet = rsCStrConstructFromszStrv(ppThis, fmt, ap); + va_end(ap); + + RETiRet; +} + + /* construct from es_str_t string * rgerhards 2010-12-03 */ @@ -256,6 +307,27 @@ rsRetVal cstrAppendCStr(cstr_t *pThis, cstr_t *pstrAppend) } +/* append a printf-style formated string + */ +rsRetVal rsCStrAppendStrf(cstr_t *pThis, uchar *fmt, ...) +{ + DEFiRet; + va_list ap; + cstr_t *pStr; + + va_start(ap, fmt); + iRet = rsCStrConstructFromszStrv(&pStr, fmt, ap); + va_end(ap); + + CHKiRet(iRet); + + iRet = cstrAppendCStr(pThis, pStr); + rsCStrDestruct(pStr); +finalize_it: + RETiRet; +} + + rsRetVal rsCStrAppendInt(cstr_t *pThis, long i) { DEFiRet; diff --git a/runtime/stringbuf.h b/runtime/stringbuf.h index bba004a0..b301f4b9 100644 --- a/runtime/stringbuf.h +++ b/runtime/stringbuf.h @@ -58,6 +58,7 @@ rsRetVal cstrConstruct(cstr_t **ppThis); rsRetVal cstrConstructFromESStr(cstr_t **ppThis, es_str_t *str); rsRetVal rsCStrConstructFromszStr(cstr_t **ppThis, uchar *sz); rsRetVal rsCStrConstructFromCStr(cstr_t **ppThis, cstr_t *pFrom); +rsRetVal rsCStrConstructFromszStrf(cstr_t **ppThis, uchar *fmt, ...); /** * Destruct the string buffer object. @@ -173,6 +174,12 @@ rsRetVal rsCStrAppendStr(cstr_t *pThis, uchar* psz); */ rsRetVal rsCStrAppendStrWithLen(cstr_t *pThis, uchar* psz, size_t iStrLen); +/** + * Append a printf-style formated string to the buffer. + * + * \param fmt pointer to the format string (see man 3 printf for details). Must not be NULL. + */ +rsRetVal rsCStrAppendStrf(cstr_t *pThis, uchar *fmt, ...); /** * Append an integer to the string. No special formatting is diff --git a/runtime/wtp.c b/runtime/wtp.c index f8d3588b..19151e7c 100644 --- a/runtime/wtp.c +++ b/runtime/wtp.c @@ -381,9 +381,9 @@ wtpWorker(void *arg) /* the arg is actually a wti object, even though we are in if(prctl(PR_SET_NAME, thrdName, 0, 0, 0) != 0) { DBGPRINTF("prctl failed, not setting thread name for '%s'\n", wtpGetDbgHdr(pThis)); } + dbgOutputTID((char*)thrdName); # endif - dbgOutputTID((char*)thrdName); pthread_cleanup_push(wtpWrkrExecCancelCleanup, pWti); wtiWorker(pWti); pthread_cleanup_pop(0); @@ -79,6 +79,7 @@ static struct cnfparamdescr cnfparamdescrProperty[] = { { "format", eCmdHdlrString, 0 }, { "position.from", eCmdHdlrInt, 0 }, { "position.to", eCmdHdlrInt, 0 }, + { "position.relativetoend", eCmdHdlrBinary, 0 }, { "field.number", eCmdHdlrInt, 0 }, { "field.delimiter", eCmdHdlrInt, 0 }, { "regex.expression", eCmdHdlrString, 0 }, @@ -712,6 +713,8 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) pTpe->data.field.options.bSecPathDrop = 1; } else if(!strcmp((char*)Buf, "secpath-replace")) { pTpe->data.field.options.bSecPathReplace = 1; + } else if(!strcmp((char*)Buf, "pos-end-relative")) { + pTpe->data.field.options.bFromPosEndRelative = 1; } else if(!strcmp((char*)Buf, "csv")) { if(pTpe->data.field.options.bJSON || pTpe->data.field.options.bJSONf) { errmsg.LogError(0, NO_ERRCODE, "error: can only specify " @@ -1057,18 +1060,27 @@ do_Parameter(uchar **pp, struct template *pTpl) #endif /* #ifdef FEATURE_REGEXP */ } - if(pTpe->data.field.iToPos < pTpe->data.field.iFromPos) { - iNum = pTpe->data.field.iToPos; - pTpe->data.field.iToPos = pTpe->data.field.iFromPos; - pTpe->data.field.iFromPos = iNum; - } - /* check options */ if(*p == ':') { ++p; /* eat ':' */ doOptions(&p, pTpe); } + if(pTpe->data.field.options.bFromPosEndRelative) { + if(pTpe->data.field.iToPos > pTpe->data.field.iFromPos) { + iNum = pTpe->data.field.iToPos; + pTpe->data.field.iToPos = pTpe->data.field.iFromPos; + pTpe->data.field.iFromPos = iNum; + } + } else { + if(pTpe->data.field.iToPos < pTpe->data.field.iFromPos) { + iNum = pTpe->data.field.iToPos; + pTpe->data.field.iToPos = pTpe->data.field.iFromPos; + pTpe->data.field.iFromPos = iNum; + } + } + + /* check field name */ if(*p == ':') { ++p; /* eat ':' */ @@ -1356,6 +1368,7 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) int re_matchToUse = 0; int re_submatchToUse = 0; int bComplexProcessing = 0; + int bPosRelativeToEnd = 0; char *re_expr = NULL; struct cnfparamvals *pvals = NULL; enum {F_NONE, F_CSV, F_JSON, F_JSONF} formatType = F_NONE; @@ -1395,6 +1408,8 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) } else if(!strcmp(pblkProperty.descr[i].name, "position.to")) { topos = pvals[i].val.d.n; bComplexProcessing = 1; + } else if(!strcmp(pblkProperty.descr[i].name, "position.relativetoend")) { + bPosRelativeToEnd = pvals[i].val.d.n; } else if(!strcmp(pblkProperty.descr[i].name, "field.number")) { fieldnum = pvals[i].val.d.n; bComplexProcessing = 1; @@ -1537,10 +1552,18 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) topos = 2000000000; /* large enough ;) */ if(frompos == -1 && topos != -1) frompos = 0; - if(topos < frompos) { - errmsg.LogError(0, RS_RET_ERR, "position.to=%d is lower than postion.from=%d\n", - topos, frompos); - ABORT_FINALIZE(RS_RET_ERR); + if(bPosRelativeToEnd) { + if(topos > frompos) { + errmsg.LogError(0, RS_RET_ERR, "position.to=%d is higher than postion.from=%d in 'relativeToEnd' mode\n", + topos, frompos); + ABORT_FINALIZE(RS_RET_ERR); + } + } else { + if(topos < frompos) { + errmsg.LogError(0, RS_RET_ERR, "position.to=%d is lower than postion.from=%d\n", + topos, frompos); + ABORT_FINALIZE(RS_RET_ERR); + } } if(fieldnum != -1 && re_expr != NULL) { errmsg.LogError(0, RS_RET_ERR, "both field extraction and regex extraction " @@ -1613,6 +1636,7 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) if(frompos != -1) { pTpe->data.field.iFromPos = frompos; pTpe->data.field.iToPos = topos; + pTpe->data.field.options.bFromPosEndRelative = bPosRelativeToEnd; } if(re_expr != NULL) { rsRetVal iRetLocal; @@ -119,6 +119,7 @@ struct templateEntry { unsigned bJSON: 1; /* format field JSON escaped */ unsigned bJSONf: 1; /* format field JSON *field* (n/v pair) */ unsigned bMandatory: 1; /* mandatory field - emit even if empty */ + unsigned bFromPosEndRelative: 1;/* is From/To-Pos relative to end of string? */ } options; /* options as bit fields */ } field; } data; diff --git a/tests/queue-persist-drvr.sh b/tests/queue-persist-drvr.sh index de597308..823fed6c 100755 --- a/tests/queue-persist-drvr.sh +++ b/tests/queue-persist-drvr.sh @@ -26,5 +26,10 @@ source $srcdir/diag.sh startup queue-persist.conf source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages ./msleep 500 $srcdir/diag.sh wait-shutdown -source $srcdir/diag.sh seq-check 0 4999 +# note: we need to permit duplicate messages, as due to the forced +# shutdown some messages may be flagged as "unprocessed" while they +# actually were processed. This is inline with rsyslog's philosophy +# to better duplicate than loose messages. Duplicate messages are +# permitted by the -d seq-check option. +source $srcdir/diag.sh seq-check 0 4999 -d source $srcdir/diag.sh exit @@ -183,10 +183,10 @@ static void* thrdStarter(void *arg) assert(pThis != NULL); assert(pThis->pUsrThrdMain != NULL); +# if HAVE_PRCTL && defined PR_SET_NAME ustrncpy(thrdName+3, pThis->name, 20); dbgOutputTID((char*)thrdName); -# if HAVE_PRCTL && defined PR_SET_NAME /* set thread name - we ignore if the call fails, has no harsh consequences... */ if(prctl(PR_SET_NAME, thrdName, 0, 0, 0) != 0) { DBGPRINTF("prctl failed, not setting thread name for '%s'\n", pThis->name); diff --git a/tools/Makefile.am b/tools/Makefile.am index 9d9bd352..938782f7 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -41,7 +41,12 @@ rsyslogd_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) # otherwise dependencies are not properly calculated (resulting in a # potentially incomplete build, a problem we had several times...) rsyslogd_LDADD = ../grammar/libgrammar.la ../runtime/librsyslog.la $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBEE_LIBS) $(LIBLOGNORM_LIBS) $(LIBUUID_LIBS) -rsyslogd_LDFLAGS = -export-dynamic +rsyslogd_LDFLAGS = -export-dynamic `libgcrypt-config --libs` +#rsyslogd_LDFLAGS = -export-dynamic $(LIBGCRYPT_LIBS) + +EXTRA_DIST = $(man_MANS) \ + rsgtutil.rst \ + recover_qi.pl if ENABLE_DIAGTOOLS sbin_PROGRAMS += rsyslog_diag_hostname msggen zpipe @@ -58,7 +63,27 @@ logctl_SOURCES = logctl.c logctl_CPPFLAGS = $(LIBMONGO_CLIENT_CFLAGS) logctl_LDADD = $(LIBMONGO_CLIENT_LIBS) endif +if ENABLE_GUARDTIME +bin_PROGRAMS += rsgtutil +rsgtutil = rsgtutil.c +rsgtutil_CPPFLAGS = $(RSRT_CFLAGS) $(GUARDTIME_CFLAGS) +rsgtutil_LDADD = ../runtime/librsgt.la $(GUARDTIME_LIBS) +rsgtutil.1: rsgtutil.rst + $(AM_V_GEN) $(RST2MAN) $< $@ +man1_MANS = rsgtutil.1 +CLEANFILES = rsgtutil.1 +EXTRA_DIST+= rsgtutil.1 +endif +if ENABLE_LIBGCRYPT +bin_PROGRAMS += rscryutil +rscryutil = rscryutil.c +rscryutil_CPPFLAGS = -I../runtime $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) +rscryutil_LDFLAGS = `libgcrypt-config --libs` +rscryutil_LDADD = ../runtime/libgcry.la $(LIBGCRYPT_LIBS) +rscryutil.1: rscryutil.rst + $(AM_V_GEN) $(RST2MAN) $< $@ +man1_MANS = rscryutil.1 +CLEANFILES = rscryutil.1 +EXTRA_DIST+= rscryutil.1 +endif endif - -EXTRA_DIST = $(man_MANS) \ - recover_qi.pl diff --git a/tools/logsigner.c b/tools/logsigner.c new file mode 100644 index 00000000..f6887696 --- /dev/null +++ b/tools/logsigner.c @@ -0,0 +1,159 @@ +/* This is a tool for offline signing logfiles via the guardtime API. + * + * NOTE: this currently is only a PoC and WiP! NOT suitable for + * production use! + * + * Current hardcoded timestamper (use this if you do not have an + * idea of which one else to use): + * http://stamper.guardtime.net/gt-signingservice + * Check the GuardTime website for the URLs of nearest public services. + * + * Copyright 2013 Adiscon GmbH + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either exprs or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <gt_base.h> +#include <gt_http.h> + +#include "librsgt.h" + + +#if 0 +void +outputhash(GTDataHash *hash) +{ + int i; + for(i = 0 ; i < hash->digest_length ; ++i) + printf("%2.2x", hash->digest[i]); + printf("\n"); +} + +void +timestampIt(GTDataHash *hash) +{ + int r = GT_OK; + GTTimestamp *timestamp = NULL; + unsigned char *der = NULL; + char *sigFile = "logsigner.TIMESTAMP"; + size_t der_len; + + /* Get the timestamp. */ + r = GTHTTP_createTimestampHash(hash, + "http://stamper.guardtime.net/gt-signingservice", ×tamp); + + if(r != GT_OK) { + fprintf(stderr, "GTHTTP_createTimestampHash() failed: %d (%s)\n", + r, GTHTTP_getErrorString(r)); + goto done; + } + + /* Encode timestamp. */ + r = GTTimestamp_getDEREncoded(timestamp, &der, &der_len); + if(r != GT_OK) { + fprintf(stderr, "GTTimestamp_getDEREncoded() failed: %d (%s)\n", + r, GT_getErrorString(r)); + goto done; + } + + /* Save DER-encoded timestamp to file. */ + r = GT_saveFile(sigFile, der, der_len); + if(r != GT_OK) { + fprintf(stderr, "Cannot save timestamp to file %s: %d (%s)\n", + sigFile, r, GT_getErrorString(r)); + if(r == GT_IO_ERROR) { + fprintf(stderr, "\t%d (%s)\n", errno, strerror(errno)); + } + goto done; + } + printf("Timestamping succeeded!\n"); +done: + GT_free(der); + GTTimestamp_free(timestamp); +} + + +void +sign(const char *buf, const size_t len) +{ + int r; + GTDataHash *hash = NULL; + + printf("hash for '%s' is ", buf); + r = GTDataHash_create(GT_HASHALG_SHA256, (const unsigned char*)buf, len, &hash); + if(r != GT_OK) { + fprintf(stderr, "GTTDataHash_create() failed: %d (%s)\n", + r, GT_getErrorString(r)); + goto done; + } + outputhash(hash); + timestampIt(hash); /* of course, this needs to be moved to once at end ;) */ +done: GTDataHash_free(hash); +} +#endif + +void +processFile(char *name) +{ + FILE *fp; + size_t len; + char line[64*1024+1]; + gtctx ctx = NULL; + + ctx = rsgtCtxNew((unsigned char*)"SIGFILE", GT_HASHALG_SHA256); + sigblkInit(ctx); + if(!strcmp(name, "-")) + fp = stdin; + else + fp = fopen(name, "r"); + + while(1) { + if(fgets(line, sizeof(line), fp) == NULL) { + if(!feof(fp)) + perror(name); + break; + } + len = strlen(line); + if(line[len-1] == '\n') { + --len; + line[len] = '\0'; + } + //sign(line, len); + sigblkAddRecord(ctx, (unsigned char*)line, len); + } + + if(fp != stdin) + fclose(fp); + sigblkFinish(ctx); + rsgtCtxDel(ctx); +} + + +int +main(int argc, char *argv[]) +{ + rsgtInit("rsyslog logsigner " VERSION); + processFile("-"); + rsgtExit(); + return 0; +} diff --git a/tools/omfile.c b/tools/omfile.c index 1c65fc59..46d882bf 100644 --- a/tools/omfile.c +++ b/tools/omfile.c @@ -17,7 +17,7 @@ * pipes. These have been moved to ompipe, to reduced the entanglement * between the two different functionalities. -- rgerhards * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -69,6 +69,8 @@ #include "unicode-helper.h" #include "atomic.h" #include "statsobj.h" +#include "sigprov.h" +#include "cryprov.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP @@ -118,6 +120,7 @@ getClockFileAccess(void) struct s_dynaFileCacheEntry { uchar *pName; /* name currently open, if dynamic name */ strm_t *pStrm; /* our output stream */ + void *sigprovFileData; /* opaque data ptr for provider use */ uint64 clkTickAccessed;/* for LRU - based on clockFileAccess */ }; typedef struct s_dynaFileCacheEntry dynaFileCacheEntry; @@ -143,6 +146,18 @@ typedef struct _instanceData { gid_t fileGID; gid_t dirGID; int bFailOnChown; /* fail creation if chown fails? */ + uchar *sigprovName; /* signature provider */ + uchar *sigprovNameFull;/* full internal signature provider name */ + sigprov_if_t sigprov; /* ptr to signature provider interface */ + void *sigprovData; /* opaque data ptr for provider use */ + void *sigprovFileData;/* opaque data ptr for file instance */ + sbool useSigprov; /* quicker than checkig ptr (1 vs 8 bytes!) */ + uchar *cryprovName; /* crypto provider */ + uchar *cryprovNameFull;/* full internal crypto provider name */ + void *cryprovData; /* opaque data ptr for provider use */ + void *cryprovFileData;/* opaque data ptr for file instance */ + cryprov_if_t cryprov; /* ptr to crypto provider interface */ + sbool useCryprov; /* quicker than checkig ptr (1 vs 8 bytes!) */ int iCurrElt; /* currently active cache element (-1 = none) */ int iCurrCacheSize; /* currently cache size (1-based) */ int iDynaFileCacheSize; /* size of file handle cache */ @@ -228,7 +243,9 @@ static struct cnfparamdescr actpdescr[] = { { "sync", eCmdHdlrBinary, 0 }, /* legacy: actionfileenablesync */ { "file", eCmdHdlrString, 0 }, /* either "file" or ... */ { "dynafile", eCmdHdlrString, 0 }, /* "dynafile" MUST be present */ - { "template", eCmdHdlrGetWord, 0 }, + { "sig.provider", eCmdHdlrGetWord, 0 }, + { "cry.provider", eCmdHdlrGetWord, 0 }, + { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = { CNFPARAMBLK_VERSION, @@ -416,15 +433,16 @@ finalize_it: * if the entry should be d_free()ed and 0 if not. */ static rsRetVal -dynaFileDelCacheEntry(dynaFileCacheEntry **pCache, int iEntry, int bFreeEntry) +dynaFileDelCacheEntry(instanceData *pData, int iEntry, int bFreeEntry) { + dynaFileCacheEntry **pCache = pData->dynCache; DEFiRet; ASSERT(pCache != NULL); if(pCache[iEntry] == NULL) FINALIZE; - DBGPRINTF("Removed entry %d for file '%s' from dynaCache.\n", iEntry, + DBGPRINTF("Removing entry %d for file '%s' from dynaCache.\n", iEntry, pCache[iEntry]->pName == NULL ? UCHAR_CONSTANT("[OPEN FAILED]") : pCache[iEntry]->pName); if(pCache[iEntry]->pName != NULL) { @@ -434,8 +452,10 @@ dynaFileDelCacheEntry(dynaFileCacheEntry **pCache, int iEntry, int bFreeEntry) if(pCache[iEntry]->pStrm != NULL) { strm.Destruct(&pCache[iEntry]->pStrm); - if(pCache[iEntry]->pStrm != NULL) /* safety check -- TODO: remove if no longer necessary */ - abort(); + if(pData->useSigprov) { + pData->sigprov.OnFileClose(pCache[iEntry]->sigprovFileData); + pCache[iEntry]->sigprovFileData = NULL; + } } if(bFreeEntry) { @@ -460,7 +480,7 @@ dynaFileFreeCacheEntries(instanceData *pData) BEGINfunc; for(i = 0 ; i < pData->iCurrCacheSize ; ++i) { - dynaFileDelCacheEntry(pData->dynCache, i, 1); + dynaFileDelCacheEntry(pData, i, 1); } pData->iCurrElt = -1; /* invalidate current element */ ENDfunc; @@ -481,6 +501,29 @@ static void dynaFileFreeCache(instanceData *pData) } +/* close current file */ +static rsRetVal +closeFile(instanceData *pData) +{ + DEFiRet; + if(pData->useSigprov) { + pData->sigprov.OnFileClose(pData->sigprovFileData); + pData->sigprovFileData = NULL; + } + strm.Destruct(&pData->pStrm); + RETiRet; +} + + +/* This prepares the signature provider to process a file */ +static rsRetVal +sigprovPrepare(instanceData *pData, uchar *fn) +{ + DEFiRet; + pData->sigprov.OnFileOpen(pData->sigprovData, fn, &pData->sigprovFileData); + RETiRet; +} + /* This is now shared code for all types of files. It simply prepares * file access, which, among others, means the the file wil be opened * and any directories in between will be created (based on config, of @@ -554,6 +597,10 @@ prepareFile(instanceData *pData, uchar *newFileName) CHKiRet(strm.SetbSync(pData->pStrm, pData->bSyncFile)); CHKiRet(strm.SetsType(pData->pStrm, STREAMTYPE_FILE_SINGLE)); CHKiRet(strm.SetiSizeLimit(pData->pStrm, pData->iSizeLimit)); + if(pData->useCryprov) { + CHKiRet(strm.Setcryprov(pData->pStrm, &pData->cryprov)); + CHKiRet(strm.SetcryprovData(pData->pStrm, pData->cryprovData)); + } /* set the flush interval only if we actually use it - otherwise it will activate * async processing, which is a real performance waste if we do not do buffered * writes! -- rgerhards, 2009-07-06 @@ -563,11 +610,14 @@ prepareFile(instanceData *pData, uchar *newFileName) if(pData->pszSizeLimitCmd != NULL) CHKiRet(strm.SetpszSizeLimitCmd(pData->pStrm, ustrdup(pData->pszSizeLimitCmd))); CHKiRet(strm.ConstructFinalize(pData->pStrm)); + + if(pData->useSigprov) + sigprovPrepare(pData, szNameBuf); finalize_it: if(iRet != RS_RET_OK) { if(pData->pStrm != NULL) { - strm.Destruct(&pData->pStrm); + closeFile(pData); } } RETiRet; @@ -598,9 +648,7 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) pCache = pData->dynCache; - /* first check, if we still have the current file - * I *hope* this will be a performance enhancement. - */ + /* first check, if we still have the current file */ if( (pData->iCurrElt != -1) && !ustrcmp(newFileName, pCache[pData->iCurrElt]->pName)) { /* great, we are all set */ @@ -622,9 +670,11 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) if(iFirstFree == -1) iFirstFree = i; } else { /* got an element, let's see if it matches */ - if(!ustrcmp(newFileName, pCache[i]->pName)) { // RG: name == NULL? + if(!ustrcmp(newFileName, pCache[i]->pName)) { /* we found our element! */ pData->pStrm = pCache[i]->pStrm; + if(pData->useSigprov) + pData->sigprovFileData = pCache[i]->sigprovFileData; pData->iCurrElt = i; pCache[i]->clkTickAccessed = getClockFileAccess(); /* update "timestamp" for LRU */ FINALIZE; @@ -651,7 +701,7 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) * but it could be triggered in the common case of a failed open() system call. * rgerhards, 2010-03-22 */ - pData->pStrm = NULL; + pData->pStrm = NULL, pData->sigprovFileData = NULL; if(iFirstFree == -1 && (pData->iCurrCacheSize < pData->iDynaFileCacheSize)) { /* there is space left, so set it to that index */ @@ -664,14 +714,11 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) * The cache array is only updated after the open was successful. -- rgerhards, 2010-03-21 */ if(iFirstFree == -1) { - dynaFileDelCacheEntry(pCache, iOldest, 0); + dynaFileDelCacheEntry(pData, iOldest, 0); STATSCOUNTER_INC(pData->ctrEvict, pData->mutCtrEvict); iFirstFree = iOldest; /* this one *is* now free ;) */ } else { /* we need to allocate memory for the cache structure */ - /* TODO: performance note: we could alloc all entries on startup, thus saving malloc - * overhead -- this may be something to consider in v5... - */ CHKmalloc(pCache[iFirstFree] = (dynaFileCacheEntry*) calloc(1, sizeof(dynaFileCacheEntry))); } @@ -694,10 +741,12 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) } if((pCache[iFirstFree]->pName = ustrdup(newFileName)) == NULL) { - strm.Destruct(&pData->pStrm); /* need to free failed entry! */ + closeFile(pData); /* need to free failed entry! */ ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); } pCache[iFirstFree]->pStrm = pData->pStrm; + if(pData->useSigprov) + pCache[iFirstFree]->sigprovFileData = pData->sigprovFileData; pCache[iFirstFree]->clkTickAccessed = getClockFileAccess(); pData->iCurrElt = iFirstFree; DBGPRINTF("Added new entry %d for file cache, file '%s'.\n", iFirstFree, newFileName); @@ -722,7 +771,9 @@ doWrite(instanceData *pData, uchar *pszBuf, int lenBuf) DBGPRINTF("write to stream, pData->pStrm %p, lenBuf %d\n", pData->pStrm, lenBuf); if(pData->pStrm != NULL){ CHKiRet(strm.Write(pData->pStrm, pszBuf, lenBuf)); - FINALIZE; + if(pData->useSigprov) { + CHKiRet(pData->sigprov.OnRecordWrite(pData->sigprovFileData, pszBuf, lenBuf)); + } } finalize_it: @@ -730,10 +781,7 @@ finalize_it: } -/* rgerhards 2004-11-11: write to a file output. This - * will be called for all outputs using file semantics, - * for example also for pipes. - */ +/* rgerhards 2004-11-11: write to a file output. */ static rsRetVal writeFile(uchar **ppString, unsigned iMsgOpts, instanceData *pData) { @@ -841,7 +889,21 @@ CODESTARTfreeInstance if(pData->bDynamicName) { dynaFileFreeCache(pData); } else if(pData->pStrm != NULL) - strm.Destruct(&pData->pStrm); + closeFile(pData); + if(pData->useSigprov) { + pData->sigprov.Destruct(&pData->sigprovData); + obj.ReleaseObj(__FILE__, pData->sigprovNameFull+2, pData->sigprovNameFull, + (void*) &pData->sigprov); + free(pData->sigprovName); + free(pData->sigprovNameFull); + } + if(pData->useCryprov) { + pData->cryprov.Destruct(&pData->cryprovData); + obj.ReleaseObj(__FILE__, pData->cryprovNameFull+2, pData->cryprovNameFull, + (void*) &pData->cryprov); + free(pData->cryprovName); + free(pData->cryprovNameFull); + } ENDfreeInstance @@ -907,6 +969,10 @@ setInstParamDefaults(instanceData *pData) pData->iIOBufSize = IOBUF_DFLT_SIZE; pData->iFlushInterval = FLUSH_INTRVL_DFLT; pData->bUseAsyncWriter = USE_ASYNCWRITER_DFLT; + pData->sigprovName = NULL; + pData->cryprovName = NULL; + pData->useSigprov = 0; + pData->useCryprov = 0; } @@ -946,6 +1012,92 @@ finalize_it: RETiRet; } +static inline void +initSigprov(instanceData *pData, struct nvlst *lst) +{ + uchar szDrvrName[1024]; + + if(snprintf((char*)szDrvrName, sizeof(szDrvrName), "lmsig_%s", pData->sigprovName) + == sizeof(szDrvrName)) { + errmsg.LogError(0, RS_RET_ERR, "omfile: signature provider " + "name is too long: '%s' - signatures disabled", + pData->sigprovName); + goto done; + } + pData->sigprovNameFull = ustrdup(szDrvrName); + + pData->sigprov.ifVersion = sigprovCURR_IF_VERSION; + /* The pDrvrName+2 below is a hack to obtain the object name. It + * safes us to have yet another variable with the name without "lm" in + * front of it. If we change the module load interface, we may re-think + * about this hack, but for the time being it is efficient and clean enough. + */ + if(obj.UseObj(__FILE__, szDrvrName, szDrvrName, (void*) &pData->sigprov) + != RS_RET_OK) { + errmsg.LogError(0, RS_RET_LOAD_ERROR, "omfile: could not load " + "signature provider '%s' - signatures disabled", + szDrvrName); + goto done; + } + + if(pData->sigprov.Construct(&pData->sigprovData) != RS_RET_OK) { + errmsg.LogError(0, RS_RET_SIGPROV_ERR, "omfile: error constructing " + "signature provider %s dataset - signatures disabled", + szDrvrName); + goto done; + } + pData->sigprov.SetCnfParam(pData->sigprovData, lst); + + dbgprintf("loaded signature provider %s, data instance at %p\n", + szDrvrName, pData->sigprovData); + pData->useSigprov = 1; +done: return; +} + +static inline rsRetVal +initCryprov(instanceData *pData, struct nvlst *lst) +{ + uchar szDrvrName[1024]; + DEFiRet; + + if(snprintf((char*)szDrvrName, sizeof(szDrvrName), "lmcry_%s", pData->cryprovName) + == sizeof(szDrvrName)) { + errmsg.LogError(0, RS_RET_ERR, "omfile: crypto provider " + "name is too long: '%s' - encryption disabled", + pData->cryprovName); + ABORT_FINALIZE(RS_RET_ERR); + } + pData->cryprovNameFull = ustrdup(szDrvrName); + + pData->cryprov.ifVersion = cryprovCURR_IF_VERSION; + /* The pDrvrName+2 below is a hack to obtain the object name. It + * safes us to have yet another variable with the name without "lm" in + * front of it. If we change the module load interface, we may re-think + * about this hack, but for the time being it is efficient and clean enough. + */ + if(obj.UseObj(__FILE__, szDrvrName, szDrvrName, (void*) &pData->cryprov) + != RS_RET_OK) { + errmsg.LogError(0, RS_RET_LOAD_ERROR, "omfile: could not load " + "crypto provider '%s' - encryption disabled", + szDrvrName); + ABORT_FINALIZE(RS_RET_CRYPROV_ERR); + } + + if(pData->cryprov.Construct(&pData->cryprovData) != RS_RET_OK) { + errmsg.LogError(0, RS_RET_CRYPROV_ERR, "omfile: error constructing " + "crypto provider %s dataset - encryption disabled", + szDrvrName); + ABORT_FINALIZE(RS_RET_CRYPROV_ERR); + } + CHKiRet(pData->cryprov.SetCnfParam(pData->cryprovData, lst)); + + dbgprintf("loaded crypto provider %s, data instance at %p\n", + szDrvrName, pData->cryprovData); + pData->useCryprov = 1; +finalize_it: + RETiRet; +} + BEGINnewActInst struct cnfparamvals *pvals; uchar *tplToUse; @@ -1013,6 +1165,10 @@ CODESTARTnewActInst pData->bDynamicName = 1; } else if(!strcmp(actpblk.descr[i].name, "template")) { pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "sig.provider")) { + pData->sigprovName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "cry.provider")) { + pData->cryprovName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else { dbgprintf("omfile: program error, non-handled " "param '%s'\n", actpblk.descr[i].name); @@ -1025,6 +1181,14 @@ CODESTARTnewActInst ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); } + if(pData->sigprovName != NULL) { + initSigprov(pData, lst); + } + + if(pData->cryprovName != NULL) { + CHKiRet(initCryprov(pData, lst)); + } + tplToUse = ustrdup((pData->tplName == NULL) ? getDfltTpl() : pData->tplName); CHKiRet(OMSRsetEntry(*ppOMSR, 0, tplToUse, OMSR_NO_RQD_TPL_OPTS)); @@ -1167,8 +1331,7 @@ CODESTARTdoHUP dynaFileFreeCacheEntries(pData); } else { if(pData->pStrm != NULL) { - strm.Destruct(&pData->pStrm); - pData->pStrm = NULL; + closeFile(pData); } } ENDdoHUP diff --git a/tools/pmrfc3164.c b/tools/pmrfc3164.c index bcded428..5dfa74f0 100644 --- a/tools/pmrfc3164.c +++ b/tools/pmrfc3164.c @@ -138,7 +138,7 @@ CODESTARTparse */ if(lenMsg > 0 && pMsg->msgFlags & PARSE_HOSTNAME) { i = 0; - while(i < lenMsg && (isalnum(p2parse[i]) || p2parse[i] == '.' || p2parse[i] == '.' + while(i < lenMsg && (isalnum(p2parse[i]) || p2parse[i] == '.' || p2parse[i] == '_' || p2parse[i] == '-') && i < (CONF_HOSTNAME_MAXSIZE - 1)) { bufParseHOSTNAME[i] = p2parse[i]; ++i; diff --git a/tools/rscryutil.c b/tools/rscryutil.c new file mode 100644 index 00000000..2591b2cc --- /dev/null +++ b/tools/rscryutil.c @@ -0,0 +1,512 @@ +/* This is a tool for processing rsyslog encrypted log files. + * + * Copyright 2013 Adiscon GmbH + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either exprs or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <unistd.h> +#include <stdio.h> +#include <getopt.h> +#include <fcntl.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <gcrypt.h> + +#include "rsyslog.h" +#include "libgcry.h" + + +static enum { MD_DECRYPT, MD_WRITE_KEYFILE +} mode = MD_DECRYPT; +static int verbose = 0; +static gcry_cipher_hd_t gcry_chd; +static size_t blkLength; + +static char *keyfile = NULL; +static char *keyprog = NULL; +static int randomKeyLen = -1; +static char *cry_key = NULL; +static unsigned cry_keylen = 0; +static int cry_algo = GCRY_CIPHER_AES128; +static int cry_mode = GCRY_CIPHER_MODE_CBC; +static int optionForce = 0; + +/* rectype/value must be EIF_MAX_*_LEN+1 long! + * returns 0 on success or something else on error/EOF + */ +static int +eiGetRecord(FILE *eifp, char *rectype, char *value) +{ + int r; + unsigned short i, j; + char buf[EIF_MAX_RECTYPE_LEN+EIF_MAX_VALUE_LEN+128]; + /* large enough for any valid record */ + + if(fgets(buf, sizeof(buf), eifp) == NULL) { + r = 1; goto done; + } + + for(i = 0 ; i < EIF_MAX_RECTYPE_LEN && buf[i] != ':' ; ++i) + if(buf[i] == '\0') { + r = 2; goto done; + } else + rectype[i] = buf[i]; + rectype[i] = '\0'; + j = 0; + for(++i ; i < EIF_MAX_VALUE_LEN && buf[i] != '\n' ; ++i, ++j) + if(buf[i] == '\0') { + r = 3; goto done; + } else + value[j] = buf[i]; + value[j] = '\0'; + r = 0; +done: return r; +} + +static int +eiCheckFiletype(FILE *eifp) +{ + char rectype[EIF_MAX_RECTYPE_LEN+1]; + char value[EIF_MAX_VALUE_LEN+1]; + int r; + + if((r = eiGetRecord(eifp, rectype, value)) != 0) goto done; + if(strcmp(rectype, "FILETYPE") || strcmp(value, RSGCRY_FILETYPE_NAME)) { + fprintf(stderr, "invalid filetype \"cookie\" in encryption " + "info file\n"); + fprintf(stderr, "\trectype: '%s', value: '%s'\n", rectype, value); + r = 1; goto done; + } + r = 0; +done: return r; +} + +static int +eiGetIV(FILE *eifp, char *iv, size_t leniv) +{ + char rectype[EIF_MAX_RECTYPE_LEN+1]; + char value[EIF_MAX_VALUE_LEN+1]; + size_t valueLen; + unsigned short i, j; + int r; + unsigned char nibble; + + if((r = eiGetRecord(eifp, rectype, value)) != 0) goto done; + if(strcmp(rectype, "IV")) { + fprintf(stderr, "no IV record found when expected, record type " + "seen is '%s'\n", rectype); + r = 1; goto done; + } + valueLen = strlen(value); + if(valueLen/2 != leniv) { + fprintf(stderr, "length of IV is %d, expected %d\n", + valueLen/2, leniv); + r = 1; goto done; + } + + for(i = j = 0 ; i < valueLen ; ++i) { + if(value[i] >= '0' && value[i] <= '9') + nibble = value[i] - '0'; + else if(value[i] >= 'a' && value[i] <= 'f') + nibble = value[i] - 'a' + 10; + else { + fprintf(stderr, "invalid IV '%s'\n", value); + r = 1; goto done; + } + if(i % 2 == 0) + iv[j] = nibble << 4; + else + iv[j++] |= nibble; + } + r = 0; +done: return r; +} + +static int +eiGetEND(FILE *eifp, off64_t *offs) +{ + char rectype[EIF_MAX_RECTYPE_LEN+1]; + char value[EIF_MAX_VALUE_LEN+1]; + int r; + + if((r = eiGetRecord(eifp, rectype, value)) != 0) goto done; + if(strcmp(rectype, "END")) { + fprintf(stderr, "no END record found when expected, record type " + "seen is '%s'\n", rectype); + r = 1; goto done; + } + *offs = atoll(value); + r = 0; +done: return r; +} + +static int +initCrypt(FILE *eifp) +{ + int r = 0; + gcry_error_t gcryError; + char iv[4096]; + + blkLength = gcry_cipher_get_algo_blklen(cry_algo); + if(blkLength > sizeof(iv)) { + fprintf(stderr, "internal error[%s:%d]: block length %d too large for " + "iv buffer\n", __FILE__, __LINE__, blkLength); + r = 1; goto done; + } + if((r = eiGetIV(eifp, iv, blkLength)) != 0) goto done; + + size_t keyLength = gcry_cipher_get_algo_keylen(cry_algo); + if(strlen(cry_key) != keyLength) { + fprintf(stderr, "invalid key length; key is %u characters, but " + "exactly %u characters are required\n", cry_keylen, + keyLength); + r = 1; goto done; + } + + gcryError = gcry_cipher_open(&gcry_chd, cry_algo, cry_mode, 0); + if (gcryError) { + printf("gcry_cipher_open failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + r = 1; goto done; + } + + gcryError = gcry_cipher_setkey(gcry_chd, cry_key, keyLength); + if (gcryError) { + printf("gcry_cipher_setkey failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + r = 1; goto done; + } + + gcryError = gcry_cipher_setiv(gcry_chd, iv, blkLength); + if (gcryError) { + printf("gcry_cipher_setiv failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + r = 1; goto done; + } +done: return r; +} + +static inline void +removePadding(char *buf, size_t *plen) +{ + unsigned len = (unsigned) *plen; + unsigned iSrc, iDst; + char *frstNUL; + + frstNUL = memchr(buf, 0x00, *plen); + if(frstNUL == NULL) + goto done; + iDst = iSrc = frstNUL - buf; + + while(iSrc < len) { + if(buf[iSrc] != 0x00) + buf[iDst++] = buf[iSrc]; + ++iSrc; + } + + *plen = iDst; +done: return; +} + +static void +decryptBlock(FILE *fpin, FILE *fpout, off64_t blkEnd, off64_t *pCurrOffs) +{ + gcry_error_t gcryError; + size_t nRead, nWritten; + size_t toRead; + size_t leftTillBlkEnd; + char buf[64*1024]; + + leftTillBlkEnd = blkEnd - *pCurrOffs; + while(1) { + toRead = sizeof(buf) <= leftTillBlkEnd ? sizeof(buf) : leftTillBlkEnd; + toRead = toRead - toRead % blkLength; + nRead = fread(buf, 1, toRead, fpin); + if(nRead == 0) + break; + leftTillBlkEnd -= nRead, *pCurrOffs += nRead; + gcryError = gcry_cipher_decrypt( + gcry_chd, // gcry_cipher_hd_t + buf, // void * + nRead, // size_t + NULL, // const void * + 0); // size_t + if (gcryError) { + fprintf(stderr, "gcry_cipher_decrypt failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + return; + } + removePadding(buf, &nRead); + nWritten = fwrite(buf, 1, nRead, fpout); + if(nWritten != nRead) { + perror("fpout"); + return; + } + } +} + + +static int +doDecrypt(FILE *logfp, FILE *eifp, FILE *outfp) +{ + off64_t blkEnd; + off64_t currOffs = 0; + int r; + + while(1) { + /* process block */ + if(initCrypt(eifp) != 0) + goto done; + if((r = eiGetEND(eifp, &blkEnd)) != 0) goto done; + decryptBlock(logfp, outfp, blkEnd, &currOffs); + gcry_cipher_close(gcry_chd); + } + r = 0; +done: return r; +} + +static void +decrypt(char *name) +{ + FILE *logfp = NULL, *eifp = NULL; + int r = 0; + char eifname[4096]; + + if(!strcmp(name, "-")) { + fprintf(stderr, "decrypt mode cannot work on stdin\n"); + goto err; + } else { + if((logfp = fopen(name, "r")) == NULL) { + perror(name); + goto err; + } + snprintf(eifname, sizeof(eifname), "%s%s", name, ENCINFO_SUFFIX); + eifname[sizeof(eifname)-1] = '\0'; + if((eifp = fopen(eifname, "r")) == NULL) { + perror(eifname); + goto err; + } + if(eiCheckFiletype(eifp) != 0) + goto err; + } + + doDecrypt(logfp, eifp, stdout); + + fclose(logfp); logfp = NULL; + fclose(eifp); eifp = NULL; + return; + +err: + fprintf(stderr, "error %d processing file %s\n", r, name); + if(logfp != NULL) + fclose(logfp); +} + +static void +write_keyfile(char *fn) +{ + int fd; + int r; + mode_t fmode; + + fmode = O_WRONLY|O_CREAT; + if(!optionForce) + fmode |= O_EXCL; + if((fd = open(fn, fmode, S_IRUSR)) == -1) { + fprintf(stderr, "error opening keyfile "); + perror(fn); + exit(1); + } + if((r = write(fd, cry_key, cry_keylen)) != (ssize_t)cry_keylen) { + fprintf(stderr, "error writing keyfile (ret=%d) ", r); + perror(fn); + exit(1); + } + close(fd); +} + +static void +getKeyFromFile(char *fn) +{ + int r; + r = gcryGetKeyFromFile(fn, &cry_key, &cry_keylen); + if(r != 0) { + fprintf(stderr, "Error %d reading key from file '%s'\n", r, fn); + exit(1); + } +} + +static void +getRandomKey(void) +{ + int fd; + cry_keylen = randomKeyLen; + cry_key = malloc(randomKeyLen); /* do NOT zero-out! */ + /* if we cannot obtain data from /dev/urandom, we use whatever + * is present at the current memory location as random data. Of + * course, this is very weak and we should consider a different + * option, especially when not running under Linux (for Linux, + * unavailability of /dev/urandom is just a theoretic thing, it + * will always work...). -- TODO -- rgerhards, 2013-03-06 + */ + if((fd = open("/dev/urandom", O_RDONLY)) > 0) { + if(read(fd, cry_key, randomKeyLen)) {}; /* keep compiler happy */ + close(fd); + } +} + + +static void +setKey() +{ + if(randomKeyLen != -1) + getRandomKey(); + else if(keyfile != NULL) + getKeyFromFile(keyfile); + else if(keyprog != NULL) + gcryGetKeyFromProg(keyprog, &cry_key, &cry_keylen); + if(cry_key == NULL) { + fprintf(stderr, "ERROR: key must be set via some method\n"); + exit(1); + } +} + +static struct option long_options[] = +{ + {"verbose", no_argument, NULL, 'v'}, + {"version", no_argument, NULL, 'V'}, + {"decrypt", no_argument, NULL, 'd'}, + {"force", no_argument, NULL, 'f'}, + {"write-keyfile", required_argument, NULL, 'W'}, + {"key", required_argument, NULL, 'K'}, + {"generate-random-key", required_argument, NULL, 'r'}, + {"keyfile", required_argument, NULL, 'k'}, + {"key-program", required_argument, NULL, 'p'}, + {"algo", required_argument, NULL, 'a'}, + {"mode", required_argument, NULL, 'm'}, + {NULL, 0, NULL, 0} +}; + +int +main(int argc, char *argv[]) +{ + int i; + int opt; + int temp; + char *newKeyFile = NULL; + + while(1) { + opt = getopt_long(argc, argv, "a:dfk:K:m:p:r:vVW:", long_options, NULL); + if(opt == -1) + break; + switch(opt) { + case 'd': + mode = MD_DECRYPT; + break; + case 'W': + mode = MD_WRITE_KEYFILE; + newKeyFile = optarg; + break; + case 'k': + keyfile = optarg; + break; + case 'p': + keyprog = optarg; + break; + case 'f': + optionForce = 1; + break; + case 'r': + randomKeyLen = atoi(optarg); + if(randomKeyLen > 64*1024) { + fprintf(stderr, "ERROR: keys larger than 64KiB are " + "not supported\n"); + exit(1); + } + break; + case 'K': + fprintf(stderr, "WARNING: specifying the actual key " + "via the command line is highly insecure\n" + "Do NOT use this for PRODUCTION use.\n"); + cry_key = optarg; + cry_keylen = strlen(cry_key); + break; + case 'a': + temp = rsgcryAlgoname2Algo(optarg); + if(temp == GCRY_CIPHER_NONE) { + fprintf(stderr, "ERROR: algorithm \"%s\" is not " + "kown/supported\n", optarg); + exit(1); + } + cry_algo = temp; + break; + case 'm': + temp = rsgcryModename2Mode(optarg); + if(temp == GCRY_CIPHER_MODE_NONE) { + fprintf(stderr, "ERROR: cipher mode \"%s\" is not " + "kown/supported\n", optarg); + exit(1); + } + cry_mode = temp; + break; + case 'v': + verbose = 1; + break; + case 'V': + fprintf(stderr, "rsgtutil " VERSION "\n"); + exit(0); + break; + case '?': + break; + default:fprintf(stderr, "getopt_long() returns unknown value %d\n", opt); + return 1; + } + } + + setKey(); + + if(mode == MD_WRITE_KEYFILE) { + if(optind != argc) { + fprintf(stderr, "ERROR: no file parameters permitted in " + "--write-keyfile mode\n"); + exit(1); + } + write_keyfile(newKeyFile); + } else { + if(optind == argc) + decrypt("-"); + else { + for(i = optind ; i < argc ; ++i) + decrypt(argv[i]); + } + } + + memset(cry_key, 0, cry_keylen); /* zero-out key store */ + cry_keylen = 0; + return 0; +} diff --git a/tools/rscryutil.rst b/tools/rscryutil.rst new file mode 100644 index 00000000..dfd447d2 --- /dev/null +++ b/tools/rscryutil.rst @@ -0,0 +1,199 @@ +========= +rscryutil +========= + +-------------------------- +Manage Encrypted Log Files +-------------------------- + +:Author: Rainer Gerhards <rgerhards@adiscon.com> +:Date: 2013-04-15 +:Manual section: 1 + +SYNOPSIS +======== + +:: + + rscryutil [OPTIONS] [FILE] ... + + +DESCRIPTION +=========== + +This tool performs various operations on encrypted log files. +Most importantly, it provides the ability to decrypt them. + + +OPTIONS +======= + +-d, --decrypt + Select decryption mode. This is the default mode. + +-W, --write-keyfile <file> + Utility function to write a key to a keyfile. The key can be obtained + via any method. + +-v, --verbose + Select verbose mode. + +-f, --force + Forces operations that otherwise would fail. + +-k, --keyfile <file> + Reads the key from <file>. File _must_ contain the key, only, no headers + or other meta information. Keyfiles can be generated via the + *--write-keyfile* option. + +-p, --key-program <path-to-program> + In this mode, the key is provided by a so-called "key program". This program + is executed and must return the key to (as well as some meta information) + via stdout. The core idea of key programs is that using this interface the + user can implement as complex (and secure) method to obtain keys as + desired, all without the need to make modifications to rsyslog. + +-K, --key <KEY> + TESTING AID, NOT FOR PRODUCTION USE. This uses the KEY specified + on the command line. This is the actual key, and as such this mode + is highly insecure. However, it can be useful for intial testing + steps. This option may be removed in the future. + +-a, --algo <algo> + Sets the encryption algorightm (cipher) to be used. See below + for supported algorithms. The default is "AES128". + +-m, --mode <mode> + Sets the ciphermode to be used. See below for supported modes. + The default is "CBC". + +-r, --generate-random-key <bytes> + Generates a random key of length <bytes>. This option is + meant to be used together with *--write-keyfile* (and it is hard + to envision any other valid use for it). + +OPERATION MODES +=============== + +The operation mode specifies what exactly the tool does with the provided +files. The default operation mode is "dump", but this may change in the future. +Thus, it is recommended to always set the operations mode explicitely. If +multiple operations mode are set on the command line, results are +unpredictable. + +decrypt +------- + +The provided log files are decrypted. Note that the *.encinfo* side files +must exist and be accessible in order for decryption to to work. + +write-keyfile +------------- + +In this mode no log files are processed; thus it is an error to specify +any on the command line. The specified keyfile is written. The key itself +is obtained via the usual key commands. If *--keyfile* is used, that +file is effectively copied. + +For security reasons, existing key files are _not_ overwritten. To permit +this, specify the *--force* option. When doing so, keep in mind that lost +keys cannot be recovered and data encrypted with them may also be considered +lost. + +Keyfiles are always created with 0400 permission, that is read access for only +the user. An exception is when an existing file is overwritten via the +*--force* option, in which case the former permissions still apply. + +EXIT CODES +========== + +The command returns an exit code of 0 if everything went fine, and some +other code in case of failures. + + +SUPPORTED ALGORITHMS +==================== + +We basically support what libgcrypt supports. This is: + + 3DES + CAST5 + BLOWFISH + AES128 + AES192 + AES256 + TWOFISH + TWOFISH128 + ARCFOUR + DES + SERPENT128 + SERPENT192 + SERPENT256 + RFC2268_40 + SEED + CAMELLIA128 + CAMELLIA192 + CAMELLIA256 + + +SUPPORTED CIPHER MODES +====================== + +We basically support what libgcrypt supports. This is: + + ECB + CFB + CBC + STREAM + OFB + CTR + AESWRAP + +EXAMPLES +======== + +**rscryutil logfile** + +Decrypts "logfile" and sends data to stdout. + + +**rscryutil --generate-random-key 16 --keyfile /some/secured/path/keyfile** + +Generates random key and stores it in the specified keyfile. + +LOG SIGNATURES +============== + +Encrypted log files can be used together with signing. To verify such a file, +it must be decrypted first, and the verification tool **rsgtutil(1)** must be +run on the decrypted file. + +SECURITY CONSIDERATIONS +======================= + +Specifying keys directly on the command line (*--key* option) is very +insecure and should +not be done, except for testing purposes with test keys. Even then it is +recommended to use keyfiles, which are also easy to handle during testing. +Keep in mind that command history is usally be kept by bash and can also +easily be monitored. + +Local keyfiles are also a security risk. At a minimum, they should be +used with very restrictive file permissions. For this reason, +the *rscryutil* tool creates them with read permissions for the user, +only, no matter what umask is set to. + +When selecting cipher algorithms and modes, care needs to be taken. The +defaults should be reasonable safe to use, but this tends to change over +time. Keep up with the most current crypto recommendations. + + +SEE ALSO +======== +**rsgtutil(1)**, **rsyslogd(8)** + +COPYRIGHT +========= + +This page is part of the *rsyslog* project, and is available under +LGPLv2. diff --git a/tools/rsgtutil.c b/tools/rsgtutil.c new file mode 100644 index 00000000..095b8066 --- /dev/null +++ b/tools/rsgtutil.c @@ -0,0 +1,431 @@ +/* This is a tool for dumpoing the content of GuardTime TLV + * files in a (somewhat) human-readable manner. + * + * Copyright 2013 Adiscon GmbH + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either exprs or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <unistd.h> +#include <gt_base.h> +#include <gt_http.h> +#include <getopt.h> + +#include "librsgt.h" + +typedef unsigned char uchar; + +static enum { MD_DUMP, MD_DETECT_FILE_TYPE, MD_SHOW_SIGBLK_PARAMS, + MD_VERIFY, MD_EXTEND +} mode = MD_DUMP; +static int verbose = 0; + +static void +dumpFile(char *name) +{ + FILE *fp; + uchar hdr[9]; + void *obj; + tlvrecord_t rec; + int r = -1; + + if(!strcmp(name, "-")) + fp = stdin; + else { + printf("Processing file %s:\n", name); + if((fp = fopen(name, "r")) == NULL) { + perror(name); + goto err; + } + } + if((r = rsgt_tlvrdHeader(fp, hdr)) != 0) goto err; + printf("File Header: '%s'\n", hdr); + while(1) { /* we will err out on EOF */ + if((r = rsgt_tlvrd(fp, &rec, &obj)) != 0) { + if(feof(fp)) + break; + else + goto err; + } + rsgt_tlvprint(stdout, rec.tlvtype, obj, verbose); + rsgt_objfree(rec.tlvtype, obj); + } + + if(fp != stdin) + fclose(fp); + return; +err: fprintf(stderr, "error %d processing file %s\n", r, name); +} + +static void +showSigblkParams(char *name) +{ + FILE *fp; + block_sig_t *bs; + uint8_t bHasRecHashes, bHasIntermedHashes; + uint64_t blkCnt = 0; + int r = -1; + + if(!strcmp(name, "-")) + fp = stdin; + else { + if((fp = fopen(name, "r")) == NULL) { + perror(name); + goto err; + } + } + if((r = rsgt_chkFileHdr(fp, "LOGSIG10")) != 0) goto err; + + while(1) { /* we will err out on EOF */ + if((r = rsgt_getBlockParams(fp, 0, &bs, &bHasRecHashes, + &bHasIntermedHashes)) != 0) + goto err; + ++blkCnt; + rsgt_printBLOCK_SIG(stdout, bs, verbose); + printf("\t***META INFORMATION:\n"); + printf("\tBlock Nbr in File...: %llu\n", blkCnt); + printf("\tHas Record Hashes...: %d\n", bHasRecHashes); + printf("\tHas Tree Hashes.....: %d\n", bHasIntermedHashes); + } + + if(fp != stdin) + fclose(fp); + return; +err: + if(r != RSGTE_EOF) + fprintf(stderr, "error %d processing file %s\n", r, name); +} + +static void +detectFileType(char *name) +{ + FILE *fp; + char *typeName; + char hdr[9]; + int r = -1; + + if(!strcmp(name, "-")) + fp = stdin; + else { + if((fp = fopen(name, "r")) == NULL) { + perror(name); + goto err; + } + } + if((r = rsgt_tlvrdHeader(fp, (uchar*)hdr)) != 0) goto err; + if(!strcmp(hdr, "LOGSIG10")) + typeName = "Log Signature File, Version 10"; + else if(!strcmp(hdr, "GTSTAT10")) + typeName = "rsyslog GuardTime Signature State File, Version 10"; + else + typeName = "unknown"; + + printf("%s: %s [%s]\n", name, hdr, typeName); + + if(fp != stdin) + fclose(fp); + return; +err: fprintf(stderr, "error %d processing file %s\n", r, name); +} + +static inline int +doVerifyRec(FILE *logfp, FILE *sigfp, FILE *nsigfp, + block_sig_t *bs, gtfile gf, gterrctx_t *ectx, uint8_t bInBlock) +{ + int r; + size_t lenRec; + char line[128*1024]; + + if(fgets(line, sizeof(line), logfp) == NULL) { + if(feof(logfp)) { + r = RSGTE_EOF; + } else { + perror("log file input"); + r = RSGTE_IO; + } + goto done; + } + lenRec = strlen(line); + if(line[lenRec-1] == '\n') { + line[lenRec-1] = '\0'; + --lenRec; + rsgt_errctxSetErrRec(ectx, line); + } + + /* we need to preserve the first line (record) of each block for + * error-reporting purposes (bInBlock==0 meanst start of block) + */ + if(bInBlock == 0) + rsgt_errctxFrstRecInBlk(ectx, line); + + r = rsgt_vrfy_nextRec(bs, gf, sigfp, nsigfp, (unsigned char*)line, lenRec, ectx); +done: + return r; +} + +/* We handle both verify and extend with the same function as they + * are very similiar. + * + * note: here we need to have the LOG file name, not signature! + */ +static void +verify(char *name) +{ + FILE *logfp = NULL, *sigfp = NULL, *nsigfp = NULL; + block_sig_t *bs = NULL; + gtfile gf; + uint8_t bHasRecHashes, bHasIntermedHashes; + uint8_t bInBlock; + int r = 0; + char sigfname[4096]; + char oldsigfname[4096]; + char nsigfname[4096]; + gterrctx_t ectx; + + if(!strcmp(name, "-")) { + fprintf(stderr, "%s mode cannot work on stdin\n", + mode == MD_VERIFY ? "verify" : "extend"); + goto err; + } else { + snprintf(sigfname, sizeof(sigfname), "%s.gtsig", name); + sigfname[sizeof(sigfname)-1] = '\0'; + if((logfp = fopen(name, "r")) == NULL) { + perror(name); + goto err; + } + if((sigfp = fopen(sigfname, "r")) == NULL) { + perror(sigfname); + goto err; + } + if(mode == MD_EXTEND) { + snprintf(nsigfname, sizeof(nsigfname), "%s.gtsig.new", name); + nsigfname[sizeof(nsigfname)-1] = '\0'; + if((nsigfp = fopen(nsigfname, "w")) == NULL) { + perror(nsigfname); + goto err; + } + snprintf(oldsigfname, sizeof(oldsigfname), + "%s.gtsig.old", name); + oldsigfname[sizeof(oldsigfname)-1] = '\0'; + } + } + + rsgtInit("rsyslog rsgtutil " VERSION); + rsgt_errctxInit(&ectx); + ectx.verbose = verbose; + ectx.fp = stderr; + ectx.filename = strdup(sigfname); + + if((r = rsgt_chkFileHdr(sigfp, "LOGSIG10")) != 0) goto done; + if(mode == MD_EXTEND) { + if(fwrite("LOGSIG10", 8, 1, nsigfp) != 1) { + perror(nsigfname); + r = RSGTE_IO; + goto done; + } + } + gf = rsgt_vrfyConstruct_gf(); + if(gf == NULL) { + fprintf(stderr, "error initializing signature file structure\n"); + goto done; + } + + bInBlock = 0; + ectx.blkNum = 0; + ectx.recNumInFile = 0; + + while(!feof(logfp)) { + if(bInBlock == 0) { + if(bs != NULL) + rsgt_objfree(0x0902, bs); + if((r = rsgt_getBlockParams(sigfp, 1, &bs, &bHasRecHashes, + &bHasIntermedHashes)) != 0) + goto done; + rsgt_vrfyBlkInit(gf, bs, bHasRecHashes, bHasIntermedHashes); + ectx.recNum = 0; + ++ectx.blkNum; + } + ++ectx.recNum, ++ectx.recNumInFile; + if((r = doVerifyRec(logfp, sigfp, nsigfp, bs, gf, &ectx, bInBlock)) != 0) + goto done; + if(ectx.recNum == bs->recCount) { + if((r = verifyBLOCK_SIG(bs, gf, sigfp, nsigfp, + (mode == MD_EXTEND) ? 1 : 0, &ectx)) != 0) + goto done; + bInBlock = 0; + } else bInBlock = 1; + } + +done: + if(r != RSGTE_EOF) + goto err; + + fclose(logfp); logfp = NULL; + fclose(sigfp); sigfp = NULL; + if(nsigfp != NULL) { + fclose(nsigfp); nsigfp = NULL; + } + + /* everything went fine, so we rename files if we updated them */ + if(mode == MD_EXTEND) { + if(unlink(oldsigfname) != 0) { + if(errno != ENOENT) { + perror("unlink oldsig"); + r = RSGTE_IO; + goto err; + } + } + if(link(sigfname, oldsigfname) != 0) { + perror("link oldsig"); + r = RSGTE_IO; + goto err; + } + if(unlink(sigfname) != 0) { + perror("unlink cursig"); + r = RSGTE_IO; + goto err; + } + if(link(nsigfname, sigfname) != 0) { + perror("link newsig"); + fprintf(stderr, "WARNING: current sig file has been " + "renamed to %s - you need to manually recover " + "it.\n", oldsigfname); + r = RSGTE_IO; + goto err; + } + if(unlink(nsigfname) != 0) { + perror("unlink newsig"); + fprintf(stderr, "WARNING: current sig file has been " + "renamed to %s - you need to manually recover " + "it.\n", oldsigfname); + r = RSGTE_IO; + goto err; + } + } + rsgtExit(); + rsgt_errctxExit(&ectx); + return; + +err: + fprintf(stderr, "error %d processing file %s\n", r, name); + if(logfp != NULL) + fclose(logfp); + if(sigfp != NULL) + fclose(sigfp); + if(nsigfp != NULL) { + fclose(nsigfp); + unlink(nsigfname); + } + rsgtExit(); + rsgt_errctxExit(&ectx); +} + +static void +processFile(char *name) +{ + switch(mode) { + case MD_DETECT_FILE_TYPE: + detectFileType(name); + break; + case MD_DUMP: + dumpFile(name); + break; + case MD_SHOW_SIGBLK_PARAMS: + showSigblkParams(name); + break; + case MD_VERIFY: + case MD_EXTEND: + verify(name); + break; + } +} + + +static struct option long_options[] = +{ + {"dump", no_argument, NULL, 'D'}, + {"verbose", no_argument, NULL, 'v'}, + {"version", no_argument, NULL, 'V'}, + {"detect-file-type", no_argument, NULL, 'T'}, + {"show-sigblock-params", no_argument, NULL, 'B'}, + {"verify", no_argument, NULL, 't'}, /* 't' as in "test signatures" */ + {"extend", no_argument, NULL, 'e'}, + {"publications-server", optional_argument, NULL, 'P'}, + {"show-verified", no_argument, NULL, 's'}, + {NULL, 0, NULL, 0} +}; + +int +main(int argc, char *argv[]) +{ + int i; + int opt; + + while(1) { + opt = getopt_long(argc, argv, "DvVTBtPs", long_options, NULL); + if(opt == -1) + break; + switch(opt) { + case 'v': + verbose = 1; + break; + case 's': + rsgt_read_showVerified = 1; + break; + case 'V': + fprintf(stderr, "rsgtutil " VERSION "\n"); + exit(0); + case 'D': + mode = MD_DUMP; + break; + case 'B': + mode = MD_SHOW_SIGBLK_PARAMS; + break; + case 'P': + rsgt_read_puburl = optarg; + break; + case 'T': + mode = MD_DETECT_FILE_TYPE; + break; + case 't': + mode = MD_VERIFY; + break; + case 'e': + mode = MD_EXTEND; + break; + case '?': + break; + default:fprintf(stderr, "getopt_long() returns unknown value %d\n", opt); + return 1; + } + } + + if(optind == argc) + processFile("-"); + else { + for(i = optind ; i < argc ; ++i) + processFile(argv[i]); + } + + return 0; +} diff --git a/tools/rsgtutil.rst b/tools/rsgtutil.rst new file mode 100644 index 00000000..37958450 --- /dev/null +++ b/tools/rsgtutil.rst @@ -0,0 +1,177 @@ +======== +rsgtutil +======== + +----------------------------------- +Manage (GuardTime) Signed Log Files +----------------------------------- + +:Author: Rainer Gerhards <rgerhards@adiscon.com> +:Date: 2013-03-25 +:Manual section: 1 + +SYNOPSIS +======== + +:: + + rsgtutil [OPTIONS] [FILE] ... + + +DESCRIPTION +=========== + +This tool performs various maintenance operations on signed log files. +It specifically supports the GuardTime signature provider. + +The *rsgtutil* tool is the primary tool to verify log file signatures, +dump signature file contents and carry out other maintenance operations. +The tool offers different operation modes, which are selected via +command line options. + +The processing of multiple files is permitted. Depending on operation +mode, either the signature file or the base log file must be specified. +Within a single call, only a single operations mode is permitted. To +use different modes on different files, multiple calles, one for each +mode, must be made. + +If no file is specified on the command line, stdin is used instead. Note +that not all operation modes support stdin. + +OPTIONS +======= + +-D, --dump + Select "dump" operations mode. + +-t, --verify + Select "verify" operations mode. + +-T, --detect-file-type + Select "detect-file-type" operations mode. + +-B, --show-sigblock-params + Select "show-sigblock-params" operations mode. + +-s, --show-verified + Prints out information about correctly verified blocks (by default, only + errors are printed). + +-v, --verbose + Select verbose mode. Most importantly, hashes and signatures are printed + in full length (can be **very** lengthy) rather than the usual abbreviation. + +-e, --extend + Select extend mode. This extends the RFC3161 signatures. Note that this + mode also implies a full verification. If there are verify errors, extending + will also fail. + +-P <URL>, --publications-server <URL> + Sets the publications server. If not set but required by the operation a + default server is used. The default server is not necessarily optimal + in regard to performance and reliability. + + +OPERATION MODES +=============== + +The operation mode specifies what exactly the tool does with the provided +files. The default operation mode is "dump", but this may change in the future. +Thus, it is recommended to always set the operations mode explicitely. If +multiple operations mode are set on the command line, results are +unpredictable. + +dump +---- + +The provided *signature* files are dumped. For each top-level record, the*u +type code is printed as well as q short description. If there is additional +information available, it will be printed in tab-indented lines below the +main record dump. The actual *log* files need not to be present. + +verify +------ + +This mode does not work with stdin. On the command line, the *log* file names +are specified. The corresponding *signature* files (ending on ".gtsig") must also +be preset at the same location as the log file. In verify mode, both the log +and signature file is read and the validity of the log file checked. If verification +errors are detected these are printed and processing of the file aborted. By default, +each file is verified individually, without taking cross-file hash chains into +account (so the order of files on the command line does not matter). + +Note that the actual amount of what can be verified depends on the parameters with +which the signature file was written. If record and tree hashes are present, they +will be verified and thus fine-granular error reporting is possible. If they are +not present, only the block signature itself is verified. + +By default, only errors are printed. To also print successful verifications, use the +**--show-verified** option. + + +extend +------ +This extends the RFC3161 signatures. This includes a full verification +of the file. If there are verification errors, extending will also fail. +Note that a signature can only be extended when the required hash has been +published. Currently, these hashes are created at the 15th of each month at +0:00hrs UTC. It takes another few days to get them finally published. As such, +it can be assumed that extending is only possible after this happend (which +means it may take slightly above a month). + +To prevent data corruption, a copy of the signature file is created during +extension. So there must be enough disk space available for both files, +otherwise the operation will fail. If the log file is named logfile, the +signature file is logfile.gtsig and the temporary work file is named +logfile.gtsig.new. When extending finished successfully, the original +signature file (logfile.gtsig in our example) is renamed with the .old +postfix (logfile.gtsig.old) and the temporary file written under the +original name. The .old file can be deleted. It is just kept as a +precaution to prevent signature loss. Note that any already existing +.old or .new files are overwritten by these operations. + + +detect-file-type +---------------- +This mode is used to detect the type of some well-know files used inside the +signature system. The detection is based on the file header. This mode is +primarily a debug aid. + + +show-sigblock-params +-------------------- +This mode is used to print signature block parameters. It is similar to *dump* +mode, but will ignore everything except signature blocks. Also, some additional +meta information is printed. This mode is primarily a debug aid. + +EXIT CODES +========== + +The command returns an exit code of 0 if everything went fine, and some +other code in case of failures. + + +EXAMPLES +======== + +**rsgtutil --verify logfile** + +This verifies the file "logfile" via its associated signature file +"logfile.gtsig". If errors are detected, these are reported to stderr. +Otherwise, rsgtutil terminates without messages. + +**rsgtutil --dump logfile.gtsig** + +This dumps the content of the signature file "logfile.gtsig". The +actual log file is not being processed and does not even need to be +present. + +SEE ALSO +======== +**rsyslogd(8)** + +COPYRIGHT +========= + +This page is part of the *rsyslog* project, and is available under +LGPLv2. diff --git a/tools/rsyslog.conf.5 b/tools/rsyslog.conf.5 index fe9e083b..07da6ffd 100644 --- a/tools/rsyslog.conf.5 +++ b/tools/rsyslog.conf.5 @@ -218,7 +218,7 @@ beginning with a slash ('/'). .B Example: .RS -*.* /var/log/traditionalfile.log;RSYSLOG_TraditionalFormat # log to a file in the traditional format +*.* /var/log/traditionalfile.log;RSYSLOG_TraditionalFileFormat # log to a file in the traditional format .RE Note: if you would like to use high-precision timestamps in your log files, diff --git a/tools/syslogd.c b/tools/syslogd.c index c2928a8f..77adb2cb 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -218,7 +218,7 @@ static ratelimit_t *dflt_ratelimiter = NULL; /* ratelimiter for submits without static ratelimit_t *internalMsg_ratelimiter = NULL; /* ratelimiter for rsyslog-own messages */ int MarkInterval = 20 * 60; /* interval between marks in seconds - read-only after startup */ int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ -static int NoFork = 0; /* don't fork - don't run in daemon mode - read-only after startup */ +static int doFork = 1; /* fork - run in daemon mode - read-only after startup */ int bHaveMainQueue = 0;/* set to 1 if the main queue - in queueing mode - is available * If the main queue is either not yet ready or not running in * queueing mode (mode DIRECT!), then this is set to 0. @@ -473,7 +473,7 @@ logmsgInternal(int iErr, int pri, uchar *msg, int flags) * permits us to process unmodified config files which otherwise contain a * supressor statement. */ - if(((Debug == DEBUG_FULL || NoFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { + if(((Debug == DEBUG_FULL || !doFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { if(LOG_PRI(pri) == LOG_ERR) fprintf(stderr, "rsyslogd: %s\n", msg); } @@ -567,6 +567,7 @@ msgConsumer(void __attribute__((unused)) *notNeeded, batch_t *pBatch, int *pbShu assert(pBatch != NULL); pBatch->pbShutdownImmediate = pbShutdownImmediate; /* TODO: move this to batch creation! */ preprocessBatch(pBatch); +dbgprintf("DDDD: batches ShutdownImmediate is %p\n", pBatch->pbShutdownImmediate); ruleset.ProcessBatch(pBatch); //TODO: the BATCH_STATE_COMM must be set somewhere down the road, but we //do not have this yet and so we emulate -- 2010-06-10 @@ -1528,6 +1529,13 @@ queryLocalHostname(void) */ glbl.SetLocalHostName(LocalHostName); glbl.SetLocalDomain(LocalDomain); + + if ( strlen((char*)LocalDomain) ) { + CHKmalloc(LocalFQDNName = (uchar*)malloc(strlen((char*)LocalDomain)+strlen((char*)LocalHostName)+1)); + if ( sprintf((char*)LocalFQDNName,"%s.%s",(char*)LocalHostName,(char*)LocalDomain) ) + glbl.SetLocalFQDNName(LocalFQDNName); + } + glbl.GenerateLocalHostNameProperty(); /* must be redone after conf processing, FQDN setting may have changed */ finalize_it: RETiRet; @@ -1614,8 +1622,7 @@ doGlblProcessInit(void) thrdInit(); - if( !(Debug == DEBUG_FULL || NoFork) ) - { + if(doFork) { DBGPRINTF("Checking pidfile '%s'.\n", PidFile); if (!check_pid(PidFile)) { @@ -1627,16 +1634,23 @@ doGlblProcessInit(void) /* stop writing debug messages to stdout (if debugging is on) */ stddbg = -1; + dbgprintf("ready for forking\n"); if (fork()) { /* Parent process */ - sleep(300); - /* Not reached unless something major went wrong. 5 - * minutes should be a fair amount of time to wait. - * Please note that this procedure is important since - * the father must not exit before syslogd isn't - * initialized or the klogd won't be able to flush its - * logs. -Joey + dbgprintf("parent process going to sleep for 60 secs\n"); + sleep(60); + /* Not reached unless something major went wrong. 1 + * minute should be a fair amount of time to wait. + * The parent should not exit before rsyslogd is + * properly initilized (at least almost) or the init + * system may get a wrong impression of our readyness. + * Note that we exit before being completely initialized, + * but at this point it is very, very unlikely that something + * bad can happen. We do this here, because otherwise we would + * need to have much more code to handle priv drop (which we + * don't consider worth for the init system, especially as it + * is going away on the majority of distros). */ exit(1); /* "good" exit - after forking, not diasabling anything */ } @@ -1645,6 +1659,7 @@ doGlblProcessInit(void) close(0); /* we keep stdout and stderr open in case we have to emit something */ i = 3; + dbgprintf("in child, finalizing initialization\n"); /* if (sd_booted()) */ { const char *e; @@ -1678,7 +1693,8 @@ doGlblProcessInit(void) i = SD_LISTEN_FDS_START + sd_fds; } for ( ; i < num_fds; i++) - (void) close(i); + if(i != dbgGetDbglogFd()) + close(i); untty(); } else { @@ -1889,7 +1905,7 @@ int realMain(int argc, char **argv) fprintf(stderr, "rsyslogd: error -m is no longer supported - use immark instead"); break; case 'n': /* don't fork */ - NoFork = 1; + doFork = 0; break; case 'N': /* enable config verify mode */ iConfigVerify = atoi(arg); @@ -1993,17 +2009,16 @@ int realMain(int argc, char **argv) if(!iConfigVerify) CHKiRet(doGlblProcessInit()); + /* Send a signal to the parent so it can terminate. */ + if(glblGetOurPid() != ppid) + kill(ppid, SIGTERM); + CHKiRet(init()); if(Debug && debugging_on) { dbgprintf("Debugging enabled, SIGUSR1 to turn off debugging.\n"); } - /* Send a signal to the parent so it can terminate. */ - if(glblGetOurPid() != ppid) - kill(ppid, SIGTERM); - - /* END OF INTIALIZATION */ DBGPRINTF("initialization completed, transitioning to regular run mode\n"); @@ -2014,7 +2029,7 @@ int realMain(int argc, char **argv) * is still in its infancy (and not really done), we currently accept this issue. * rgerhards, 2009-06-29 */ - if(!(Debug == DEBUG_FULL || NoFork)) { + if(doFork) { close(1); close(2); ourConf->globals.bErrMsgToStderr = 0; |