summaryrefslogtreecommitdiffstats
path: root/runtime
Commit message (Collapse)AuthorAgeFilesLines
...
| * | | | | | | | | | | | | | | | | | | | solved design issue with queue terminationRainer Gerhards2009-05-267-105/+223
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... and also improved the test suite. There is a design issue in the v3 queue engine that manifested to some serious problems with the new processing mode. However, in v3 shutdown may take eternally if a queue runs in DA mode, is configured to preserve data AND the action fails and retries immediately. There is no cure available for v3, it would require doing much of the work we have done on the new engine. The window of exposure, as one might guess from the description, is very small. That is probably the reason why we have not seen it in practice.
| * | | | | | | | | | | | | | | | | | | | Merge branch 'master' into ultra-reliableRainer Gerhards2009-05-251-0/+29
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
| * \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Merge branch 'master' into ultra-reliableRainer Gerhards2009-05-253-2/+2
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: tests/Makefile.am tests/diskqueue.sh
| * | | | | | | | | | | | | | | | | | | | | | minor cleanup: remove compiler warningRainer Gerhards2009-05-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | Merge branch 'master' into ultra-reliableRainer Gerhards2009-05-2210-78/+115
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: runtime/rsyslog.h tests/Makefile.am tools/syslogd.c
| * | | | | | | | | | | | | | | | | | | | | | | free last processed message in all casesRainer Gerhards2009-05-206-68/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | so far, the last processed message was only freed when the next one was processed. This has been changed now. More precisely, a better algorithm has been selected for the queue worker process, which also involves less overhead than the previous one. The fix for "free last processed message" as then more or less a side-effect (easy to do) of the new algorithm.
| * | | | | | | | | | | | | | | | | | | | | | | yield() no longer needed on uniproc thanks to new algorithmsRainer Gerhards2009-05-203-42/+8
| | | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | | solved the intended-discard-during-dequeue issueRainer Gerhards2009-05-193-11/+19
| | | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | | some cleanupRainer Gerhards2009-05-192-87/+0
| | | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | | queue size calculation now based on logical/physical dequeueRainer Gerhards2009-05-192-49/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... needed to split the old single counter into two. I wouldn't bet that I made some mistakes while doing so, but at least some ad-hoc tests plus the testbench do no longer indicate errors.
| * | | | | | | | | | | | | | | | | | | | | | | Merge branch 'master' into tmpRainer Gerhards2009-05-191-0/+1
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: tests/Makefile.am
| * | | | | | | | | | | | | | | | | | | | | | | | removed queue's UngetObj() callRainer Gerhards2009-05-182-105/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... which is no longer needed thanks to the new queue design.
| * | | | | | | | | | | | | | | | | | | | | | | | t-delete list implemented, queue store drivers updated...Rainer Gerhards2009-05-185-57/+337
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... on the way to the ultra-reliable queue modes (redesign doc). This version does not really work, but is a good commit point. Next comes queue size calculation. DA mode does not yet work.
| * | | | | | | | | | | | | | | | | | | | | | | | moved user object destruction to queue itselfRainer Gerhards2009-05-132-4/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | So far, the consumer was responsible for destroying objects. However, this does not work well with ultra-reliable queues. This is the first move to support them.
| * | | | | | | | | | | | | | | | | | | | | | | | added test for property replacer field functionality to testbenchRainer Gerhards2009-05-131-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | plus some cosmetic changes
| * | | | | | | | | | | | | | | | | | | | | | | | action batch processing implementedRainer Gerhards2009-05-122-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... passed initial tests, but of course more are needed
| * | | | | | | | | | | | | | | | | | | | | | | | one astrisk too much - and we have a segfault... - fixed ;)Rainer Gerhards2009-05-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | | | moving to a cleaner implementation of batchesRainer Gerhards2009-05-128-44/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... now that we know what we need from a theoretical POV.
| * | | | | | | | | | | | | | | | | | | | | | | | Merge branch 'master' into multi-dequeueRainer Gerhards2009-05-121-0/+1
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: runtime/rsyslog.h
| * | | | | | | | | | | | | | | | | | | | | | | | | first shot at action state machine implemention (untested)Rainer Gerhards2009-05-072-3/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I am commiting it so that the code is visible, but will no begin with the test environment.
| * | | | | | | | | | | | | | | | | | | | | | | | | begin implementation of new transactional output module interfaceRainer Gerhards2009-04-274-2/+112
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | code is not complete, error cases are not handled.
| * | | | | | | | | | | | | | | | | | | | | | | | | fixed abort condition in DA modeRainer Gerhards2009-04-233-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | | | | fixing a small (newly-introduced) memory leakRainer Gerhards2009-04-233-17/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... plus simplifying free() calls after agreement on mailing list that we no longer need to check if the pointer is non-NULL
| * | | | | | | | | | | | | | | | | | | | | | | | | added $MainMsgQueueDequeueBatchSize and $ActionQueueDequeueBatchSize ↵Rainer Gerhards2009-04-235-20/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | configuration directives
| * | | | | | | | | | | | | | | | | | | | | | | | | Merge branch 'master' into multi-dequeueRainer Gerhards2009-04-231-4/+4
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | | | | | | | | | | | | | | | improving debugging info a bitRainer Gerhards2009-04-231-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | | | | | | | | | | now batches are handed down to the actual consumerRainer Gerhards2009-04-222-18/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... but the action consumer does not do anything really intelligent with them. But the DA consumer is already done, as is the main message queue consumer.
| * | | | | | | | | | | | | | | | | | | | | | | | | | first attempt at dequeueing multiple batches inside the queueRainer Gerhards2009-04-227-79/+149
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... but this code has serious problems when terminating the queue, also it is far from being optimal. I will commit a series of patches (hopefully) as I am on the path to the final implementation.
| * | | | | | | | | | | | | | | | | | | | | | | | | | Merge branch 'master' into multi-dequeueRainer Gerhards2009-04-221-0/+2
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | | | | | | | | | | | | | | | | some cleanupRainer Gerhards2009-04-171-39/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | | | | | bugfix: missing initialization during timestamp creationv4.5.0Rainer Gerhards2009-07-032-0/+5
| |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This could lead to timestamps written in the wrong format, but not to an abort.
* | | | | | | | | | | | | | | | | | | | | | | | | | | now put the new property-based methods to good useRainer Gerhards2009-07-014-40/+100
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... hopefully reducing the number of allocs/frees as well as overall memory usage in a busy system (plus that these shared properties hopefully remain in cache longer than its single-instance counterparts...)
* | | | | | | | | | | | | | | | | | | | | | | | | | | RcvFromIP now also a propertyRainer Gerhards2009-07-012-34/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This sets stage to enable use of the property-interface to speed up things (mildly), the next step to be done. I have also fixed one regression of yesterday's changes.
* | | | | | | | | | | | | | | | | | | | | | | | | | | removed a debugging-left-over, caused compilation failure on some platformsRainer Gerhards2009-07-011-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... because LARGFILE macros were not defined consistenly
* | | | | | | | | | | | | | | | | | | | | | | | | | | introduced a new way of handling the RcvFrom propertyRainer Gerhards2009-06-309-50/+137
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... plus a fix for a long-time bug in obj-types.h. That lead to the object pointer only then to become NULL when the object was actually destructed, I discovered this issue during introduction of the pRcvFrom property in msg_t, but it potentially had other effects, too. I am not sure if some experienced instability resulted from this bug OR if its fix will cause harm to so-far "correctly" running code. The later may very well be. Thus I will change it only for the current branch and also the beta, but not in all old builds. Let's see how things evolve.
* | | | | | | | | | | | | | | | | | | | | | | | | | | finished transition to using inputName propertyRainer Gerhards2009-06-303-106/+180
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... plus some celanup and adding minor missing functionality (the rule debug info again tell the property name, not just number).
* | | | | | | | | | | | | | | | | | | | | | | | | | | introduced the idea of detached propertiesRainer Gerhards2009-06-299-37/+152
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | some things inside the message can be used over a large number of messages and need to to be allocated and re-written every time. I now begin to implement this as a "prop_t" object, first use for the inputName. Some input modules are already converted, some others to go. Will do a little performance check on the new method before I go further. Also, this commit has some cleanup and a few bug fixes that prevented compiliation in debug mode (I overlooked this as I did not compile for debug, what I normally do, and the automatted test also does not do that)
* | | | | | | | | | | | | | | | | | | | | | | | | | | separate willRun and runInput calls for input modulesMichael Terry2009-06-292-0/+2
| |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Rainer Gerhards <rgerhards@adiscon.com>
* | | | | | | | | | | | | | | | | | | | | | | | | | optimization: propert names are now internally identified by integersRainer Gerhards2009-06-267-137/+296
| | | | | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | | | some optimization, declared some frequently called small stringbuf functions ↵Rainer Gerhards2009-06-262-60/+62
| |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | inline
* | | | | | | | | | | | | | | | | | | | | | | | | got rid of the recursive requirement for msg_t mutex (finally!)Rainer Gerhards2009-06-265-168/+121
| | | | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | | some more stringbuffer optimizationRainer Gerhards2009-06-253-68/+30
| | | | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | | bugfix: msg_t mutex was sometimes initialized twiceRainer Gerhards2009-06-251-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | | backported "clean" increment of memory trim counter (not protected by mutex)Rainer Gerhards2009-06-252-3/+6
| |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | named queue worker threadsRainer Gerhards2009-06-251-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... but I don't see the name anywhere...?
* | | | | | | | | | | | | | | | | | | | | | | | cleanup (removed now-unused parameters)Rainer Gerhards2009-06-253-17/+17
| | | | | | | | | | | | | | | | | | | | | | | |
* | | | | | | | | | | | | | | | | | | | | | | | bugfix: subtle potential issue during queue shutdownRainer Gerhards2009-06-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... this one could cause trouble, but I really don't think it caused any actual harm.
* | | | | | | | | | | | | | | | | | | | | | | | improvements/fixes in queue termination timeout handlingRainer Gerhards2009-06-252-3/+7
| |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - bugfix: subtle (and usually irrelevant) issue in timout processing timeout could be one second too early if nanoseconds wrapped - set a more sensible timeout for shutdow, now 1.5 seconds to complete processing (this also removes those cases where the shutdown message was not written because the termination happened before it)
* | | | | | | | | | | | | | | | | | | | | | | some memory accesses are now explicitely atomicRainer Gerhards2009-06-253-10/+24
| |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... as far as I think this mostly is to keep the thread debuggers happy
* | | | | | | | | | | | | | | | | | | | | | Merge branch 'beta'Rainer Gerhards2009-06-241-0/+2
|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | | | | | | | | | | |