Friday, February 29, 2008

rsyslog work log

Yesterday's rsyslog work log:
- updated "mysql paper" to include information for other databases, too
- worked a bit on the rsyslog/syslog-ng comparsion - slowly gets in better
shape ;)
- wrote doc on how to use the expression engine
- changed ABNF to fully support old property names
- added case-insensitive comparison operations
- MILESTONE: initial expression support completed
- (finally;)) released 3.12.0
- implemented environment-settable debug options
- enabled debug-support to pull runtime options from environment (bug 18)
- bugfix: removed debugging code that I forgot to remove before releasing
3.12.0 (does not cause harm and happened only during startup)
- added "help" command to runtime debug flags
- switched to git as a test run; currently I maintain both the cvs
and git repositories (via manual file copies). If all goes well, will switch
to git-only when I feel good about it (a week or so?)

Thursday, February 28, 2008

rsyslog work log

Yesterday's rsyslog work log:
- bugfix: queue cancel cleanup handler could be called with
invalid pointer if dequeue failed
- bugfix: imfile could abort under extreme stress conditions
(when it was terminated before it could open all of its
to be monitored files)
- bugfix: queue disk file were not properly persisted when
immediately after closing an output file rsyslog was stopped
or huped (the new output file open must NOT have happend at
that point) - this lead to a sparse and invalid queue file
which could cause several problems to the engine (unpredictable
results). This situation should have happened only in very
rare cases. tracker:
- bugfix: during queue shutdown, an assert invalidly triggered when
the primary queue's DA worker was terminated while the DA queue's
regular worker was still executing. This could result in a segfault
during shutdown.
- bugfix: object property deserializer did not handle negative numbers
- bugfix: queue aborted when it was shut down, DA-enabled, DA mode
was just initiated but not fully initialized (a race condition)
- bugfix: queue properties sizeOnDisk, bytesRead were persisted to
disk with wrong data type (long instead of int64) - could cause
problems on 32 bit machines
- fixed a problem introduced today, on-disk queue size was now
wrongly calculated (but not in any released version)

Wednesday, February 27, 2008

rsyslog work log

Yesterday's rsyslog work log:
- implemented data type conversion
- released 3.11.5
- implemented majority of comparison operations
- added some temporary testing aids to conf.c, so that we can debug
expression support as it is implemented
- fixed a couple of bugs in expression system
- added more operations to virtual machine - now works well with
- added PUSHMSGVAR operation to vm
- included expression support in filter module (and it works ;))
- added sysvar class
- added PUSHSYSVAR operation to vm
- added string concatenation operator & to RainerScript
- fixed segfault when pure string values were tried to be added
- some stage work for library modules
- fixed wrong pointer in imgssapi config coding
- added $InputGSSServerMaxSessions config directive
- worked on queue stability

A Web Site for RainerScript ;)

Just to make sure nobody will grab the name RainerScript, I have just created a web site for RainerScript. Of course, it does not contain any real useful information, but at least it shows I am there. As far as I know, this can be helpful in some cases ;)

Tuesday, February 26, 2008

Why RainerScript?

After my initial announcement of RainerScript, I got some good feedback. I'd like to reproduce an especially useful discussion here:

> I wondered if using a complete scripting language for configuraton
> is not making it overly complex and error prone. I specifically means
> function definition etc. here.

The key is that I will model it so that the complexity is only seen when
it is actually needed. Most importantly, it will continue to understand
plain old syslog.conf format. So most peoply will probably never see the
scripting features. HOWEVER, I think it is useful to have these for a
few select folks who really need them. Function definitions, for
example, will counter syslog-ng's feature to define a filter. I don't
know if it is actually worth and useful to define a filter to reuse it.
But with the function, you can do it. So those coming from syslog-ng can
find their favorite feature ;)

I think it is probably worth to think about how I came to the idea of
scripting support. The root cause is complex logic, complex expressions.
A few days ago I started to implement them ... and quickly found out
that actually the easiest way is "to do it right". So I now have a
parse, bytecode and an interpreter/virtual machine running in rsyslog.
All of this "just" to support complex expressions. While I thought about
it, I finally realized that it takes just a little bit to extend these
things so that the end result will be an easy to use scripting facility.
That also solves the config file issues that's dangeling so long and, as
a side-effect, brings up some interesting options (for those that have
need for it).

For example, I thought about how to best handle rewriting message
content. Now, the solution is obvious. I'll implement a "set" facility
in the language that enables message variables to be rewritten on the
fly (e.g. for folks who would like to drop some parts of the message). I
agree, all not pretty mainstream, but useful in some cases. And keep in
mind that it comes at a (complexity) cost only if you need it. That's
based on the fact that expression support was originally planned as a
by-line of the current config format, so it needed to work with the rest
in any way ;)

> But if you really need that kind of flexibility and scripting
> capabilites, have you considered to use existing languages, like lua
> , which were specifically designed to be embedded into
> applications?

Not really, because of the way it has evolved. A know a new language is
already evil, but I see big benefit for the project this time. The
reason is that I can very tightly integrate it into rsyslog's features
like the upcoming loader. So it will be very cleanly and efficiently
integrated. For example, I don't think that I could support old-style
format *just within* the new style format with any third-party engine.

rsyslog work log

Yesterday's rsyslog work log:
- implemented data type conversion
- released 3.11.5
- implemented majority of comparison operations
- added some temporary testing aids to conf.c, so that we can debug
expression support as it is implemented
- fixed a couple of bugs in expression system
- added more operations to virtual machine - now works well with
- added PUSHMSGVAR operation to vm
- included expression support in filter module (and it works ;))
- added sysvar class
- added PUSHSYSVAR operation to vm
- added string concatenation operator & to RainerScript
- fixed segfault when pure string values were tried to be added

Monday, February 25, 2008

rsyslog work log

Past day's rsyslog work log:
- integrated patch from Peter Vrabec to change the build process to produce and from imtcp.c (in support of new gassapi
input module) - many thanks, Peter!
- applied patch by varmojfekoj to allow gssapi functionality to be
build as a separate plugin (so that gssapi and plain tcp
functionality can be individually distributed). Also inclulded
some other enhancements, most importantly initial compatibility
mode system
- added some information on old-style config to ABNF (probably
useful in getting us to a new config format)
- fixed bug in duplicate module load detection
- corrected config doc - IETF no longer intends to add facilities
- updated compatibility notes doc
- added some doc for imgssapi and imtcp input modules
- completed initial vmstk implementation
- begun implementing rsyslog virtual machine (vm class)
- worked a bit on type conversion (specified the interface)
- simplified var object, now only supports strings and numbers as
a single type
- worked a bit on var_t data type conversion
- added some thoughts on RainerScript
- worked a bit on conversion functions

Saturday, February 23, 2008

Introducing rainerscript ;) - and some rsyslog futures

What the heck is this? "rainerscript"? An acute case of megalomania?

Wait a moment... What's this all about? It all starts with expression support in rsyslog and the notorious question on what the future configuration file format for rsyslog should be. From the technical perspective, I think I am finally approaching a conclusion, which will probably look quite a bit like a script language. A script language that will be specifically tailored (and well suited) to processing (system) events.

Heck, another new term... Well, if you know Adiscon's MonitorWare family, its not new to you. In MonitorWare (where I consider rsyslog still to be a part of), we always thought about generic system events, and not syslog messages, event logs, snmp traps and all the like. The same concept somewhat naturally will apply to rsyslog and it begins to manifest with multiple input plugins. The imfile plugin is probably the first extension that brings this spirit to life. While all other plugins so far process syslog messages, imfile processes plain text files. Of course, they convert easily to syslog messages. But from a high-level design perspective you can think of event messages - some being syslog, some being file lines, some other being... use your imagination (or visit the MonitorWare site to get an idea where else events can come from).

The bottom line is that rsyslog has evolved and is further evolving into something that of course can process syslog messages. But it's not limited to that. It can process whatever system event occurs. It's just a matter of the input plugins. Of course, the rest of the engine must also support event classes other than syslog. That's, among others, what I am currently working on.

To be a really useful tool, its configuration must be advanced from what it currently is. Most specifically, we must be able to change event (read: message) properties on the fly, supply different parsing capabilities, maybe have some user-supplied logic during the event flow... and so on.

The most natural solution is to provide some (more or less powerful) scripting capabilities. I am convinced that's the right route, especially now that I have done the initial work on rsyslog expressions. It's clean, its very flexible, its extensible and -- it's easy to use. And in order to provide freedom of choice, I'll most probably introduce a very similar concept into Adiscon's commercial products. And finally, over time others may find it appealing to, so I can't outrule there will be other implementations of the same idea.

This made me come to the conclusion that I needed a better name than "rsyslog.conf" for this concept. Initially, a name of "syslogscript" came up my mind. But as I said, it's not just about syslog. The same applies to "logscript", we are *not* just talking about logs (at least in the future). I begun to think and after some time arrived at "EventScript". A perfect match - but also a third party trademark (even a registered one!). So forget about it. "rscript" - also a trademark. I ran out of good names and I also became impatient. I would also like to make sure that nobody again comes along to cause grief to me, just as Huwai did with it recent junk syslog patent. So I remembered a trick a German TV journal publisher used when his competitors filed law suit after law suite over name violations of his journal when he tried to start a new publication in a field that was obviously monopolized by a few insiders: he simply used his own name in the journal title. Of course, that doesn't make a name totally immune against legal action, but it seams to be a quite strong weapon against all these patent and tradmark trolls out there.

And so I finally arrived at the idea that a good name for the event scripting language may be "rainerscript". Some folks (not me! ;-) ) may even abbreviate it to rscript, fishing in trademark-mined territory. Don't do that or at least do it at your own risk ;)

I hope you find the choice of name acceptable -- and the overall direction we are taking a good one. As a side-note please let me assure that rsyslog's easy and simple sysklogd-like configuration will not go away. Rainerscript will fully support that syntax, as well as all currently existing $-configuration lines. While the previous configuration format will become part of rainerscript, you actually do not need to know anything about the scripting language unless you intend to do some pretty advanced things. Rainerscript is not something available immediately. It will evolve over time. A first glimpse at it will be available in 3.12.0 in form of the rsyslog expression support. With it, you can do things like (all of the below is on one line, which is important):

if $message contains 'test' and $facility == local7 then -/var/log/local7test.log

Notice the new if and the old style action specifier ("-/var/log..."). The script code is just a natural extension to existing configuration format.

Friday, February 22, 2008

rsyslog work log

Yesterday's rsyslog work log:
- changed rsCStrObj name to cstr_t, which is more inline with the
rest of rsyslog (now) and also much easier to type
- fixed some doc errors - thanks to Michael Biebl for pointing
them out
- some cleanup for 3.11.4 release
- first steps in implementing object interfaces (stage work for
later dynamic class loading)
- changed calling interface for some more objects
- some more interface changes
- changed tokenizer to utilize var class instead of scalar types
- modified parser and tokenizer to support slight ABNF modifications
from yesterday
- change in ABNF was wrong - made a slightly different change
- fixed bugs in tokenizer
- expression compiler finished (except bugs, of course ;))
- added empty vmstk class
- defined some entry points in vmstk

Thursday, February 21, 2008

rsyslog work log

Yesterday's rsyslog work log:
- created new class ctok_token
- created some notes on powertop and imfile
- basic implementation of expression parser parsing done
- improved ABNF a bit
- added support for C-like inline comments (/* comment... */)
- created design diagram for expression execution
- created var class out of property_t
- changed rsCStrDestruct() to use the new interface conventions
- $MainMessageQueueDiscardSeverity can now also handle textual severities
(previously only integers)
- bugfix: message object was not properly synchronized when the
main queue had a single thread and non-direct action queues were used
- added vmop class (stage for expression execution)
- added "contains" and "startwith" comparison operations
- defined initial set of opcodes
- created initial vmprg class
- used new classes in expr.c
- begun expr compile process, first steps done

Wednesday, February 20, 2008

on the way to expression support

If you have followed my work logs, you already know that I currently have targeted expression support for rsyslog. I have just finished the basic tokenizer/parser to parse expressions from the configuration file. They base on flexible ABNF definition and will operate on a typeless data type. Initially, expressions will only be available for filters, but further on they will move to more and more places in rsyslog.

The surprising fact is that expressions will also bring a virtual machine implementation to rsyslog. I thought a while, but a vm is actually the easiest (and cleanest) way to implement arbitrary expressions in rsyslog. As a side-effect, the vm, once there, will probably lay the foundation for very interesting future developments inside rsyslog. It also affects the future rsyslog.conf file format. Maybe we extend it into a programming-language like construct ("syslogscript" comes up my mind) ;)

In the mean time, here is a preliminary object model for the expression part of rsyslog as I plan to implement it (click on the image for a larger version):

rsyslog work log

Yesterday's rsyslog work log:
- added professional support options to doc set (also somthing that
we need to have in order to be an alternative to syslog-ng)
- added doc on how expressions will work
- cleaned up the stringbuf Construct interface
- did some cleanup on stringbuf calls - we now have much better
interfaces and macros
- moved config file code to its own file
- finally made CONT_LINES in config the only standard support (the
code contained code for other case, which were never executed by
the preprocessor)
- some include file cleanup
- added ctok class (the config tokenizer)
- done stage work to begin implement tokenizer
- implemented initial tokenizer (stage work for expr parser)
- begun implementation of expression parsing logic
- implemented, simpstr, var, number in tokenizer
- implemented function in tokenizer

Tuesday, February 19, 2008

Huwaii's Junk Patent on Syslog...

I wanted to share my excitement about a new patent with you:

I have just been able to find the one of the uttermost pieces of carp that the wrotten patent system has produced. May I introduce you to Miao's genius invention for the syslog protocol:

As a side-note, it has quite some technical flaws and shows some fundamental misunderstanding on how syslog works. But maybe that's what the real invention is...

I also find it quite disturbing how the tuning profile selection process from RFC 3080 is being re-invented in this invention. And it is also a nice steal to see my architecture slides

being represented in a childish way (or did you steal them from a freshmen's course book?). Thanks Huwaii and Miao for this great invention and stealing my work. I really appreciate it. I bet Marshall T. Rose will also be delighted.


rsyslog work log

Yesterday's rsyslog work log:
- changed interface "printchopped()" so that it looks more like
a generic message submission interface. Part of the ongoing
modularization effort.
- bugfix: invalid kernel log format -- see bug
- bugfix: fixed abort when invalid template was provided to an action
- applied patch from Michael Biebl to auto-detect new libdbi version
- bugfix: default port was not used on $UDPServerRun when none was
specified -
- removed no longer necessary signal from threads
- changed debug output request signal to SIGUSR2 (as originally
intented), restored SIGUSR1 semantics
- documented at least a bit about the debug settings
- released 3.11.3

Monday, February 18, 2008

rsyslog work log

Past days' rsyslog work log:
- enhanced file monitor doc
- implemented $InputFilePollInterval config directive
- error handling and cleanup in imfile
- improved file polling algorithm for more rapid file data delivery
- some more cleanup
- moved decoding of syslog names to a more appropriate place
- added new facility and severity syntaxes to cfsysline handler
- implemented $InputFileFacility config directive
- implemented $InputFileSeverity config directive
- created a bare template for omlibdbi (dbi output action)
- created an initial version of omlibdbi (does not yet work)
- did some more work on omlibdbi, but did not yet get libdbi working.
I guess its a compile problem, but have not found it so far.
- the libdbi problem was actually related to libdbi/distro packages;
fixed that by installing from source, now omlibdbi basically works
removed some debug code
- cleaned up omlibdbi - works now
- implemented $ActionLibdbiDriverDirectory config directive
- some cleanup
- doc improvements
- patched libdbi to work better with plugins
- adopted omlibdbi to use new version of libdbi
- improved man page a bit for the novice user

Friday, February 15, 2008

rsyslog work log

Yesterday's rsyslog work log:
- enhanced file monitor doc
- implemented $InputFilePollInterval config directive
- error handling and cleanup in imfile
- improved file polling algorithm for more rapid file data delivery
- some more cleanup
- moved decoding of syslog names to a more appropriate place
- added new facility and severity syntaxes to cfsysline handler
- implemented $InputFileFacility config directive
- implemented $InputFileSeverity config directive
- created a bare template for omlibdbi (dbi output action)
- created an initial version of omlibdbi (does not yet work)

Thursday, February 14, 2008

New Web Interface for rsyslog

I just wanted to make you aware that we now have seriously begun to work on phpLogCon v2, the new web interface for rsyslog. It's currently in the design phase and thus it is a very good time for suggestions, feature requests, and, and, and...

Current discussion can be found here:

As you probably know, I wasn't very happy with phpLogCon v1 the past months. It also received very low attention and consequently could not take off. We have decided the change that dramatically. You'll see a much enhanced, hopefully very useful interface. Of course, it will take some time. I hope to have a very rough, but useful, first version available some time in March.

I personally will only be involved in the design of the web app, not its actual implementation. I think this will be a very interesting project and it also offers a lot of potential.

rsyslog work log

Yesterday's rsyslog work log:
- added some code to expr.c - not yet to be used
- cleaned up imfile.c
- changed interface of logmsg() to make it more straightforward
- introduced a new, more powerful, message submission interface
submitMsg() in additon to logmsg()
- a first, rough implementation of imfile that is able to read files
(but does not persist or handle rotation or whatever)
- removed some left-over unnecessary dbgprintf's
- added ability to monitor file accross rotation
- fixed a race condition in DoDie() - cosmetic issue in debugging mode,
could not happen in production
- added the ability to persist current read location for the file
- some cleanup
- created initial doc for imfile plugin
- released a preview of 3.11.2 (with the file monitor)

Wednesday, February 13, 2008

rsyslog work log

Yesterday's rsyslog work log:
- still helped a bit with omsnmp
- applied patch from Michael Biebl that fixed my doc change from yesterday
which was somewhat incomplete
- fixed a bug in stringbuf.c related to STRINGBUF_TRIM_ALLOCSIZE, which
wasn't supposed to be used with rsyslog. Put a warning message up that
tells this feature is not tested and probably not worth the effort.
Thanks to Anders Blomdell fro bringing this to our attention
- somewhat improved performance of rsCStr obj
- fixed bug that caused invalid treatment of tabs (HT) in rsyslog.conf
- released 2.0.2
- reduced volume of debug output
- bugfix: setting for $EscapeCopntrolCharactersOnReceive was not
properly initialized
- clarified usage of space-cc property replacer option
- bugfix: discard action and backup actions did not work due to
problem in direct queue mode. Now fixed. Tracker was
- improved diagnostic information for abort cases
- some initial effort for malloc/free debugging support
- bugfix: using dynafile actions caused rsyslogd abort
- fixed man errors thanks to Michael Biebl
- released 3.11.1

Tuesday, February 12, 2008

rsyslog-to-rsyslog communication

I admit it is just a quick note, but I guess it tell a lot to those that are working in the protocol area:

In the long term, I'll move the protocol stack to RFC3080/3081 (including RFC 3195) as the primary rsyslog-to-rsyslog mode. I thought hard about it, but it is the best choice to have a) plugins only loaded on demand but b) the ability to decide upon runtime on the highest level of features/confidentiality. With liblogging, I already made serious investment in that protocol suite. It still not a trivial thing to do it in the way I intend to (with profile plugins) and it will probably an at least 50% rewrite of the 3080/81 code, but I came to the conclusion it is worth it. I think you'll like it when you see it in all its glory ;) But other things are currently more important...

If you don't get the idea what this is all about, just forget it. Really... it's not important to you then ;)

rsyslog work log

Yesterday's rsyslog work log:
- added x-info field to rsyslogd startup/shutdown message. Hopefully
points users to right location for further info (many don't even know
they run rsyslog ;))
- bugfix: trailing ":" of tag was lost while parsing legacy syslog messages
without timestamp - thanks to Anders Blomdell for providing a patch!
- worked on integrating omsnmp

Friday, February 01, 2008

Todays's ryslog worklog...

Today's rsyslog work log:
- applied documentation fix by Michael Biebl -- many thanks!
- added input-plugin interface specification in form of a (copy) template
input module
- cleaned up some no longer needed files, thanks to Michael Biebl for pointing
this out
- added sample config file to distribution tarball
- bugfix: immark did not have MARK flags set...
- very quickly hacked a rought outline of the file monitor (without any

rsyslog work log for an important day ;)

Yesterday's rsyslog work log:
- added some advanced features usually in demand to sample rsyslog.conf,
but left it commented out
- rename $TimoutWorkerThreadShutdown to $WorkerTimoutThreadShutdown
for consistency reasons
- changed default for action queue size to 1000 elements (more reasonable here)
- fixed bug in sample rsyslog.conf
- fixed wrong action suspend/resume handling
- we have some issue with the mutx in dbgoprint, but that is acceptable for
the time being, I just removed the deadlock codition (debug system only)
- bugfix: dbgoprint mutex - was too simple once I wrote the tracker item ;)
- bugfix: having fun with 32/64 bit portability - after 15 years, I finally
was trapped again ;) -- now fixed, sizes > 2GB supported on 32bit platforms

simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. ...