Tuesday, April 15, 2008

on the rsyslog runtime

I had a conversation on the new runtime design for rsyslog. I think it had a quite some good technical information (and brief), so I reproduce it here:

> The new design is:
> rsyslog core (GPL)
> rsyslog runtime (LGPL)
> modules (whatever)

Yes, actually I intended to say that ;)

> what is the interface between
> rsyslog core/syslog-ng <> runtime? Pipe/linked
> runtime <> module? Pipe/linked
The runtime always needs to be linked. A few cases:

rsyslog <> runtime --> always linked
syslog-ng/whatever <> runtime... linked
runtime <> module -> linked

BUT the interesting case is:

rsyslog <> plugin -> linked or pipe
syslog-ng/whatever -> linked or pipe

In these cases, it depends on build parameters (all of this of course
not yet implemented). I would anticipate these combinations to be found
in practice:

rsyslog <> plugin -> linked [pipe for new functionality on old engine]
syslog-ng/whatever -> pipe

So with rsyslog <> plugin, that would just a fallback if you need it for
some reason (e.g. run a v3 engine with a v4 plugin WITHOUT the need to
backport it [requires a v3 and v4 runtime to be present on the system,

For non-rsyslog syslogd's I expect that pipe is always used, because I
do not think they'll change their environment to adopt to the runtime.

rsyslog will provide wrappers for either interface. They will come as
separate binaries. There will be an input and output plugin to allow any
process (not necessarily rsyslog technology) to utilize a standard unix
pipe interface for producing and consuming messages.

Library modules will never use the pipe interface - it's too slow and
too loosely coupled.

linking offers much greater performance and is lossless.
pipe is comparatively slow and may lose some messages.

Pipe is obviously a much easier and thus universal interface than the
plugin interface.

No comments:

simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. ...