Tuesday, February 26, 2008

Why RainerScript?

After my initial announcement of RainerScript, I got some good feedback. I'd like to reproduce an especially useful discussion here:

> I wondered if using a complete scripting language for configuraton
> is not making it overly complex and error prone. I specifically means
> function definition etc. here.

The key is that I will model it so that the complexity is only seen when
it is actually needed. Most importantly, it will continue to understand
plain old syslog.conf format. So most peoply will probably never see the
scripting features. HOWEVER, I think it is useful to have these for a
few select folks who really need them. Function definitions, for
example, will counter syslog-ng's feature to define a filter. I don't
know if it is actually worth and useful to define a filter to reuse it.
But with the function, you can do it. So those coming from syslog-ng can
find their favorite feature ;)

I think it is probably worth to think about how I came to the idea of
scripting support. The root cause is complex logic, complex expressions.
A few days ago I started to implement them ... and quickly found out
that actually the easiest way is "to do it right". So I now have a
parse, bytecode and an interpreter/virtual machine running in rsyslog.
All of this "just" to support complex expressions. While I thought about
it, I finally realized that it takes just a little bit to extend these
things so that the end result will be an easy to use scripting facility.
That also solves the config file issues that's dangeling so long and, as
a side-effect, brings up some interesting options (for those that have
need for it).

For example, I thought about how to best handle rewriting message
content. Now, the solution is obvious. I'll implement a "set" facility
in the language that enables message variables to be rewritten on the
fly (e.g. for folks who would like to drop some parts of the message). I
agree, all not pretty mainstream, but useful in some cases. And keep in
mind that it comes at a (complexity) cost only if you need it. That's
based on the fact that expression support was originally planned as a
by-line of the current config format, so it needed to work with the rest
in any way ;)

> But if you really need that kind of flexibility and scripting
> capabilites, have you considered to use existing languages, like lua
> , which were specifically designed to be embedded into
> applications?

Not really, because of the way it has evolved. A know a new language is
already evil, but I see big benefit for the project this time. The
reason is that I can very tightly integrate it into rsyslog's features
like the upcoming loader. So it will be very cleanly and efficiently
integrated. For example, I don't think that I could support old-style
format *just within* the new style format with any third-party engine.

No comments:

Introducing new team member

Good news: we have some new folks working on the rsyslog project. In a small mini-series of two blog postings I'd like to introduce the...