Thursday, March 24, 2011

using failover and asynchronous actions in rsyslog

I wanted to point out that failover actions and asynchronous processing does not work well in rsyslog -- at least if a simple approach is used. The reason is that there is a conceptual problem with them working both together: async actions will - by design - always return an "everything went OK" status (because we don't know otherwise, as things are async!). So this means failover processing will never see an error.

Now look the following config:

$modload imuxsock
$ActionQueueType LinkedList
$RepeatedMsgReduction on
*.* @@
$ActionExecOnlyWhenPreviousIsSuspended on
& @@
& -/home/logfile2
$ActionExecOnlyWhenPreviousIsSuspended off

If the Action Queue Type is set to linked list (and thus the action executed asynchronously), the other two actions will never be executed - because the async action always "succeeds".

There are two ways to solve this situation:
1) do run the action synchronously -- depending on your needs, this may be a solution or not
2) if you need to run it async, you need to define a new ruleset, which includes the config WITHOUT the async processing. Then, use omruleset to execute the newly defined ruleset as whole asynchronously. This is obviously a bit more complex, but will do what you need.

I hope this resolves some confusion about the failover functionality (and, yes, a better config language would make this less painful -- hopefully we will finally be able to write one ;)).

1 comment:

Greg O'Brien said...

Hi Rainer,
First of all thanks for all of the work you do on this project; it is an invaluable tool.
I currently find myself running into unexpected failover (or lack thereof) behavior, and I think the issue illustrated in this post is the culprit. Could you please explain the second solution you have posed here? I am unsure of how to accomplish it. Any examples would help tremendously as well.
Thank you,
Greg O'Brien

simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. ...