Thursday, March 13, 2008

RELP - the reliable event logging protocol

Please welcome RELP, the "reliable event logging protocol". What's that???

I am currently really bugged by shortcomings in the plain tcp syslog protocol in its current form. And I finally made up my mind. Instead of wasting time on fixing broken plain tcp syslog transport mode (e.g. by implementing half-duplex, which isn't standard anyhow), I'll do "the right thing". I thought rfc 3195 is the right thing. But it carries a lot of overhead that I really don't need. And, most importantly, any standard additions takes ages to go through the IETF (I know what I am talking about, have finally succeeded to get a better syslog rfc though it in "just" 4 (or 5?) years -- and it is still waiting to be published...).

So instead of adding on any of these existing protocols, I'll do a new, lightweight but capable protocol specifically designed to solve the shortcomings we currently experience. Please welcome RELP, the "reliable event logging protocol" (name based on the quite successful selp [simple event logging protocol] effort: http://www.monitorware.com/en/workinprogress/selp.txt).

Relp will evolve over time. I hope to do something useful relatively soon, and it will be extended as the project progresses. The ultimate goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog communications. I'll don't care about the outside world, so I can do the best of breed solution. For the rest of the world, rsyslog will of course continue to support plain tcp syslog and will get better support for rfc 3195. But if you wanna go hardcore on high-reliability, high-performance event logging, relp will be your choice.

Technically, I'll split this off into rsyslog relp input and output plugins AND an independent librelp, which provides core protocol functionality (just in case somebody else wants to support it in the long term, so this will not be tied to rsyslog).

2 comments:

Emilio Joel said...

Hi Rainer

Thanks for you article about rsyslog reliable.

I am trying to do the same with RELP but , my reliability tests dont work.

My client configuration:


$ModLoad imuxsock.so # provides support for local system logging (e.g. via
logger command)
$ModLoad imklog.so # provides kernel logging support (previously done by
rklogd)
$ModLoad imuxsock
$ModLoad omrelp

$WorkDirectory /var/spool/rsyslog # location for work (spool) file
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

*.info;mail.none;authpriv.none;cron.none /var/log/messages

authpriv.* /var/log/secure

mail.* -/var/log/maillog

cron.* /var/log/cron

*.emerg *

uucp,news.crit /var/log/spooler

local7.* /var/log/boot.log

local1.info /var/log/webadmin.log
:HOSTNAME, contains, "webadmin01" ~
local1.info :omrelp:192.168.6.219:2514



My server configuration:


$ModLoad imuxsock.so # provides support for local system logging (e.g. via
logger command)
$ModLoad imklog.so # provides kernel logging support (previously done by
rklogd)
$ModLoad imrelp
$InputRELPServerRun 2514

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

*.info;mail.none;authpriv.none;cron.none /var/log/messages

authpriv.* /var/log/secure

mail.* -/var/log/maillog

cron.* /var/log/cron

*.emerg *

uucp,news.crit /var/log/spooler

local7.* /var/log/boot.log

local1.info /var/log/webadmin.log



The transmition between client and server work fine but if the server reboot ,
I lose all the msg in this time.
the msg never are resend to the server.


Could you please tell me if my configuration is well? How I can test the
reliability?

Thank in advance
Emilio

Rainer said...

Hi Emilio,

thanks for your message, but please re-post to the rsyslog forum, as this makes it much easier to track ;)

Thanks,
Rainer