Friday, March 30, 2012

Current state of ommongodb default schema

Rsyslog's ommongodb provides a default schema which is used for syslog data if no other is specified. It tries to align with the lumberjack project, so the schema may change in the next weeks, as hopefully a standard field set is defined there. I originally started with a very small set of fields (based on early cee/lumberjack schema), but it turned out to be too small to be really useful for real-world applications. So I have added a couple of fields today. The currently supported fields are:
  • sys - name of the system the message originated from (STRING)
  • time - timestamp from the syslog message (UTC_DATETIME)
  • time_rcvd - timestamp when the rsyslog instance received the message (UTC_DATETIME)
  • msg - the free-form message text (STRING)
  • syslog_fac - the syslog facility in numerical form, see RFC5424 to decode (INT32)
  • syslog_sever - the syslog severity in numerical form, see RFC5424 to decode (INT32)
  • syslog_tag - the traditional syslog tag (STRING)
  • procid - the name of the process that emitted the message (STRING)
  • pid  - the process id of the the process that emitted the message (STRING)
  • level - a severity level based on the lumberjack schema definition (STRING)
Please also see my previous blog post on cee/lumberjack schema mapping, which most importantly describes the current level mapping. 

Note that the default schema currently does NOT contain data obtained by parsing cee-enhanced syslog JSON part of the message. Current thinking is that we probably best include this as a sub-elements, maybe together with other structured data like RFC5424 structured data. This is currently being worked on. It's less missing time to implement but the desire to avoid re-doing things as the spec changes. Anyhow, I'll probably have a "timeout" after which I will implement at least some idea (after all, not too much work will be lost if things change).

If you use this schema, please keep in mind that it is experimental. At this stage I will not try to remain backwards compatible when I do changes. So excpect that newer versions may break your things!

No comments:

Busy at the moment...

Some might have noticed that I am not as active as usual on the rsyslog project . As this seems to turn out to keep at least for the upcomi...