Monday, February 12, 2018

simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. The current method is not hard either, but often looks a bit clumsy. The new way of doing things will most probably be part of the 8.33 release.

You now can define a template as follows:

template(name="outfmt" type="list" option.jsonf="on") {
         property(outname="@timestamp"
                  name="timereported" 
                  dateFormat="rfc3339" format="jsonf")
         property(outname="host"
                  name="hostname" format="jsonf")
         property(outname="severity"
                  name="syslogseverity-text" caseConversion="upper" format="jsonf")
         property(outname="facility"
                  name="syslogfacility-text" format="jsonf")
         property(outname="syslog-tag"
                  name="syslogtag" format="jsonf")
         property(outname="source"
                  name="app-name" format="jsonf")
         property(outname="message"
                  name="msg" format="jsonf")

     }

This will generate JSON. Here is a pretty-printed version of the generated output:
{
  "@timestamp": "2018-03-01T01:00:00+00:00",
  "host": "172.20.245.8",
  "severity": "DEBUG",
  "facility": "local4",
  "syslog-tag": "app[1666]",
  "source": "app",
  "message": " this is my syslog message"
}
Note: the actual output will be compact on a single "line", as this is most useful with RESTful APIs.

Future versions of rsyslog may see additional simplifications in generating the JSON. For example, I currently think about removing the need to give format="jsonf" for each property.

The functionality described here is being added via this pull request.

simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. ...