Friday, November 12, 2010

Call for Log Samples

My log normalization effort made good progress and I have a very rough first proof of concept available. It will take a log sample database, and transform input log files to a CEE-like output format.

Now I am looking at ways to practice-test it. So I'd appreciate if you could point me to some sources of log files. It mustn't be terabytes, but they should be anonymized and be usable in the public Internet. For obvious reasons, it would be good if they are from widely deployed devices.

I would use a subset of these samples to extract usable sample database entries and see how the run through the normalizer.



Js Opdebeeck said...


Can you detail
1. what kind of logs you like ? Exotic ones, or also apache, Msft, Firewall.
2. The format ... raw (dump) or output from (r)syslog(-ng)

Kind regards

Rainer said...

Sure, and thanks for the attention. I am actually after all I can get hold off, it mustn't be exotic. In fact, a couple of Cisco IOS logs would be very good.

Also raw and written file is equally well. I'll use that to torture the normalizer.


Brandon McGinty said...

Try here:

Rainer said...


thanks for the pointer, this is very useful. Unfortunately, it lacks Cisco IOS logs, which I would really like to have some publically usable sample from.


simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. ...