Wednesday, December 21, 2011

Happy Holidays to Everyone!

           *             ,
                      <     >
     *                 /.-.\         *
              *        `/&\`                   *
                     /_o.I %_\    *
        *           (`'--:o(_@;
                   /`;--.,__ `')             *
                  ;@`o % O,*`'`&\ 
            *    (`'--)_@ ;o %'()\      *
                /&*,()~o`;-.,_ `""`)
     *          /`,@ ;+& () o*`;-';\
               (`""--.,_0 +% @' &()\
               /-.,_    ``''--....-'`)  *
          *    /@%;o`:;'--,.__   __.'\
              ;*,&(); @ % &^;~`"`o;@();         *
              /(); o^~; & ().o@*&`;&%O\
        jgs   `"="==""==,,,.,="=="==="`
         '`         \)_`"""""`
                 .--' ')
               o(  )_-\ 
Happy holidays to everyone! It was fun communicating with all of your and bringing new features online. This year, we did not only extend rsyslog, but worked an a couple of enhancement in related topics, like log normalization, the log store, better Windows support and many improvements in the web based viewer.

Special thanks to all of who contributed code, good bug reports, ideas and encouragement - or helped funding the project by purchasing support contracts or incidents.

It was a busy year and I will try to relax a bit the next days. So please bear with me if I do not respond as quickly as usual (I will try to even be a few days offline ;)).

Thanks again guys, have a great holiday season and all the best, most importantly health, for 2012!


PS: if you enjoy the ASCII art, have a look at the artist's web site!

Wednesday, December 14, 2011

Feedback Request for digitally signed log store

I have just written about how I plan to implement digital signatures in LogStore, the secure store used by LogTools. The log store digital signature proposal details how and when signatures are written and provides reasoning why it will probably happen this way. There are two goals for the proposal: one is to document how things will work and the other, probably more important, one is to draw some feedback. It is easy to get security tools wrong, and even those with highest experience in that area (which I have not!) can fail. So it would be very beneficial to have some other folks read the proposal and comment on weaknesses they find - or simply things they would do differently or add to the overall idea. With that said, please read the (small) paper and provide feedback ;-).

Please keep on your mind that his is not only related to syslog but can  be used with any text-based log (including binary logs that are converted to text, e.g. by base64 encoding them). So it can affect you even if you are not interested in syslog itself. My (mostly uneducated) assumption is that this could be a toolset of great use for computer forensics.

Tuesday, December 13, 2011

LogTools 0.1.0 Released

I finally managed to do the initial public release of LogTools, a set of useful utilities for log data processing. Their current most important feature is the initial version of LogStore, a tamper-proof way to store textual log data especially designed for long-term archival. Note that LogTools perfectly process syslog messages, but can be used for anything that is text-based (like Apache or other application text logs).

I am very happy to finally have the initial release ready. Full details can be found in the release announcement. I initially thought it would become available early last week, but I wanted to create some packages. As I had never done this before, it turned out to become a bit of a problem for me. For the time being, I have settled to do an experimental Debian package via checkinstall. While this is obviously a quick and dirty solution, it enables folks to obtain LogTools via the easy way. Also, I don't think it is too problematic, because in essence only some user-tools are installed that do not affect anything else on the system. But, of course, I'll think about better packaging as the project continues.

At this point, I would be very interested in feedback both on the current tools as well on what would be considered a plus in the future. Please let me know!

Tuesday, December 06, 2011

Announcing LogStore

While it probably is a bit early for a "real" announcement,  I wanted to tell a bit about the project I have been working on the past days, a dedicated storage for (sys)log messages. It will be available as part of LogTools, the actual project I am working with. A key feature of the LogStore format will be its tamper-proofness. I wanted to write such an improved storage system for quite a while. However, I have to admit that the recent journald proposal brought more life to it. While the journald proposal aims at building a kind of Window Event Log for Linux, the LogTools effort is more interested in traditional text log files (but I won't outrule going beyond that in the future).

It is important to note that LogTools, and their storage format LogStore, can protect any kind of text file. Of course, it is great for syslog logs, but you may also secure things like Apache http logs or whatever else you have in text format.

You may probably remember that I was - and still am - very skeptic about the way journald tries to secure logs via hash chains (be sure to read the comments as well!). While the journald propsal has some technical deficits, I learned that many folks we interested in this kind of hash-chaining, even though they knew it would not be truly tamperproof (I followed a lot of forum posts). Seeing this, I thought it may be useful to provide this level of protection inside some simple to use tools, what gave birth to LogTools. Still, my assesment in regard to journald holds to the current LogStore format as well: it is far from being real cryptography, it is insecure and it may be counter-productive if it generates a false sense of security. However, if one knows the limits, it can provide some useful function. So be sure to know what you do when you use these tools!

For LogStore, I have also planned to employ some real cryptography and cryptographically sign the hash chain. This will actually make the log tamperproof in a very strong sense as long as the signing key is not compromised. That functionality will be addressed once the initial release is out.

The LogStore format itself is deliberately defined in a text-tool friendly way and well documented. For your initial review, I have included its man page below.

The LogTools project is currently available only via the LogTools git. I plan to finish the remaining man pages soon (at latest this week), and then create distribution tarballs (and hopefully some simple packages, but this needs to be seen).

Feedback on this effort is appreciated.


logstore - enhanced log message storage  


The logstore is an enhanced log message storage. It can be used to store log messages in a way that secures their integrity. Currently, all data is stored in sequential files.
The logstore provides integrity checks by chaining of SHA1-checksums. Each log record (except the first) is hashed, together with the hash of the previous record. As such, manipulations inside the log store can be detected, as long as the checksums of all records are not also recomputed. Sequential logstore files are pure text files.


A sequential logstore is a text file containing variable-length records. Within each record, there is recordtype, cryptodesignator and content.
A single character designating the type of record. Currently only "m" is defined, which specified original message text.
Variable length, terminated by a colon. The is printable data that has some cryptographic function. For example, for "m"-type recrods it is the message's chained SHA1 hash.
Variable length content terminated by a LF (. For obvious reasons, LF is not permitted within the message. Also, the US-ASCII NUL character ( ) is forbidden in order to prevent trouble with text based tools. It is suggested that only printable characters are used inside the message, but this is currently not enforced.
The structure is recordtype Rsyslog has a modular design. Consequently, there is a growing number of modules. See the html documentation for their full description.


In its current form, logstore provides limited security. While it is possible to verify the correctness of the hash chain, an attacker may simply rewrite the complete file, computing new hashes. However, protection can be (manually) gained by saving the last hash inside the file to a separate location. If so, one can compare the last hash with this previously saved information and check if it is still valid. If someone mangeled the store, this will not be the case. Once the authenticy of the last hash has been proven, it is easy to verify the rest ot the file. The logreader (1) tool can be used to do that.
In the future, cryptographic signatures based on public key cryptography will be used to protect the hash chain.


The following is a minimalistic 4-line sample of a sequential logstore file.
m04e3324670626451755aa2257a9b92395e26c2e4:line 1
m347d4500ea11fa41800a58972699e57e0c0d7cd7:line 2
m3c329d40e37ae20c475c06bfaab892892ef4579d:line 3
m807cc61d7b04cbc1f048810df9d3a652988d745e:line 4
Note that all records have "m" in the first postion, designating them as message records. The cryptographic hash in line one is a SHA1 hash of just line one's content, whereas the hashes for lines n (with n>1) are taken after the concatenation of hash(n-1) and line(n), without the colon. So in order to obtain line two's hash, the following string is hashed:


logreader(1), logwriter(1), liblogtools(3)


Rainer Gerhards (