So the typical logging problem, as seen from the syslog perspective, is:
- there exists events that need to be logged
- a single "higher-level" event E may consist of a
number of fine-grained lower level events e_i - each of the e_i's may be on different
systems / proxies - each e_i consists of a subset of properties
p_j from a set of all possible common properties P - in order to gain higher-level knowledge, the
high-level event E must be reconstructed from
e_i's obtained from *various* sources - a transport mechanism must exist to move event
e_i records from one system to another, e.g., to
a central correlator - systems from many different suppliers may be involved,
resulting in different syntax and semantic of
the higher-level objects - there is potentially a massive amount of events
- events potentially need to be stored for
an extended period of time - quick review of at least the current event data
(today, past week) is often desired - there exists lots of noise data
- the data needs to be fed into backend processes,
like billing systems
2 comments:
In Healthcare this is the problem behind the "Account of Disclosures" problem. I am constantly blogging to help people understand this. Too many influential people think that one can capture at the E level. http://healthcaresecprivacy.blogspot.com/
Hi John,
I feel (and share ;)) your pain.
Eric Fitzgerald (then of the Microsoft Event Log "Division") gave a pretty good -and brief- description of this "thin vs. fat log" problem on the loganalysis public mailing list. While the list archive has long gone away, I have quoted him in one of my (unfinished...) papers. It probably is a good read:
http://www.monitorware.com/en/workinprogress/needle-in-haystack.php
Search for "thin vs. fat".
I've also visited your blog and it contains some very good posts. I will keep an eye on it.
Thanks for sharing,
Rainer
Post a Comment