Friday, October 04, 2013

New Queue Defaults in rsyslog 7.5

As regular readers of my blog know, we are moving towards preferring enterprise needs vs. low-end system needs in rsyslog. This is part of the changes in the logging world induced by systemd journal (the full story can be found here).

Many of the main queue and ruleset queue default parameters were a compromise, and much more in favor of low-end systems than enterprises. Most importantly, the queue sizes were very small, done so in an approach to save virtual memory space. With the 7.5.4 release, this will change. While the default size was 10,000 msgs so far, it has been increased 10-fold to 100,000. The main reason is that the inputs nowadays batch together quite some messages, which gives us very good performance on busy systems. It is not uncommon that e.g. the tcp input submits 2,000 messages as once. With the previous defaults, that meant the main queue could hold 5 such submission. Now we got much more head room.

Note that some other changes were made alongside. The dequeue batch size has been increased to 256 from the previous value of 32. The max number of worker threads has been increased to two, removing our previous conservative setting of one. At the same time, we now require at least 40,000 messages to be inside the queue before the second worker is activated. So this will only happen on very busy systems. Note that the previous value of 100 messages was really an artifact of long gone-away times and usually meant immediate activate of the maximum number of workers, what was quite contrary to the intention of that parameter.

Of course, these are just changed defaults. They can always be overridden by explicit settings. For those configs that already did this, nothing changes at all.

No comments:

rsyslog 8.31 - an important release

Today, we release rsyslog 8.31. This is probably one of the biggest releases in the past couple of years. While it also offers great new fu...