Hello Alan, thanks for your comments. I try to explain it. On Friday 17 of April 2015 17:41:08 Alan DeKok wrote:
On Apr 16, 2015, at 2:44 PM, Rygl Aleš <ales@rygl.net> wrote:
I am trying to migrate the freeradius 2.2.5 to 3.0.7. The server is doing just Acounting request processing and is loading requests to MariaDB with radacct table on InnoDB engine allowing row level locking. There are 128 detail files derived from Calling-Station-Id attribute using mod 128: Why?
The goal is to have server which - responds as fast as possible. - is resilient to occasional peaks in the accounting traffic with magnitude of 10 or more - drops packets as little as possible - utilizes parallel processing on the DB side as much as possible I was not able to reach this with InnoDB engine with just sigle queue. It was pretty fast with MEMORY engine but it suffers from locking on table level (and uses just a single connection to DB). I was affraid that talking to DB directly (without fallback to a file buffer) would had problems in case of peaks. So the idea behind is to rather write to a file which IHMO cheaper that perform a DB transaction where I have to wait for a lock. While testing the performance of 2.2.5 server with multiple queues I have found out that 128 queues was giving the best results from the throughput point of view. The DB could be down for an hour and about 3 GB of data in file were loaded then to DB within 20 mins.
There shouldn't be a need to create 128 detail file writers / readers. Version 2 was a little slow reading the detail files. Version 3 is much faster, and more robust.
I can confirm that it is definitely faster. I have followed Arran's recomendations and reduced the number of queues to 12 and it looks good. I will play with this more.
I'm not sure. The file descriptor should NOT be closed while it's being used.
And for v3, you should set "track = yes". It means that server re-starts don't cause re-reads of old accounting packets.
I will try again. Actually I have two more virtual servers in the config with a single queue processing just couple of packets per second and this error was there too. I will do some debugs here.
Don't configure 128 queues. There shouldn't be a need to do that.
I will try the direct connection to DB with a fallback to file. Thanks Ales