Freeradius 3.0.7 and multiple buffered-sql servers - detail file issues

[Rygl Aleš]

Hello Alan,
thanks for your comments. I try to explain it.

On Friday 17 of April 2015 17:41:08 Alan DeKok wrote:
> On Apr 16, 2015, at 2:44 PM, Rygl Aleš <ales at rygl.net> wrote:
> > I am trying to migrate the freeradius 2.2.5 to 3.0.7. The server is doing 
just Acounting request processing and is loading requests to MariaDB with 
radacct table on InnoDB engine allowing row level locking.  There are 128 
detail files derived from Calling-Station-Id attribute using mod 128:
>   Why?

The goal is to have server which
- responds as fast as possible.  
- is resilient to occasional peaks in the accounting traffic with magnitude of 
10 or more
- drops packets as little as possible 
- utilizes parallel processing on the DB side as much as possible

I was not able to reach this with InnoDB engine with just sigle queue. It was 
pretty fast with MEMORY engine but it suffers from locking on table level (and 
uses just a single connection to DB). I was affraid that talking to DB 
directly (without fallback to a file buffer) would had problems in case of 
peaks. So the idea behind is to rather write to a file which IHMO cheaper that 
perform a DB transaction where I have to wait for a lock.

While testing the performance of 2.2.5 server with multiple queues I have 
found out that 128 queues was giving the best results from the throughput 
point of view.  The DB could be down for an hour and about 3 GB of data in 
file were loaded then to DB within 20 mins.

 >   There shouldn't be a need to create 128 detail file writers / readers. 
> Version 2 was a little slow reading the detail files.  Version 3 is much
> faster, and more robust.

I can confirm that it is definitely faster. I have followed Arran's 
recomendations and reduced the number of queues to 12 and it looks good. I 
will play with this more.

>   I'm not sure.  The file descriptor should NOT be closed while it's being
> used.
> 
>   And for v3, you should set "track = yes".  It means that server re-starts
> don't cause re-reads of old accounting packets.

I will try again. Actually I have two more virtual servers in the config with 
a single queue processing just couple of packets per second and this error was 
there too. I will do some debugs here.

>   Don't configure 128 queues.  There shouldn't be a need to do that.

I will try the direct connection to DB with a fallback to file.

Thanks
Ales