Detail file handling

[Peter Nixon]

On Fri 04 May 2007, Alan DeKok wrote:
> Peter Nixon wrote:
> > Only thing that comes to mind is that you may want to add a "delay"
> > option that pauses the read thread for X milliseconds in between packets
> > to allow throttling of packets. This is not critical, but rather a "nice
> > to have" option that should be trivial to implement while you are
> > digging around in that section of the code..
>
>   OK.  Right now, it reads the packets as fast as the back-end can
> process them.  Throttling means that the machine is still responsive in
> the event of a large backlog of accounting packets.

Right. My production PostgreSQL server which has a redundant pair of 
FreeRADIUS 2.0pre servers with SQL auth, accounting and dual sqlippools each 
poking at it sits on a 15min system load average of 0.12

When running detail2db.pl (the perl import script I posted to the list a few 
weeks ago) which only uses a single thread and a single DB socket, from a 
remote machine (same machine as freeradius, not same as PostgreSQL) the DB 
server load spikes to 2+ and the DB sometimes responds slow enough that 
FreeRADIUS Auth queries fail and errors are logged (not always.. depends on 
traffic levels)

For that reason I have added a 3000 usec sleep in between processing of each 
packet, which keeps the DB load at about 0.8 while processing a months worth 
of detail files. Note that perl should be a bit slower than C so a good 
default sleep time is probably around 5000usec..

Cheers
-- 

Peter Nixon
http://www.peternixon.net/
PGP Key: http://www.peternixon.net/public.asc