Detail file handling

[Alan DeKok]

Peter Nixon wrote:
> When running detail2db.pl (the perl import script I posted to the list a few 
> weeks ago) which only uses a single thread and a single DB socket, from a 
> remote machine (same machine as freeradius, not same as PostgreSQL) the DB 
> server load spikes to 2+ and the DB sometimes responds slow enough that 
> FreeRADIUS Auth queries fail and errors are logged (not always.. depends on 
> traffic levels)

  Ouch.  That's not nice.  I had thought that the priority queuing would
ensure that authentication packets get handled before detail files.

  On the other hand, if the CPU is pegged from handling the detail
packets, it won't have time to notice that an authentication packet has
arrived.

> For that reason I have added a 3000 usec sleep in between processing of each 
> packet, which keeps the DB load at about 0.8 while processing a months worth 
> of detail files. Note that perl should be a bit slower than C so a good 
> default sleep time is probably around 5000usec..

  Hmm... Hard-coded sleep times don't adapt well to changing
circumstances.  Maybe changing the code to require *2* waiting threads
would be better.  That way, the "max active threads" limit wouldn't be
reached.  Since that's not reached, more threads won't ever be created
to handle the flood of detail packets.  And, there will always be one
waiting thread, which can handle any authentication packet that comes in.

  Alan DeKok.
--
  http://deployingradius.com       - The web site of the book
  http://deployingradius.com/blog/ - The blog