speed of detail reader server

[Arran Cudbard-Bell]

On Jun 22, 2010, at 4:25 PM, Michael Fowler wrote:

> On Mon, Jun 21, 2010 at 07:48:19PM +0300, Alexandru Oprisan wrote:
>>    I'm using freeradius 2.1.6 on gentoo to do decoupled accounting.
>>    I have everything set up, the only problem seems to be the speed of  
>> the 'detail reader server'. I do quite a lot off accounting packets on  
>> the box ( no auth ), the detail writer has already written 1.9G of  
>> details ( in about 5 hours ) to hourly files, but the detail reader is  
>> lagging far behind (still working on the first file). I only see about 1  
>> row inserted per second, but I know the server can do far more.
>>    I have set load-factor to 100, doesn't seem to help.
> 
> The detail reader is serial in nature, meaning it will read a packet,
> transmit it, and wait for a response.  Upon receipt of an ack it will
> repeat for the next packet.  If you have any sort of latency on your
> link you can easily get heavily backlogged.
> 
> We ended up implementing a proxy in between the detail file reader and
> the remote end; the proxy (a custom Perl script) transmits across
> several sockets, which the remote end seems to be able to deal with
> properly.  This has alleviated any backlog.
> 
> I have grand plans to modify the reader to transmit non-serially, but so
> far have not had the tuits to apply to the problem.  I'm not sure I would
> recommend the proxy solution, but if you can manage it, it may be a
> reasonable stop-gap.

The simplest solution is to load balance across multiple detail writer/reader instances based on NAS IP Address, you can then make the queue as wide as you want without dealing with mis-ordered packets.

I used the expr module with the modulo operator to do the balancing.

-Arran


> 
> --
> Michael Fowler
> www.shoebox.net
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html