2.2.0 & dhcp: regression

[Eugene Grosbein]

On 12.07.2013 18:39, Phil Mayers wrote:

>> Our database is powerful enough to deal with so many requests.
>> We may easily get that many requests and want to be able to process
>> them in parallel without needless queueing.
> 
> With respect, this is a pretty basic logic.
> 
> The figure of merit here is offered load in terms of request/sec, and 
> the average/max processing time per-request.
> 
> If you have 1000 request/sec and each request takes 1 millisecond to 
> process, a single thread is sufficient.
> 
> If you have 100,000 request/sec and each request takes 10 milliseconds, 
> then you need 100 threads.
> 
> If your database is "so powerful" it shouldn't be taking too long, so 
> unless you have a truly enormous number of request/sec, you don't need 
> 1000 threads.

Yes, we may have large number of request/sec and need to be able to process them instantly.

> If you really do have that many request/sec, you probably should look at 
> some form of load balancing, rather than having enormous thread pool. 
> The thread pool performance will not scale linearly - various server 
> internal data structures are locked, and you will probably run into lock 
> contention at high thread counts.

We performed extensive testing and found no noticeable contention
while using 2.1.12. With lesser pool size we have unneeded queueing delays.

Anyway, that's another topic. The regression in 2.2.0 is what I wanted to discuss here.

>>> Anyway, the problem is almost certainly system ulimits. I don't know why
>>> it's different under 2.2 to 2.1, but I would look into ulimit.
>>
>> ulimit for files is over 11000 here. And radiusd successfully opens more
>> than 1024 files. It just breaks afterwards.
> 
> It could be the use of select() then. If rlm_perl opens FDs 5-1023, then 
> the radius server needs to open some sockets, it will get FD #1024, 
> which might cause select to complain.

I rebuilt freeradius-2.2.0 with CFLAGS containing -DFD_SETSIZE=4096,
that changed nothing.

> Try running the server under "strace" and see if you can see where it 
> goes wrong, and what it calls just before it does.

I'll try, thanks.