2.2.0 & dhcp: regression

[Eugene Grosbein]

On 12.07.2013 17:17, Eugene Grosbein wrote:
> Hi!
> 
> We have been running FreeRADIUS 2.1.12/dhcp module with success for long time
> with FreeBSD 8.
> 
> Our DHCP perl script opens two file descriptors (per thread):
> one for database connection TCP socket and one for syslog
> (/var/run/log unix domain socket). With pool size consisting of 1000 threads,
> this worked just fine with 2.1.12
> 
> Now, as we upgraded to 2.2.0, this schema stopped to work.
> It works if I decrease maximum pool size so that total amount of radiusd's
> open file descriptors does not exceed 1023, this corresponds to
> 501 threads in the pool. If I use 502 threads or more, it initially works
> until the process creates enough threads to open 1024th file descriptor,
> then it starts to write to its log:
> 
> Fri Jul 12 17:09:13 2013 : Info: WARNING: Child is hung for request 727 in component post-auth module perl.
> Fri Jul 12 17:09:14 2013 : Info: WARNING: Child is hung for request 765 in component post-auth module perl.
> 
> I've tried to recompile freeradius with CFLAGS including -DFD_SIZE=4096,
> that changed nothing.
> 
> If I disable syslog usage in my perl script thus decreasing open files
> from 2 per thread to only one, then 2.2.0 runs with 1000 threads just fine.
> Again, increase of thread pool size breaks after open of 1024th file descriptor.
> 
> Please help. We need at least 1000 concurrent threads to deal with the load here.
> Our hardware has enough raw power and we do not like to create useless queueing delays.

Forgot to mention that operating system's open files limit for freeradius is over 11000.
And file descriptors are numbered starting from zero, so descriptor 1024 is really 1025th.

radiusd works fine until it has descriptors 0-1023 only and breaks with creation
of descriptor 1024: it stops calling perl routing post_auth and thus,
processing of DHCP requests ceases.