FATAL: Thread create failed: Cannot allocate memory

[A.L.M.Buxey at lboro.ac.uk]

Hi,

> We had one of our MAC-auth radius server instances hang up with this
> error at about 0200 this morning.
> 
> That server receives pretty heavy load, and it's bursty, so we see this
> a couple of times a day:
> 
> The maximum number of threads (32) are active, cannot spawn new thread
> to handle request
> 
> ...but it does not cause problems. An inability to create a new thread
> is an entirely different matter though; it implies <max threads are
> running, the server tried to create a new one, and the OS couldn't
> allocate a thread.
> 
> Any ideas how to resolve this? Version is FreeRadius 1.1.6 (only reason
> we haven't upgraded is change control, it's due shortly)

we recently had a similar issue after migrating to using FR for VMPS
handling. the system may deal with many hundreds of requests per second
as with VMPS the switch re-auths all ports at exactly the same time.
with 48 port switches this gets interesting. anyway. the issue was
that we had the following config

radiusd.conf  max_servers = X
experimental.conf - perl, max_clones = Y 

where X != Y

this is a big problem and you get the above mentioned errors. you ALSO
get the error when X = Y and the load/demand is very high. in this case
the radius thread appears to be trying to launch a new PERL instace
before the old ones have gone.  anyway, a rapid increase of the values
helped straight away.  as did a proper optimization of the DB to get
much much faster PERL code.

what you have to ensure in these cases is you dont see these ones:

Fri Feb 11 16:00:11 2006 : Error: Discarding duplicate request from client BLAH port 49464 - ID: 10313 due to unfinished request 6

as although seemingly okay the client isnt getting an answer from its requests.

alan