anyone know what actually causes this? "FATAL! Server is too busyto process requests"

[Tony Redstone]

Hi Michael,

On 5/18/06, Mitchell, Michael J <Michael.Mitchell at team.telstra.com> wrote:
> Hi Tony,
>
> I've run into this problem in the past. What version of freeRADIUS are
> you running?

until recently we were running 1.0.5 but we've just upgraded to 1.1.1
and we see the same issue (which doesn't surprise me since the
relevant threads.c code doesn't appear to have changed).

> Like you I found that it appears more often when proxying requests to a
> home server - I guess the reqeusts sit in the queue longer waiting for a
> reply.
>
> Alan was kind enough to supply a patch within hours for 1.1.0 and later.
> It was bug #331.

looks quite relevant, I'm going to try to understand the patch and
then perhaps apply the patch on one of our servers to see if it helps.

> However, we are still using 1.0.1, which couldn't take advantange of
> that patch. I had to make some small bandaid changes to request_dequeue,
> which reduced the change of this happening, at the expense of dropping
> some requests. Let me know if you need those patches.

this also sounds interesting, under extreme conditions, when, in our
case, BT dropped around 30k users simultaneously during one
maintenance window (SDH MUX software changes), we had to resort to
forcibly killing and restarting our freeradius proxies every 5s in
order to get users back online.  it would handle requests for the
first 2-3s after being started and then pegged at 100% cpu and served
no further requests.  any patches which touch the request load are of
definite interest to us.  I would appreciate it if you could forward
them to me?

Tony

>
>
> regards,
> Mike
>
>
>
> > -----Original Message-----
> > From:
> > freeradius-users-bounces+michael.mitchell=team.telstra.com at lis
> > ts.freeradius.org
> > [mailto:freeradius-users-bounces+michael.mitchell=team.telstra
> > .com at lists.freeradius.org] On Behalf Of Tony Redstone
> > Sent: Friday, 19 May 2006 12:03 AM
> > To: freeradius-users at lists.freeradius.org
> > Subject: anyone know what actually causes this? "FATAL!
> > Server is too busyto process requests"
> >
> > we occasionally get these errors in our logs:
> > Thu May 18 09:31:05 2006 : Error: FATAL!  Server is too busy
> > to process requests
> >
> > and the server dies.  I've found the core in src/main/threads.c that
> > spits out this message but it's not clear to me under what
> > circumstances this would/should happen.  It happens on several boxes
> > ranging from redhat 8.0 to fc4, some processes are talking to backend
> > SQL databases some aren't (in fact, it's our plain radius
> > passthrough/routing proxy servers with file only based config that
> > seem to experience this issue the most).
> >
> > does anyone know what really causes this and how to fix it?   we're
> > serving around 50k DSL users with a nominal authentication load of
> > around 200-300 requests per minute (this is for authentication only,
> > accounting is handled by separate processes which don't experience the
> > same problem).  peak load can run into several thousands per minute
> > when recovering from planned maintenance on part of the DSL network.
> >
> > Tony
> >
> > -
> > List info/subscribe/unsubscribe? See
> > http://www.freeradius.org/list/users.html
> >
>
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
>