3.0.11 - all threads blocked in "linelog"

[Alan DeKok]

On Jun 22, 2016, at 9:34 AM, Chaigneau, Nicolas <nicolas.chaigneau at capgemini.com> wrote:
> We've installed on production yesterday FreeRADIUS version 3.0.11 (upgraded from 3.0.8).
> 
> Today we've noticed that the server got blocked.
> All the threads got blocked on module "linelog", which we can see in the logs:
> 
> Wed Jun 22 11:57:11 2016 : Error: (1774170) Ignoring duplicate packet from client *** port 21687 - ID: 186 due to unfinished request in component post-auth module linelog

  That issue was fixed in the v3.0.x branch already.

> So I'm suspecting this is related to FreeRADIUS.

  Yes.

> I've noticed the following commit related to locking (which is configurable for detail, but not linelog):
> 
> https://github.com/FreeRADIUS/freeradius-server/commit/dd2a06aa6ba8d6b819712a5008213d3d28375158
> 
> Note that I didn't see anything wrong with the code, just noticed the "locking" part which made me suspicious... maybe I'm wrong, but I'd like to get your opinion.

  It's related.

> Could the locking be responsible for the behaviour we've observed ?
> Should we patch our linelog to force "locking" to false ?

  That may help.

> Also, I think FreeRADIUS threads should never get blocked forever when trying to acquire a lock. (if that's what is happening)

  While I agree, the problem is that it's difficult to know whether a thread is blocked due to a lock, or just busy.  And you can't just signal a thread which is in a lock, the code will just retry the lock.  And you can't cancel the thread, because it then leaks all memory which the thread was using.

  You can't just say "software should work perfectly no matter what".  Things go wrong, and sometimes they just can't be fixed.

  Alan DeKok.