FreeRADIUS Thread Behavior

[Alan DeKok]

On Oct 16, 2017, at 4:07 PM, Doug Wussler <doug.wussler at fsu.edu> wrote:
> 
> Doug says:
> - The question I am trying to answer is why the threads turn over at all, and in particular
> so frequently.  Here is my config for the settings I thought control the thread behaviour:

  Re-posting the configuration doesn't help.  I've already seen it.

  Re-posting the whole back & forth is also not useful.

> Doug responds:
> Yes, I agree the doc is clear.  That is my point, I do not see my system behaving like the doc describes, or
> at least how one would expect it to behave.  Further testing indicates the thread testing loop behaves like this:
>                Check if idle threads are less than min.
>                     Always find fewer than min spare threads so spawn more idle threads.
>              Check to see if idle threads are more than max.
>                    Always find more idle threads than max so kill oldest threads.
>              Rest
>             Return to beginning.

  That doesn't happen.

  If it spawns new threads, it doesn't delete idle threads within the same second.

> One iteration of the loop happens so fast that, if you look, you almost never see the newly spawned spare threads along
> with the full set of threads that were just tested for idleness.  Right after the new threads are spawned,
> older ones are terminated.  Not only are threads being unintentionally retired and spawned, the LDAP
> and SQL connections are lost and must be restarted too.

  The checking is done once per second.  So that may be "fast" for a person.  It's not fast for a computer.

> So, I’m not complaining, I just thought you would want to know.  As you say, you could set:
> 
>     start_servers = n
>     max_servers = n
>     max_spare_servers = n
>     min_spare_servers = 0
> 
> and that will put a stop to the unwanted dying and spawning of threads.  This seems like an approach to
> be vastly preferred over the way the current non-zero spare thread implementation is working, even if it wastes
> some resources.

  You might be aware that RADIUS traffic is bursty.  i.e. it gets periodic bursts of traffic, even within the same second.  You might also be aware that it takes less than a second to process a request.  So one thread can go from busy to idle, multiple times in a second.

  i.e. your assumptions about the current implementation are based on a faulty understanding of how the system works.

> However, to minimize wasted resources while ensuring peak demands are met,
> one first needs to determine a “good” number for n for their particular server, and, more
> importantly, it remains to be seen if threads that never die are subject to some sort of memory leakage
> or data-structure corruption between server reboots (which for us is daily at log-rotate time).

  The server doesn't leak memory or have resource corruption.  We work very hard on that.

  It's nice that you're trying to educate me as to how to design a RADIUS server.  But, be aware that I do have *some* clue as to (a) how the server works, and (b) how the server *should* work.

  If you don't want the server create new "spare" threads, or deleting old "spare" threads, then follow my  previous suggestion.  Don't argue about how it actually works.  Especially if you get that part wrong.

  Alan DeKok.