rlm_perl does not honor start_servers?

[Alan DeKok]

On Apr 1, 2016, at 9:53 AM, Igor Novgorodov <igor at novg.net> wrote:
> 
> I've got latest 3.0.11 FreeRADIUS and Perl authorization module under rlm_perl.
> The server is relatively highly loaded (around 1000 access-requests per sec).
> 
> And although i have "start_servers = 256" and "min_spare_servers = 256" in the thread pool
> configuration, FreeRADIUS does not initialize those threads until the requests start to hit the daemon.

  FreeRADIUS starts all 256 threads.  The "spare" threads are created as needed.

> After some requests in parallel it spawns all required 256 threads and then this number stays constant.
> 
> The problem is that the thread creation in rlm_perl seems to be quite expensive timewise (or i may be doing something wrong?).

  Thread creation in Perl is very different than thread creation in the server core.  The server starts 256 threads.  BUT the Perl module still has to initialize any thread-specific context.  That doesn't happen until the module is called.

> It is clearly seen by testing with radclient - after daemon restart it is capable of only 30 req/s.
> Then, when all the threads come alive - it goes up to 1k req/s and more.

  That sounds about right.

> So unless i warm up the server with a few thousand test queries to create all threads - it gets
> overwhelmed for several minutes when the real load shows up, which causes a lot of warnings
> in the logs and massive retransmits from NASes (there are about 900 of them with lots of clients each),
> which is nasty.

  Having the server initialize Perl 256 times won't help.  It will just push the delay *slightly* earlier.

  i.e. the server will either block and do nothing for a long time while it's initializing 256 Perl threads, or it will slowly process packets while it's initializing the Perl threads.

  Either choice isn't perfect.  And there's no real way around it.  If Perl is slow to initialize... then it's slow to initialize.  There's no magic you can do to make it faster.

> So the question is - this behavior is expected? I can, of course, warm up server after each restart but
> it seems irrational for me.

  Then you don't understand how computers work.

  My guess here is that the problem isn't starting up 256 Perl threads, the problem is that your Perl code is tens of thousands of lines long.  And compiling *that* is taking forever.

  So the solution is to not use complex Perl scripts.

  Alan DeKok.