rlm_perl does not honor start_servers?

[Igor Novgorodov]

On 01.04.2016 17:00, Alan DeKok wrote:
> On Apr 1, 2016, at 9:53 AM, Igor Novgorodov <igor at novg.net> wrote:
>> I've got latest 3.0.11 FreeRADIUS and Perl authorization module under rlm_perl.
>> The server is relatively highly loaded (around 1000 access-requests per sec).
>>
>> And although i have "start_servers = 256" and "min_spare_servers = 256" in the thread pool
>> configuration, FreeRADIUS does not initialize those threads until the requests start to hit the daemon.
>    FreeRADIUS starts all 256 threads.  The "spare" threads are created as needed.
>
>> After some requests in parallel it spawns all required 256 threads and then this number stays constant.
>>
>> The problem is that the thread creation in rlm_perl seems to be quite expensive timewise (or i may be doing something wrong?).
>    Thread creation in Perl is very different than thread creation in the server core.  The server starts 256 threads.  BUT the Perl module still has to initialize any thread-specific context.  That doesn't happen until the module is called.
Ok, so there's no way to make it to initialize Perl's thread stuff when 
the server core threads are spawned?

>> It is clearly seen by testing with radclient - after daemon restart it is capable of only 30 req/s.
>> Then, when all the threads come alive - it goes up to 1k req/s and more.
>    That sounds about right.
>
>> So unless i warm up the server with a few thousand test queries to create all threads - it gets
>> overwhelmed for several minutes when the real load shows up, which causes a lot of warnings
>> in the logs and massive retransmits from NASes (there are about 900 of them with lots of clients each),
>> which is nasty.
>    Having the server initialize Perl 256 times won't help.  It will just push the delay *slightly* earlier.
Well, it does really help, actually.
After daemon startup, i'm doing "radclient -q -r 1 -t 10 -p 64 -s -f 
radius.256.test localhost auth secret" several times and then direct the 
load on the server.
In this case it works fine, as the threads are already there to process 
the requests.
>
>    i.e. the server will either block and do nothing for a long time while it's initializing 256 Perl threads, or it will slowly process packets while it's initializing the Perl threads.
>
>    Either choice isn't perfect.  And there's no real way around it.  If Perl is slow to initialize... then it's slow to initialize.  There's no magic you can do to make it faster.
I don't want magic, i want Perl threads to initialize automatically 
after radiusd startup, not after the queries come in.

>
>> So the question is - this behavior is expected? I can, of course, warm up server after each restart but
>> it seems irrational for me.
>    Then you don't understand how computers work.
Hmm.. i can object, but i won't :)

>
>    My guess here is that the problem isn't starting up 256 Perl threads, the problem is that your Perl code is tens of thousands of lines long.  And compiling *that* is taking forever.
>
>    So the solution is to not use complex Perl scripts.
My script is only 470 lines long, but it uses DBI module (and some minor 
others). Maybe it's quite complex and causing the slowdown.

>
>    Alan DeKok.