rlm_perl with huge load (~1000-1500 request per minute)
Hello! i use freeradiusd 1.7, rlm_perl with thread conf: thread pool { start_servers = 100 max_servers = 1500 min_spare_servers = 1 max_spare_servers = 10 max_requests_per_server = 10 } rlm_sql uses mysql. All works fine, but sometime, at moment with high load: Error: Discarding duplicate request from client nas_4:63429 - ID: 154 due to unfinished request 2176280 and radiusd like "frozen", take 99% of CPU time, and not response for client's request. FreeBSD 6.3, Hardware is adequate - 2xXeon with 2 Gb Ram. any ideas? Tnx. -- WBR, John mailto:test@bi.com.ua
John S. Doe wrote:
i use freeradiusd 1.7, rlm_perl with thread conf:
thread pool { ... max_requests_per_server = 10
This should always be zero.
and radiusd like "frozen", take 99% of CPU time, and not response for client's request.
Run it in debugging mode (-fxxx for threading) to see what's going on. Alan DeKok.
Hi,
Hello!
i use freeradiusd 1.7, rlm_perl with thread conf:
thread pool { start_servers = 100 max_servers = 1500 min_spare_servers = 1 max_spare_servers = 10 max_requests_per_server = 10 }
rlm_sql uses mysql.
All works fine, but sometime, at moment with high load:
Error: Discarding duplicate request from client nas_4:63429 - ID: 154 due to unfinished request 2176280
and radiusd like "frozen", take 99% of CPU time, and not response for client's request.
FreeBSD 6.3, Hardware is adequate - 2xXeon with 2 Gb Ram.
any ideas?
yep - your database cant keep up with either the authentication traffic, the accounting traffic or both. try using sql_log for the accounting - to take it out of 100% realtime for each incoming packet. also your config looks a bit wonky.
thread pool { start_servers = 100 max_servers = 1500 min_spare_servers = 1 max_spare_servers = 10 max_requests_per_server = 10 }
start servers = 100 max spare servers = 10 so the first thing it has to do it kill a load off(!) - set the spare to 100. how many SQL threads are you running? alan
John S. Doe wrote:
Hello!
i use freeradiusd 1.7, rlm_perl with thread conf:
thread pool { start_servers = 100 max_servers = 1500 min_spare_servers = 1 max_spare_servers = 10 max_requests_per_server = 10 }
rlm_sql uses mysql.
All works fine, but sometime, at moment with high load:
Error: Discarding duplicate request from client nas_4:63429 - ID: 154 due to unfinished request 2176280
and radiusd like "frozen", take 99% of CPU time, and not response for client's request.
FreeBSD 6.3, Hardware is adequate - 2xXeon with 2 Gb Ram.
Yikes. You're telling it to start 100 threads, and maybe start 1500 threads, but to start closing down threads when there are 1-10 inactive. You're also telling it to close each thread after it's done 10 requests, rather than just running forever. That is a silly config. The defaults are a long way from that, for a good reason. Set "max_requests_per_server = 0" and max_servers no greater than double start_servers (at a guess - certainly 1500 is a stupidly large number, your OS will never, every successfully schedule 1500 contending threads) As for why it's freezing; it could be related to the thread config, but more likely your perl module (or another module) is simply taking too long to respond. Spawning more threads won't help that - you need to solve why the module is responding slowly. What's the full config? What other modules are you running, any database lookups? Why did you change the thread config to something so extreme? How many NASes do you have? What's your load like?
any ideas?
Tnx.
participants (4)
-
A.L.M.Buxey@lboro.ac.uk -
Alan DeKok -
John S. Doe -
Phil Mayers