Hi Alan, I have below config in sql mod. pool { min = 50 max = ${thread[pool].max_servers} spare = 10 uses = 0 lifetime = 0 idle_timeout = 60 } I have below config in radiusd.conf max_request_time = 30 cleanup_delay = 10 max_requests = 81920 hostname_lookups = no thread pool { start_servers = 64 max_servers = 256 min_spare_servers = 50 max_spare_servers = 100 max_requests_per_server = 300 auto_limit_acct = yes } Sometime I get these messages. Wed Oct 14 04:30:00 2020 : Warning: rlm_sql (sql): 60 of 60 connections in use. You probably need to increase "spare" Wed Oct 14 04:30:00 2020 : Error: rlm_sql (sql): Cannot open new connection, connection spawning already in progress Wed Oct 14 04:30:00 2020 : Warning: rlm_sql (sql): 60 of 60 connections in use. You probably need to increase "spare" Wed Oct 14 04:30:00 2020 : Error: rlm_sql (sql): Cannot open new connection, connection spawning already in progress Wed Oct 14 04:30:00 2020 : Warning: rlm_sql (sql): 60 of 60 connections in use. You probably need to increase "spare" Wed Oct 14 04:30:00 2020 : Error: rlm_sql (sql): Cannot open new connection, connection spawning already in progress Wed Oct 14 04:30:00 2020 : Warning: rlm_sql (sql): 60 of 60 connections in use. You probably need to increase "spare" Wed Oct 14 04:30:00 2020 : Error: rlm_sql (sql): Cannot open new connection, connection spawning already in progress Wed Oct 14 04:30:00 2020 : Warning: rlm_sql (sql): 61 of 61 connections in use. You probably need to increase "spare" And these Wed Oct 14 04:29:53 2020 : Info: rlm_sql (sql): Opening additional connection (209059) Wed Oct 14 04:29:54 2020 : Info: rlm_sql (sql): 0 of 44 connections in use. Need more spares Wed Oct 14 04:29:54 2020 : Info: rlm_sql (sql): Opening additional connection (209060) Wed Oct 14 04:29:55 2020 : Info: rlm_sql (sql): 6 of 45 connections in use. Need more spares Wed Oct 14 04:29:55 2020 : Info: rlm_sql (sql): Opening additional connection (209061) Wed Oct 14 04:29:55 2020 : Info: rlm_sql (sql): Closing connection (209028): Hit idle_timeout, was idle for 61 seconds Wed Oct 14 04:29:55 2020 : Warning: rlm_sql (sql): You probably need to lower "min" Wed Oct 14 04:29:55 2020 : Info: rlm_sql (sql): Closing connection (209030): Hit idle_timeout, was idle for 61 seconds Wed Oct 14 04:29:55 2020 : Info: rlm_sql (sql): Closing connection (209022): Hit idle_timeout, was idle for 61 seconds Wed Oct 14 04:29:55 2020 : Info: rlm_sql (sql): Closing connection (209023): Hit idle_timeout, was idle for 61 seconds I can see at DB that connections are connected from freeradius. zero connection might means that its currently no connection is being used. But below errors are continuous. Wed Oct 14 14:00:23 2020 : Error: (34868042) Ignoring duplicate packet from client BRAS port 50044 - ID: 9 due to unfinished request in component <core> module Wed Oct 14 14:00:23 2020 : Error: (34868065) Ignoring duplicate packet from client BRAS port 56885 - ID: 56 due to unfinished request in component <core> module I have added below tweaking in sysctl.conf for TCP and UDP tweaking. # allow testing with buffers up to 128MB net.core.rmem_max = 134217728 net.core.wmem_max = 134217728 # increase the length of the processor input queue net.core.netdev_max_backlog = 250000 # increase Linux autotuning TCP buffer limit to 64MB net.ipv4.tcp_rmem = 4096 87380 67108864 net.ipv4.tcp_wmem = 4096 65536 67108864 # recommended default congestion control is htcp net.ipv4.tcp_congestion_control=htcp net.ipv4.udp_rmem_min=131072 net.ipv4.udp_wmem_min=131072 I have checked by turning off the firewall on host. I have checked by tcpdump for 10mins. Approx 75% of auth request and 78% of accounting requests are being responded by freeradius. I have tuned mysql to its optimal level as well but didn't fixed the problem. The system have 200k connected users with 30mins of interm update. Need your advise how to troubleshoot this problem further as the DB and freeradius are on same node (duel 2640v4 processors + 256GB RAM + 800GB SSD) Ammad Ali On 10/13/2020 6:21 PM, Alan DeKok wrote:
On Oct 13, 2020, at 9:03 AM, Ammad Ali <ammad.ali@rapidcompute.com> wrote:
My database is working fine. I am running 4 radius servers using same database. But surprisingly the radius process running same node with the database I am getting the below errors alot ignoring duplicate errors. *Something* is blocking the RADIUS server. And *something* is making it have *zero* connections to the database.
It's really simple. I've been doing this for about 20 years. I've seen those errors well over a thousand times at this point. It's almost always something *external* blocking the RADIUS server.
So you can believe me and fix the problem. Or, believe that your system is somehow unique, and you've run into an issue never seen before.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html