Hello, I am trying to track down an odd behaviour. I have set up a simple FreeRADIUS server as proxy load balancer for two back end FR servers. All are running 2.2.8. I can’t seem to get the proxy to notice when one of the two backends is shut down. It used to be (I was running 2.2.6 before upgrading to 2.2.8) that after a few seconds the proxy would notice that the backend was not responding and start sending the requests to the other server. As in the following (radiusd -X ) where we can see it marking the server as zombie and sending status-server requests: Listening on authentication address 172.20.20.246 port 1812 Listening on accounting address 172.20.20.246 port 1813 Listening on authentication address 172.20.20.245 port 1812 as server pf.cluster Listening on accounting address 172.20.20.245 port 1813 as server pf.cluster Listening on command file /var/run/radiusd/radiusd.sock Listening on authentication address 127.0.0.1 port 18120 as server inner-tunnel Listening on proxy address 172.20.20.246 port 1814 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 61315, id=176, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x4e8b8278c6926c4400d0c821ee79a136 server pf.cluster { # Executing section authorize from file /etc/raddb/sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 169 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313736 Proxying request 0 to home server 172.20.20.184 port 1812 Sending Access-Request of id 169 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313736 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 61315, id=176, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 169 Sending Access-Request of id 169 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313736 Waking up in 1.3 seconds. Cleaning up request 0 ID 176 with timestamp +13 Marking home server 172.20.20.184 port 1812 as zombie (it looks like it is dead). Sending Status-Server of id 212 to 172.20.20.184 port 1812 Message-Authenticator := 0x00000000000000000000000000000000 NAS-Identifier := "Status Check. Are you alive?" Whereas now in 2.2.8 I get the following where it never seems to notice the backend server has gone away: Listening on authentication address 172.20.20.246 port 1812 Listening on accounting address 172.20.20.246 port 1813 Listening on authentication address 172.20.20.245 port 1812 as server pf.cluster Listening on accounting address 172.20.20.245 port 1813 as server pf.cluster Listening on command file /var/run/radiusd/radiusd.sock Listening on authentication address 127.0.0.1 port 18120 as server inner-tunnel Listening on proxy address 172.20.20.246 port 1814 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=94, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xd591a8d2aa2abec4228666e36f21e764 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 226 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Proxying request 0 to home server 172.20.20.184 port 1812 Sending Access-Request of id 226 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=94, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 226 Sending Access-Request of id 226 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Waking up in 1.3 seconds. Cleaning up request 0 ID 94 with timestamp +7 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=94, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xd591a8d2aa2abec4228666e36f21e764 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 167 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Proxying request 1 to home server 172.20.20.184 port 1812 Sending Access-Request of id 167 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=144, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xee899074a654c74b561a8a2096b9b869 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 133 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Proxying request 2 to home server 172.20.20.184 port 1812 Sending Access-Request of id 133 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Going to the next request Waking up in 0.6 seconds. Cleaning up request 1 ID 94 with timestamp +17 Waking up in 0.3 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=144, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 133 Sending Access-Request of id 133 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Waking up in 1.3 seconds. Cleaning up request 2 ID 144 with timestamp +22 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=144, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xee899074a654c74b561a8a2096b9b869 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 191 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Proxying request 3 to home server 172.20.20.184 port 1812 Sending Access-Request of id 191 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=111, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x17409eb12db9ee58f23c08b0b5697fab server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 18 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Proxying request 4 to home server 172.20.20.184 port 1812 Sending Access-Request of id 18 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Going to the next request Waking up in 0.6 seconds. Cleaning up request 3 ID 144 with timestamp +32 Waking up in 0.3 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=111, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 18 Sending Access-Request of id 18 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Waking up in 1.3 seconds. Cleaning up request 4 ID 111 with timestamp +37 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=111, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x17409eb12db9ee58f23c08b0b5697fab server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 93 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Proxying request 5 to home server 172.20.20.184 port 1812 Sending Access-Request of id 93 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. Both of these outputs are from the same server and the requests were generated using radclient. These are two stock configuration as much as I could make them. The 2.2.6 was installed from packages ( 2.2.6-4.el6 on CentOS 6) whereas the 2.2.8 was compiled from the latest tarball available on http://freeradius.org/download.html. All I did was ./configure && make and then make install. The only files I had changed were the radiusd.conf, clients.conf and a sites-enabled/pf-proxy file of which the contents follow. radiusd.conf and clients.conf are the same (I reused the same file). [ pf-proxy] # cat /usr/local/etc/raddb/sites-enabled/pf-proxy listen { ipaddr = 172.20.20.245 port = 0 type = auth virtual_server = pf.cluster } listen { ipaddr = 172.20.20.245 port = 0 type = acct virtual_server = pf.cluster } home_server pf0.cluster { type = auth+acct ipaddr = 172.20.20.245 port = 1812 secret = test1234 response_window = 6 status_check = status-server revive_interval = 120 check_interval = 30 num_answers_to_alive = 3 } home_server pf1.cluster { type = auth+acct ipaddr = 172.20.20.184 port = 1812 secret = test1234 response_window = 6 status_check = status-server revive_interval = 120 check_interval = 30 num_answers_to_alive = 3 } # Put all of the servers into a pool. home_server_pool pf_pool.cluster { type = client-port-balance home_server = pf0.cluster home_server = pf1.cluster } home_server_pool pfacct_pool.cluster { type = load-balance home_server = pf0.cluster home_server = pf1.cluster } realm packetfence { auth_pool = pf_pool.cluster acct_pool = pfacct_pool.cluster } server pf.cluster { pre-proxy { # Insert pre-proxy rules here } post-proxy { } authorize { update control { Proxy-To-Realm := "packetfence" } } authenticate { } accounting { update control { Proxy-To-Realm := "packetfence" } } } When running 2.2.6 I can clearly see the status-server requests received by the (shutdown) backend: Status-Server Id 212 172.20.20.246:1814 -> 172.20.20.184:1812 +560.025 Message-Authenticator = 0x530277e3be1d44304b4d8da644b5cf79 NAS-Identifier = "Status Check. Are you alive?" No such requests are received from 2.2.8. So my question is, has proxying configuration changed between 2.2.6 and 2.2.8 in some way that would explain this? I’d be happy to provide any additional information. Any help is appreciated. -- Louis Munro lmunro@inverse.ca :: www.inverse.ca +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (www.packetfence.org)