has proxying configuration changed from 2.2.6-2.2.8 ?
Hello, I am trying to track down an odd behaviour. I have set up a simple FreeRADIUS server as proxy load balancer for two back end FR servers. All are running 2.2.8. I can’t seem to get the proxy to notice when one of the two backends is shut down. It used to be (I was running 2.2.6 before upgrading to 2.2.8) that after a few seconds the proxy would notice that the backend was not responding and start sending the requests to the other server. As in the following (radiusd -X ) where we can see it marking the server as zombie and sending status-server requests: Listening on authentication address 172.20.20.246 port 1812 Listening on accounting address 172.20.20.246 port 1813 Listening on authentication address 172.20.20.245 port 1812 as server pf.cluster Listening on accounting address 172.20.20.245 port 1813 as server pf.cluster Listening on command file /var/run/radiusd/radiusd.sock Listening on authentication address 127.0.0.1 port 18120 as server inner-tunnel Listening on proxy address 172.20.20.246 port 1814 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 61315, id=176, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x4e8b8278c6926c4400d0c821ee79a136 server pf.cluster { # Executing section authorize from file /etc/raddb/sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 169 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313736 Proxying request 0 to home server 172.20.20.184 port 1812 Sending Access-Request of id 169 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313736 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 61315, id=176, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 169 Sending Access-Request of id 169 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313736 Waking up in 1.3 seconds. Cleaning up request 0 ID 176 with timestamp +13 Marking home server 172.20.20.184 port 1812 as zombie (it looks like it is dead). Sending Status-Server of id 212 to 172.20.20.184 port 1812 Message-Authenticator := 0x00000000000000000000000000000000 NAS-Identifier := "Status Check. Are you alive?" Whereas now in 2.2.8 I get the following where it never seems to notice the backend server has gone away: Listening on authentication address 172.20.20.246 port 1812 Listening on accounting address 172.20.20.246 port 1813 Listening on authentication address 172.20.20.245 port 1812 as server pf.cluster Listening on accounting address 172.20.20.245 port 1813 as server pf.cluster Listening on command file /var/run/radiusd/radiusd.sock Listening on authentication address 127.0.0.1 port 18120 as server inner-tunnel Listening on proxy address 172.20.20.246 port 1814 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=94, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xd591a8d2aa2abec4228666e36f21e764 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 226 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Proxying request 0 to home server 172.20.20.184 port 1812 Sending Access-Request of id 226 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=94, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 226 Sending Access-Request of id 226 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Waking up in 1.3 seconds. Cleaning up request 0 ID 94 with timestamp +7 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=94, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xd591a8d2aa2abec4228666e36f21e764 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 167 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Proxying request 1 to home server 172.20.20.184 port 1812 Sending Access-Request of id 167 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x3934 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=144, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xee899074a654c74b561a8a2096b9b869 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 133 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Proxying request 2 to home server 172.20.20.184 port 1812 Sending Access-Request of id 133 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Going to the next request Waking up in 0.6 seconds. Cleaning up request 1 ID 94 with timestamp +17 Waking up in 0.3 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=144, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 133 Sending Access-Request of id 133 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Waking up in 1.3 seconds. Cleaning up request 2 ID 144 with timestamp +22 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=144, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xee899074a654c74b561a8a2096b9b869 server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 191 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Proxying request 3 to home server 172.20.20.184 port 1812 Sending Access-Request of id 191 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313434 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=111, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x17409eb12db9ee58f23c08b0b5697fab server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 18 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Proxying request 4 to home server 172.20.20.184 port 1812 Sending Access-Request of id 18 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Going to the next request Waking up in 0.6 seconds. Cleaning up request 3 ID 144 with timestamp +32 Waking up in 0.3 seconds. Waking up in 4.9 seconds. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=111, length=82 Sending duplicate proxied request to home server 172.20.20.184 port 1812 - ID: 18 Sending Access-Request of id 18 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Waking up in 1.3 seconds. Cleaning up request 4 ID 111 with timestamp +37 Ready to process requests. rad_recv: Access-Request packet from host 172.20.20.14 port 63846, id=111, length=82 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x17409eb12db9ee58f23c08b0b5697fab server pf.cluster { # Executing section authorize from file /usr/local/etc/raddb//sites-enabled/pf-proxy +group authorize { ++update control { ++} # update control = noop +} # group authorize = noop } # server pf.cluster WARNING: Empty pre-proxy section. Using default return values. Sending Access-Request of id 93 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Proxying request 5 to home server 172.20.20.184 port 1812 Sending Access-Request of id 93 to 172.20.20.184 port 1812 User-Name = "aabbccddeeff" User-Password = "aabbccddeeff" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x00000000000000000000000000000000 Proxy-State = 0x313131 Going to the next request Waking up in 0.9 seconds. Waking up in 4.9 seconds. Both of these outputs are from the same server and the requests were generated using radclient. These are two stock configuration as much as I could make them. The 2.2.6 was installed from packages ( 2.2.6-4.el6 on CentOS 6) whereas the 2.2.8 was compiled from the latest tarball available on http://freeradius.org/download.html. All I did was ./configure && make and then make install. The only files I had changed were the radiusd.conf, clients.conf and a sites-enabled/pf-proxy file of which the contents follow. radiusd.conf and clients.conf are the same (I reused the same file). [ pf-proxy] # cat /usr/local/etc/raddb/sites-enabled/pf-proxy listen { ipaddr = 172.20.20.245 port = 0 type = auth virtual_server = pf.cluster } listen { ipaddr = 172.20.20.245 port = 0 type = acct virtual_server = pf.cluster } home_server pf0.cluster { type = auth+acct ipaddr = 172.20.20.245 port = 1812 secret = test1234 response_window = 6 status_check = status-server revive_interval = 120 check_interval = 30 num_answers_to_alive = 3 } home_server pf1.cluster { type = auth+acct ipaddr = 172.20.20.184 port = 1812 secret = test1234 response_window = 6 status_check = status-server revive_interval = 120 check_interval = 30 num_answers_to_alive = 3 } # Put all of the servers into a pool. home_server_pool pf_pool.cluster { type = client-port-balance home_server = pf0.cluster home_server = pf1.cluster } home_server_pool pfacct_pool.cluster { type = load-balance home_server = pf0.cluster home_server = pf1.cluster } realm packetfence { auth_pool = pf_pool.cluster acct_pool = pfacct_pool.cluster } server pf.cluster { pre-proxy { # Insert pre-proxy rules here } post-proxy { } authorize { update control { Proxy-To-Realm := "packetfence" } } authenticate { } accounting { update control { Proxy-To-Realm := "packetfence" } } } When running 2.2.6 I can clearly see the status-server requests received by the (shutdown) backend: Status-Server Id 212 172.20.20.246:1814 -> 172.20.20.184:1812 +560.025 Message-Authenticator = 0x530277e3be1d44304b4d8da644b5cf79 NAS-Identifier = "Status Check. Are you alive?" No such requests are received from 2.2.8. So my question is, has proxying configuration changed between 2.2.6 and 2.2.8 in some way that would explain this? I’d be happy to provide any additional information. Any help is appreciated. -- Louis Munro lmunro@inverse.ca :: www.inverse.ca +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (www.packetfence.org)
On Aug 13, 2015, at 11:13 PM, Louis Munro <lmunro@inverse.ca> wrote:
All are running 2.2.8.
I can’t seem to get the proxy to notice when one of the two backends is shut down.
It used to be (I was running 2.2.6 before upgrading to 2.2.8) that after a few seconds the proxy would notice that the backend was not responding and start sending the requests to the other server.
There may be a regression in 2.2.8. I've put some fixes in which should help. Please try the v2.x.x branch from github. Alan DeKok.
On Aug 14, 2015, at 5:01 , Alan DeKok <aland@deployingradius.com> wrote:
There may be a regression in 2.2.8. I've put some fixes in which should help. Please try the v2.x.x branch from github.
Yes. That seems to have done it. I now get status-server requests after a few answered requests. See radsniff output below. # radsniff Access-Request Id 72 172.20.20.14:63985 -> 172.20.20.245:1812 +0.000 User-Name = "aabbccddeeff" User-Password = "\365#3\343x\333]\306#\337n\203\361\206\027\235" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x54d0b48a7fe13ec820f86a6569889002 Access-Request Id 122 172.20.20.246:1814 -> 172.20.20.184:1812 +0.000 User-Name = "aabbccddeeff" User-Password = "\312\020\n,\023\3324\263\330\031\235 \253j\265'" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xa91110b861554dd78132297f0c6a19e3 Proxy-State = 0x3732 Access-Request Id 72 172.20.20.14:63985 -> 172.20.20.245:1812 +4.347 User-Name = "aabbccddeeff" User-Password = "\365#3\343x\333]\306#\337n\203\361\206\027\235" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0x54d0b48a7fe13ec820f86a6569889002 Access-Request Id 122 172.20.20.246:1814 -> 172.20.20.184:1812 +4.348 User-Name = "aabbccddeeff" User-Password = "\312\020\n,\023\3324\263\330\031\235 \253j\265'" NAS-IP-Address = 192.168.239.141 NAS-Port = 11 Message-Authenticator = 0xa91110b861554dd78132297f0c6a19e3 Proxy-State = 0x3732 Status-Server Id 143 172.20.20.246:1814 -> 172.20.20.184:1812 +6.002 Message-Authenticator = 0x8561c3bc9e1baaa1dc4b3d7660c5bc80 NAS-Identifier = "Status Check. Are you alive?” Thank you for your help. -- Louis Munro lmunro@inverse.ca :: www.inverse.ca +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (www.packetfence.org)
Worth pushing out a 2.2.9 to mop this up and try to avoid too many distributions still using 2.x taking 2.2.8 verbatim? Nick
On Aug 14, 2015, at 4:36 PM, Nick Lowe <nick.lowe@gmail.com> wrote:
Worth pushing out a 2.2.9 to mop this up and try to avoid too many distributions still using 2.x taking 2.2.8 verbatim?
I'll be pushing out a 2.2.9 next week. There are one or two other issues which need addressing. Rare, but they need fixing. Then, so far as I can tell, 2.2.9 will be the last release of version 2. Alan DeKok.
Hi, Just wondering when 2.2.9 with the proxying (and other) bugfixes will be released. Regards J. On Fri, Aug 14, 2015 at 6:23 PM, Alan DeKok <aland@deployingradius.com> wrote:
On Aug 14, 2015, at 4:36 PM, Nick Lowe <nick.lowe@gmail.com> wrote:
Worth pushing out a 2.2.9 to mop this up and try to avoid too many distributions still using 2.x taking 2.2.8 verbatim?
I'll be pushing out a 2.2.9 next week. There are one or two other issues which need addressing. Rare, but they need fixing.
Then, so far as I can tell, 2.2.9 will be the last release of version 2.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
participants (4)
-
Alan DeKok -
Jonathan -
Louis Munro -
Nick Lowe