Thanks for the information. I have made some progress on this. Now it appears that when I start freeradius the load balancing works as expected for a short time, up to 10 minutes. Then the home servers are marked as zombies and then as dead.
The clients are multiple Cisco switches and the production RADIUS servers are 3 Cisco ACS RADIUS servers.
As suggested I have added the status-server for the home servers in the proxy.conf.
proxy server {
default_fallback = no
}
home_server serverxnmx0007 {
type = auth+acct
ipaddr = 192.168.254.150
port = 1812
secret = secret
response_window = 45
response_timeouts = 5
status_check = status-server
username = "testuser"
password = "pass"
}
home_server serverxnmx0011 {
type = auth+acct
ipaddr = 192.168.254.152
port = 1812
secret = secret
response_window = 45
response_timeouts = 5
status_check = status-server
username = "testuser"
password = "pass"
}
home_server serverxnmx0012 {
type = auth+acct
ipaddr = 192.168.254.153
port = 1812
secret = secret
response_window = 45
response_timeouts = 5
status_check = status-server
username = "testuser"
password = "pass"
}
The radius.log file shows the servers being marked as zombie and then dead. The log below shows one server being marked dead and after some time another one will be marked the same.
Thu Sep 11 16:47:17 2014 : Info: Loaded virtual server <default>
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 41379
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 47256
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 60777
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 37802
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 51392
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 37221
Thu Sep 11 16:47:18 2014 : Info: ... adding new socket proxy address * port 53560
Thu Sep 11 16:47:18 2014 : Info: Ready to process requests.
Thu Sep 11 16:53:12 2014 : Auth: Login OK: [host/HOSTXWKX0050.foo/<no User-Password attribute>] (from client 2150a_ port 36 cli 00-24-1D-AB-CA-83)
Thu Sep 11 16:54:53 2014 : Auth: Login OK: [host/HOSTXWKX0116.foo/<no User-Password attribute>] (from client 2030c_ port 5 cli 00-24-1D-AC-B4-8B)
Thu Sep 11 16:54:56 2014 : Error: Discarding duplicate request from client 2030c_ port 1645 - ID: 88 due to unfinished request 15
Thu Sep 11 16:54:59 2014 : Error: Discarding duplicate request from client 2030c_ port 1645 - ID: 88 due to unfinished request 15
Thu Sep 11 16:55:11 2014 : Auth: Login OK: [host/HOSTXWKX0116.foo/<no User-Password attribute>] (from client 2030c_ port 5 cli 00-24-1D-AC-B4-8B)
Thu Sep 11 16:56:31 2014 : Auth: Login incorrect (Home Server says so): [NA\\SVC./<no User-Password attribute>] (from client 2030c_ port 10 cli 00-24-1D-AD-FF-DF)
Thu Sep 11 17:02:30 2014 : Auth: Login OK: [host/HOSTXWKX0267.foo/<no User-Password attribute>] (from client 2030c_ port 38 cli 90-B1-1C-72-C2-AC)
Thu Sep 11 17:02:48 2014 : Auth: Login OK: [host/HOSTXWKX0105.foo/<no User-Password attribute>] (from client 2030c_ port 17 cli 00-24-1D-AD-FE-AF)
Thu Sep 11 17:02:49 2014 : Auth: Login OK: [host/HOSTXWKX0107.foo/<no User-Password attribute>] (from client 2030c_ port 21 cli 00-24-1D-AD-F8-8B)
Thu Sep 11 17:02:49 2014 : Auth: Login OK: [host/HOSTXWKX0111.foo/<no User-Password attribute>] (from client 2030c_ port 23 cli 00-24-1D-AD-FE-8E)
Thu Sep 11 17:02:52 2014 : Auth: Login OK: [host/HOSTXWKX0103.foo/<no User-Password attribute>] (from client 2030c_ port 15 cli 00-24-1D-AD-F8-99)
Thu Sep 11 17:03:06 2014 : Auth: Login OK: [host/HOSTXWKX0104.foo/<no User-Password attribute>] (from client 2030c_ port 16 cli 00-24-1D-AD-FF-01)
Thu Sep 11 17:03:08 2014 : Auth: Login OK: [host/HOSTXWKX0102.foo/<no User-Password attribute>] (from client 2030c_ port 19 cli 00-24-1D-AD-F9-A3)
Thu Sep 11 17:03:15 2014 : Auth: Login OK: [host/HOSTXWKX0101.foo/<no User-Password attribute>] (from client 2030c_ port 18 cli 00-24-1D-AD-F9-21)
Thu Sep 11 17:07:00 2014 : Proxy: Marking home server 192.168.254.152 port 1812 as zombie (it looks like it is dead).
Thu Sep 11 17:07:04 2014 : Error: No response to status check 132 for home server 192.168.254.152 port 1812
Thu Sep 11 17:07:04 2014 : Proxy: Marking home server 192.168.254.152 port 1812 as dead.
Thu Sep 11 17:07:34 2014 : Error: No response to status check 133 for home server 192.168.254.152 port 1812
Thu Sep 11 17:08:08 2014 : Error: No response to status check 134 for home server 192.168.254.152 port 1812
Thu Sep 11 17:08:42 2014 : Error: No response to status check 135 for home server 192.168.254.152 port 1812
During this time I am able to use radtest to get a response from all of the servers:
server (root@serverxb6x5121 radius)# radtest testuser pass 192.168.254.150 10 secret
Sending Access-Request of id 116 to 192.168.254.150 port 1812
User-Name = "testuser"
User-Password = "pass"
NAS-IP-Address = 192.168.105.34
NAS-Port = 10
Message-Authenticator = 0x00000000000000000000000000000000
rad_recv: Access-Reject packet from host 192.168.254.150 port 1812, id=116, length=38
Message-Authenticator = 0xd1ba4f1a6028b42436dd2f5ebea322b9
server (root@serverxb6x5121 radius)# radtest testuser pass 192.168.254.152 10 secret
Sending Access-Request of id 216 to 192.168.254.152 port 1812
User-Name = "testuser"
User-Password = "pass"
NAS-IP-Address = 192.168.105.34
NAS-Port = 10
Message-Authenticator = 0x00000000000000000000000000000000
rad_recv: Access-Reject packet from host 192.168.254.152 port 1812, id=216, length=38
Message-Authenticator = 0x2872a2f5177592f69d007406a4a89999
server (root@serverxb6x5121 radius)# radtest testuser pass 192.168.254.153 10 secret
Sending Access-Request of id 124 to 192.168.254.153 port 1812
User-Name = "testuser"
User-Password = "pass"
NAS-IP-Address = 192.168.105.34
NAS-Port = 10
Message-Authenticator = 0x00000000000000000000000000000000
rad_recv: Access-Reject packet from host 192.168.254.153 port 1812, id=124, length=38
Message-Authenticator = 0xef0ac9fdee194cd6a455b3bf0574d0a
I have read the documentation and am unable to determine why the servers are being marked as dead. Am I missing something in my configuration or are the ACS servers discarding packets and that is causing the zobie/dead state?
If more logs are needed please let me know. Any help would be greatly appreciated.
Thanks
Neil