I managed to find an hour or two in between various things. I've taken a look at the proxy code, and added some fixes in "master". To be polite, I didn't quite understand the previous code. So... I've deleted things, and simplified other things. The result is that the "zombie" home server detection should be better. I don't know if it fixes the "crash on dead home server" issue that people were seeing. I wasn't able to reproduce that. But, the relevant code is now different. So I hope it's fixed. As always, blame Arran for any bugs. :) Alan DeKok.
On 27/10/12 12:49, Alan DeKok wrote:
I managed to find an hour or two in between various things. I've taken a look at the proxy code, and added some fixes in "master".
To be polite, I didn't quite understand the previous code. So... I've deleted things, and simplified other things. The result is that the "zombie" home server detection should be better.
I don't know if it fixes the "crash on dead home server" issue that people were seeing. I wasn't able to reproduce that. But, the relevant code is now different. So I hope it's fixed.
As always, blame Arran for any bugs. :)
Just a quick note - I'm not sure the home server "status" is working properly in the new code; I'm getting: # radmin -d /etc/raddb3 -n eduroam -e 'show home_server list' 194.82.174.185 1812 udp auth unknown 0 194.82.174.185 1813 udp acct unknown 0 194.83.56.233 1812 udp auth unknown 0 194.83.56.233 1813 udp acct unknown 0 194.83.56.249 1812 udp auth unknown 0 194.83.56.249 1813 udp acct unknown 0 ...even though the servers are up and responding to packets. Same over the RADIUS-based stats interface (minor related note - the "FreeRADIUS-Stats-Server-State" attribute is missing a definition for value "3" i.e. "Unknown") I think it might be the logic in process.c:remove_from_proxy_hash_nl - that function seems to set the state to "UNKNOWN" if there are zero outstanding packets, which I guess is not right?
On 7 Jan 2013, at 17:59, Phil Mayers <p.mayers@imperial.ac.uk> wrote:
On 27/10/12 12:49, Alan DeKok wrote:
I managed to find an hour or two in between various things. I've taken a look at the proxy code, and added some fixes in "master".
To be polite, I didn't quite understand the previous code. So... I've deleted things, and simplified other things. The result is that the "zombie" home server detection should be better.
I don't know if it fixes the "crash on dead home server" issue that people were seeing. I wasn't able to reproduce that. But, the relevant code is now different. So I hope it's fixed.
As always, blame Arran for any bugs. :)
Just a quick note - I'm not sure the home server "status" is working properly in the new code; I'm getting:
# radmin -d /etc/raddb3 -n eduroam -e 'show home_server list' 194.82.174.185 1812 udp auth unknown 0 194.82.174.185 1813 udp acct unknown 0 194.83.56.233 1812 udp auth unknown 0 194.83.56.233 1813 udp acct unknown 0 194.83.56.249 1812 udp auth unknown 0 194.83.56.249 1813 udp acct unknown 0
...even though the servers are up and responding to packets. Same over the RADIUS-based stats interface (minor related note - the "FreeRADIUS-Stats-Server-State" attribute is missing a definition for value "3" i.e. "Unknown")
I think it might be the logic in process.c:remove_from_proxy_hash_nl - that function seems to set the state to "UNKNOWN" if there are zero outstanding packets, which I guess is not right?
Yeah that's only applicable if you're not using status or request. -Arran
Phil Mayers wrote:
Just a quick note - I'm not sure the home server "status" is working properly in the new code; I'm getting:
# radmin -d /etc/raddb3 -n eduroam -e 'show home_server list' 194.82.174.185 1812 udp auth unknown 0
Yup. If it isn't actively replying to packets, the status is unknown. You'll need to look at the "last packet" timestamp to know more.
...even though the servers are up and responding to packets. Same over the RADIUS-based stats interface (minor related note - the "FreeRADIUS-Stats-Server-State" attribute is missing a definition for value "3" i.e. "Unknown")
Fixed, thanks.
I think it might be the logic in process.c:remove_from_proxy_hash_nl - that function seems to set the state to "UNKNOWN" if there are zero outstanding packets, which I guess is not right?
It's right. It was the only way to solve the weird logic and inconsistencies in handling home servers. I could go poke command.c so that it *reports* "alive" if the state is "unknown" AND if the "last packet" was within the last 30 seconds. But the logic in the server core shouldn't change. The new code is understandable, and working. Alan DeKok.
Alan DeKok wrote:
I could go poke command.c so that it *reports* "alive" if the state is "unknown" AND if the "last packet" was within the last 30 seconds.
Done. It uses the "last received packet + ping_interval" to determine if the home server is alive. Alan DeKok.
On 07/01/13 23:47, Alan DeKok wrote:
Alan DeKok wrote:
I could go poke command.c so that it *reports* "alive" if the state is "unknown" AND if the "last packet" was within the last 30 seconds.
Done. It uses the "last received packet + ping_interval" to determine if the home server is alive.
I think stats.c needs something similar, but I'm not sure how that would work; some of the stats are only returned if state==alive, but AFACIT this is never true for any length of time - sub-second? Zero? FWIW we're using the per-home-server stats for Nagios probes. Absent that, we can't tell if it's up or not. Is "unknown" a bit of misnomer? "idle" maybe?
Phil Mayers wrote:
I think stats.c needs something similar, but I'm not sure how that would work; some of the stats are only returned if state==alive, but AFACIT this is never true for any length of time - sub-second? Zero?
Yes... that should be addressed.
FWIW we're using the per-home-server stats for Nagios probes. Absent that, we can't tell if it's up or not.
It is an issue.
Is "unknown" a bit of misnomer? "idle" maybe?
Probably. I'll go poke it. Alan DeKok.
On 08/01/13 13:37, Alan DeKok wrote:
Phil Mayers wrote:
I think stats.c needs something similar, but I'm not sure how that would work; some of the stats are only returned if state==alive, but AFACIT this is never true for any length of time - sub-second? Zero?
Yes... that should be addressed.
FWIW we're using the per-home-server stats for Nagios probes. Absent that, we can't tell if it's up or not.
It is an issue.
For the stats interface, might be easiest to just expose the "last" timestamp; easy to compare this in a service check.
participants (3)
-
Alan DeKok -
Arran Cudbard-Bell -
Phil Mayers