Re: FreeRadius with Eduroam - Accounting
On Tue, 4 Oct 2011, Mike Diggins wrote:
I'm running FreeRadius 2.1.3 on RedHat Enterprise Linux configured as an Eduroam Radius proxy server. My Cisco Wireless Lan Controllers are constantly failing over the Accounting Servers, due to lack of response from the Home Servers, or so says the log. However, I believe the issue is that some remote institutions Radius Servers are ignoring the Accounting packets, and timing out my end, making it believe the Home Servers have failed to respond. FreeRadius responds by marking the Home server dead. It then sends a status-server query, to which is gets a reply, and enables the Dead Home server. I believe that's the sequence of events anyway. I captured some of that in debug mode:
Rejecting request 288 due to lack of any response from home server x.x.x.x port 1813
Finished request 288.
Cleaning up request 288 ID 205 with timestamp +1161
PROXY: Marking home server x.x.x.x port 1813 as zombie (it looks like it is dead).
Sending Status-Server of id 55 to x.x.x.x port 1813 Message-Authenticator := 0x00000000000000000000000000000000 NAS-Identifier := "Status Check. Are you alive?" Waking up in 3.9 seconds.
rad_recv: Access-Accept packet from host x.x.x.x port 1813, id=55, length=806
I don't have any control over Accounting Packets being accepted, or not, by other Eduroam members. Some do, some don't I imagine. Is there a configuration for FreeRadius that handles this situation cleanly? Seems to me that FR should check the Home server first, before marking it dead (at least).
Accounting feature on the WLAN controllers (for now), I noticed that a similar failure is a happening on the Authentication side. Some authentication requests proxied to other radius servers (via Eduroam) are either failing or taking a long time to respond, which also causes my FreeRadius to mark the Home Server as DOWN. That also seems to cause a chain reaction of backed up requests, causing my WLAN controllers to failover the radius server. So, similar to my Accounting problem, is there anyway to prevent a single Authentication failure from backing up the works!? Does FR answer queries in sequence only? I don't really understand why this sort of failure has such a nasty consequence. -Mike
On 13 Oct 2011, at 19:47, Mike Diggins wrote:
On Tue, 4 Oct 2011, Mike Diggins wrote:
I'm running FreeRadius 2.1.3 on RedHat Enterprise Linux configured as an Eduroam Radius proxy server. My Cisco Wireless Lan Controllers are constantly failing over the Accounting Servers, due to lack of response from the Home Servers, or so says the log. However, I believe the issue is that some remote institutions Radius Servers are ignoring the Accounting packets, and timing out my end, making it believe the Home Servers have failed to respond. FreeRadius responds by marking the Home server dead. It then sends a status-server query, to which is gets a reply, and enables the Dead Home server. I believe that's the sequence of events anyway. I captured some of that in debug mode:
Rejecting request 288 due to lack of any response from home server x.x.x.x port 1813
Finished request 288.
Cleaning up request 288 ID 205 with timestamp +1161
PROXY: Marking home server x.x.x.x port 1813 as zombie (it looks like it is dead).
Sending Status-Server of id 55 to x.x.x.x port 1813 Message-Authenticator := 0x00000000000000000000000000000000 NAS-Identifier := "Status Check. Are you alive?" Waking up in 3.9 seconds.
rad_recv: Access-Accept packet from host x.x.x.x port 1813, id=55, length=806
I don't have any control over Accounting Packets being accepted, or not, by other Eduroam members. Some do, some don't I imagine. Is there a configuration for FreeRadius that handles this situation cleanly? Seems to me that FR should check the Home server first, before marking it dead (at least).
Accounting feature on the WLAN controllers (for now), I noticed that a similar failure is a happening on the Authentication side. Some authentication requests proxied to other radius servers (via Eduroam) are either failing or taking a long time to respond, which also causes my FreeRadius to mark the Home Server as DOWN. That also seems to cause a chain reaction of backed up requests, causing my WLAN controllers to failover the radius server.
Are you using status-server? It should figure out the home server is back up pretty quickly, in fact it should never mark it dead at all (see below). Yes i'm going to the special kind of hell for people who post images to public lists... Arran Cudbard-Bell a.cudbardb@freeradius.org Betelwiki, Betelwiki, Betelwiki.... http://wiki.freeradius.org/ !
Mike Diggins wrote:
Accounting feature on the WLAN controllers (for now), I noticed that a similar failure is a happening on the Authentication side. Some authentication requests proxied to other radius servers (via Eduroam) are either failing or taking a long time to respond, which also causes my FreeRadius to mark the Home Server as DOWN. That also seems to cause a chain reaction of backed up requests, causing my WLAN controllers to failover the radius server.
There's really very little you can do about that in RADIUS. FreeRADIUS figures out that a home server is down because it stops responding to requests. So if it stops responding... it looks like it's dead.
So, similar to my Accounting problem, is there anyway to prevent a single Authentication failure from backing up the works!? Does FR answer queries in sequence only? I don't really understand why this sort of failure has such a nasty consequence.
What, exactly, is the server supposed to do when the next hop isn't responding to packets? Is the next hop up? Is it down? How can you tell? It's this kind of thing that makes me think keep-alives should become standard for eduroam. The extra few packets every couple of seconds are a small cost to pay for ensuring that authentication works. Alan DeKok.
On Fri, 14 Oct 2011, Alan DeKok wrote:
Mike Diggins wrote:
Accounting feature on the WLAN controllers (for now), I noticed that a similar failure is a happening on the Authentication side. Some authentication requests proxied to other radius servers (via Eduroam) are either failing or taking a long time to respond, which also causes my FreeRadius to mark the Home Server as DOWN. That also seems to cause a chain reaction of backed up requests, causing my WLAN controllers to failover the radius server.
There's really very little you can do about that in RADIUS. FreeRADIUS figures out that a home server is down because it stops responding to requests.
So if it stops responding... it looks like it's dead.
Does FreeRadius work synchronously only, so a slow response from one remote server stops any other pending authentications from completing until that first one is finished?
So, similar to my Accounting problem, is there anyway to prevent a single Authentication failure from backing up the works!? Does FR answer queries in sequence only? I don't really understand why this sort of failure has such a nasty consequence.
What, exactly, is the server supposed to do when the next hop isn't responding to packets? Is the next hop up? Is it down? How can you tell?
I'm not sure. If my assumption above is correct, then I don't see a good solution. I'm thinking of a method like Squid proxy server, where a number of authenticators are used, so one that's slow or fails doesn't affect the others. The only suggestion I can think of right now is to send the server-status message to the next hop first before marking it dead. I think that would be a safer assumption when proxying anyway.
It's this kind of thing that makes me think keep-alives should become standard for eduroam. The extra few packets every couple of seconds are a small cost to pay for ensuring that authentication works.
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-Mike
On 14 Oct 2011, at 15:33, Mike Diggins wrote:
On Fri, 14 Oct 2011, Alan DeKok wrote:
Mike Diggins wrote:
Accounting feature on the WLAN controllers (for now), I noticed that a similar failure is a happening on the Authentication side. Some authentication requests proxied to other radius servers (via Eduroam) are either failing or taking a long time to respond, which also causes my FreeRadius to mark the Home Server as DOWN. That also seems to cause a chain reaction of backed up requests, causing my WLAN controllers to failover the radius server.
There's really very little you can do about that in RADIUS. FreeRADIUS figures out that a home server is down because it stops responding to requests.
So if it stops responding... it looks like it's dead.
Does FreeRadius work synchronously only, so a slow response from one remote server stops any other pending authentications from completing until that first one is finished?
No. Not in normal multithreaded mode.
So, similar to my Accounting problem, is there anyway to prevent a single Authentication failure from backing up the works!? Does FR answer queries in sequence only? I don't really understand why this sort of failure has such a nasty consequence.
What, exactly, is the server supposed to do when the next hop isn't responding to packets? Is the next hop up? Is it down? How can you tell?
You start sending it status server packets to check, which is what happens if you have status-server configured. That one of the primary reasons for having status-server in the first place, and the reason you don't proxy status-server packets.
I'm not sure. If my assumption above is correct, then I don't see a good solution. I'm thinking of a method like Squid proxy server, where a number of authenticators are used, so one that's slow or fails doesn't affect the others.
That's the case here.
The only suggestion I can think of right now is to send the server-status message to the next hop first before marking it dead. I think that would be a safer assumption when proxying anyway.
Which is what happens already (see diagram in previous post). -Arran Arran Cudbard-Bell a.cudbardb@freeradius.org Betelwiki, Betelwiki, Betelwiki.... http://wiki.freeradius.org/ !
Mike Diggins wrote:
Does FreeRadius work synchronously only, so a slow response from one remote server stops any other pending authentications from completing until that first one is finished?
No. You're probably seeing other authentications fail because they share the same home server... which is down. Since it's down, it can't be used to authenticte *anyone*.
The only suggestion I can think of right now is to send the server-status message to the next hop first before marking it dead. I think that would be a safer assumption when proxying anyway.
FreeRADIUS does that. Read raddb/proxy.conf. It marks the home server "zombie", and starts sending it Status-Server messages. If the home server doesn't support Status-Server, then it ignores them. This makes it look (even more) like it's down. Alan DeKok.
participants (3)
-
Alan DeKok -
Arran Cudbard-Bell -
Mike Diggins