John Paul wrote:
The issue is that if a machine is authenticated and the server that did the authentication is down, the switch will contact the other server and the EAP conversation will fail, causing authentication to fail. Research indicates that this is because the client and server have agreed upon session specific symmetric keys that the new server does not know about.
I don't think it's because of the establishment of symmetric session keys. Once a user has been authenticated, the *next* authentication session is completely independent.
I think it's that if fail-over happens in the *middle* of an EAP authentication, the new server won't have been participating in the TLS setup. Therefore, it doesn't know about the EAP conversation, and it rejects the session.
It's not happening in the middle of the conversation. Server 1 will send an "Access-Accept" packet and the switch enables the port. Then if server 1 goes down and you attempt to reauthenticate the port, the switch tries server 2. That is when it fails. When I tested this the first time, authentications to server 1 worked and to server 2 did not. When I couldn't figure it out, I turned the test machines off and left for the day. The next day I had server 1 turned off - I turned the test machines on and authentications to server 2 worked fine, but would not work on server 1 once I powered it on. The common denominator here was that authentications work against the first server it tries but not any others. That's why I postulated that there must be some sort of session resumption going on.
Is there a way to tell FreeRadius to tear down the session once the user has been authenticated so that the next authentication will work if using a different server? If not, is anyone working on a patch or other change to enable this? I'll be happy to write the patch but am unfamiliar with the code. Can you tell me roughly where to look?
Please check first that the server isn't establishing session keys. Since FreeRADIUS doesn't do fast session resumption, I have no idea how "session specific symmetric keys" could affect anything.
Was a shot in the dark based on some googling I had done awhile back. I assumed FreeRadius and Windows were agreeing on some sort of "fast session resumption" that would include symmetric encryption keys.
i.e. authentication one user via EAP. Stop ALL other authentications. Turn off one RADIUS server, so it fails over to the other. Try to re-authenticate that one user.
The user *should* be authenticated.
That's how I've been testing it and it's not working, see above description. If I run the servers in debug mode (radiusd -X) the output starts to get different at line 420 of the middle of the conversation. The working server waits 6 seconds and then continues processing the next Access-Request packet: (Note that these servers are in a test environment, so there are no additional access-request packets reaching the servers, just the packets from the testing machine) Finished request 0 Going to the next request --- Walking the entire request list --- Waking up in 6 seconds... rad_recv: Access-Request packet from host 172.22.30.170:1812, id=29, length=171 NAS-IP-Address = 172.22.30.170 NAS-Port = 50016 NAS-Port-Type = Ethernet ...... while the non-working server cleans up the request, then receives an Access-Request packet: Sending Access-Challenge of id 37 to 172.22.30.170 port 1812 EAP-Message = 0x011300061920 Message-Authenticator = 0x00000000000000000000000000000000 State = 0x466470a3816b19a56b24e71b7dff8089 Finished request 0 Going to the next request --- Walking the entire request list --- Waking up in 6 seconds... --- Walking the entire request list --- Cleaning up request 0 ID 37 with timestamp 47556860 Nothing to do. Sleeping until we see a request. rad_recv: Access-Request packet from host 172.22.30.170:1812, id=38, length=171 NAS-IP-Address = 172.22.30.170 NAS-Port = 50016 NAS-Port-Type = Ethernet ..... I can send the debugs in their entirety if you would like but thought I'd keep them off the list for now. What else could be going on here? Thanks for looking at this, if I can send you anything else please let me know.
John Paul wrote:
When I tested this the first time, authentications to server 1 worked and to server 2 did not. When I couldn't figure it out, I turned the test machines off and left for the day. The next day I had server 1 turned off - I turned the test machines on and authentications to server 2 worked fine, but would not work on server 1 once I powered it on. The common denominator here was that authentications work against the first server it tries but not any others. That's why I postulated that there must be some sort of session resumption going on.
FreeRADIUS does not do session resumption. If the supplicant tries to do session resumption, I don't know what will happen. You should ensure that the supplicant has session resumption disabled.
Was a shot in the dark based on some googling I had done awhile back. I assumed FreeRadius and Windows were agreeing on some sort of "fast session resumption" that would include symmetric encryption keys.
Windows may support session resumption. FreeRADIUS does not. There are patches to enable this, but they have not, as yet, been integrated. In any case, they won't help you to fail over from one server to another. If the Windows client has session resumption enable, *should* notice that session resumption has failed, and re-authenticate from scratch. I suspect that the issue is "fast session resumption" on the Windows box. Turn it off. If that doesn't fix it, the Windows client is broken. Try another one. Alan DeKok.
FreeRADIUS does not do session resumption. If the supplicant tries to do session resumption, I don't know what will happen. You should ensure that the supplicant has session resumption disabled.
Windows does support it but it's switched off by default and I have verified this
Windows may support session resumption. FreeRADIUS does not.
There are patches to enable this, but they have not, as yet, been integrated. In any case, they won't help you to fail over from one server to another.
I'm not interested in doing fast session resumption, I'd just as soon have the client start fresh every time.
If the Windows client has session resumption enable, *should* notice that session resumption has failed, and re-authenticate from scratch.
I would think it would too, but it does not seem to, even after it is given several minutes to get its act together
I suspect that the issue is "fast session resumption" on the Windows box. Turn it off.
It is indeed turned off
If that doesn't fix it, the Windows client is broken. Try another one.
I'll be happy to try another client - is there one you would recommend or suggest that I try
There are patches to enable this, but they have not, as yet, been integrated. In any case, they won't help you to fail over from one server to another.
If/when those patches get integrated, it would be highly useful to support failover between servers. I guess the requirements for this would be: 1. Expose the openssl session cache config, so that distcache can be configured to share the SSL sessions between servers 2. Implement some way of attaching the PEAP/TTLS tunnel state to the session cache, or otherwise be reachable by the other FreeRadius server, so that when resumption occurs the inner info can be (re)used for authorization. I don't know much about the OpenSSL session API, so the 2nd could be either very hard or trivial ;o)
Phil Mayers wrote:
There are patches to enable this, but they have not, as yet, been integrated. In any case, they won't help you to fail over from one server to another.
If/when those patches get integrated, it would be highly useful to support failover between servers. I guess the requirements for this would be:
Bleah. I guess it's possible, but it's pretty ugly.
1. Expose the openssl session cache config, so that distcache can be configured to share the SSL sessions between servers
As always, patches are welcome. :) On a related note, sharing the RADIUS packets between servers would be a good idea. It would avoid duplicate handling of Access-Request or Accounting-Request.
2. Implement some way of attaching the PEAP/TTLS tunnel state to the session cache, or otherwise be reachable by the other FreeRadius server, so that when resumption occurs the inner info can be (re)used for authorization.
You can register callbacks to store OpenSSL contexts somewhere outside of main memory. It's not hard, but it requires someone to write the code. Alan DeKok.
Debug the switch. It's quite likely that it isn't marking the radius server that is down as dead but it tries it again when it recieves the challenge. Ivan Kalik Kalik informatika ISP Dana 4/12/2007, "John Paul" <JDPAUL@gocolumbiamo.com> piše:
John Paul wrote:
The issue is that if a machine is authenticated and the server that did the authentication is down, the switch will contact the other server and the EAP conversation will fail, causing authentication to fail. Research indicates that this is because the client and server have agreed upon session specific symmetric keys that the new server does not know about.
I don't think it's because of the establishment of symmetric session keys. Once a user has been authenticated, the *next* authentication session is completely independent.
I think it's that if fail-over happens in the *middle* of an EAP authentication, the new server won't have been participating in the TLS setup. Therefore, it doesn't know about the EAP conversation, and it rejects the session.
It's not happening in the middle of the conversation. Server 1 will send an "Access-Accept" packet and the switch enables the port. Then if server 1 goes down and you attempt to reauthenticate the port, the switch tries server 2. That is when it fails.
When I tested this the first time, authentications to server 1 worked and to server 2 did not. When I couldn't figure it out, I turned the test machines off and left for the day. The next day I had server 1 turned off - I turned the test machines on and authentications to server 2 worked fine, but would not work on server 1 once I powered it on. The common denominator here was that authentications work against the first server it tries but not any others. That's why I postulated that there must be some sort of session resumption going on.
Is there a way to tell FreeRadius to tear down the session once the user has been authenticated so that the next authentication will work if using a different server? If not, is anyone working on a patch or other change to enable this? I'll be happy to write the patch but am unfamiliar with the code. Can you tell me roughly where to look?
Please check first that the server isn't establishing session keys. Since FreeRADIUS doesn't do fast session resumption, I have no idea how "session specific symmetric keys" could affect anything.
Was a shot in the dark based on some googling I had done awhile back. I assumed FreeRadius and Windows were agreeing on some sort of "fast session resumption" that would include symmetric encryption keys.
i.e. authentication one user via EAP. Stop ALL other authentications. Turn off one RADIUS server, so it fails over to the other. Try to re-authenticate that one user.
The user *should* be authenticated.
That's how I've been testing it and it's not working, see above description.
If I run the servers in debug mode (radiusd -X) the output starts to get different at line 420 of the middle of the conversation. The working server waits 6 seconds and then continues processing the next Access-Request packet:
(Note that these servers are in a test environment, so there are no additional access-request packets reaching the servers, just the packets from the testing machine)
Finished request 0 Going to the next request --- Walking the entire request list --- Waking up in 6 seconds... rad_recv: Access-Request packet from host 172.22.30.170:1812, id=29, length=171 NAS-IP-Address = 172.22.30.170 NAS-Port = 50016 NAS-Port-Type = Ethernet .......
while the non-working server cleans up the request, then receives an Access-Request packet:
Sending Access-Challenge of id 37 to 172.22.30.170 port 1812 EAP-Message = 0x011300061920 Message-Authenticator = 0x00000000000000000000000000000000 State = 0x466470a3816b19a56b24e71b7dff8089 Finished request 0 Going to the next request --- Walking the entire request list --- Waking up in 6 seconds... --- Walking the entire request list --- Cleaning up request 0 ID 37 with timestamp 47556860 Nothing to do. Sleeping until we see a request. rad_recv: Access-Request packet from host 172.22.30.170:1812, id=38, length=171 NAS-IP-Address = 172.22.30.170 NAS-Port = 50016 NAS-Port-Type = Ethernet ......
I can send the debugs in their entirety if you would like but thought I'd keep them off the list for now. What else could be going on here?
Thanks for looking at this, if I can send you anything else please let me know.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On 12/4/2007 at 10:01 AM, in message <fVJXevUg.1196784061.7601270.tnt@kalik.co.yu>, <tnt@kalik.co.yu> wrote: Debug the switch. It's quite likely that it isn't marking the radius server that is down as dead but it tries it again when it recieves the challenge.
Bingo, we have a winner. The switch was attempting to contact the downed server after the failover server had sent an access challenge back to the client. I'm not sure why this made it fail but it did. If I set the switch's radius server deadtime this seems to have fixed the issue. Thanks for everyone's help.
John Paul wrote:
John Paul wrote:
The issue is that if a machine is authenticated and the server that did the authentication is down, the switch will contact the other server and the EAP conversation will fail, causing authentication to fail. Research indicates that this is because the client and server have agreed upon session specific symmetric keys that the new server does not know about. I don't think it's because of the establishment of symmetric session keys. Once a user has been authenticated, the *next* authentication session is completely independent.
I think it's that if fail-over happens in the *middle* of an EAP authentication, the new server won't have been participating in the TLS setup. Therefore, it doesn't know about the EAP conversation, and it rejects the session.
It's not happening in the middle of the conversation. Server 1 will send an "Access-Accept" packet and the switch enables the port. Then if server 1 goes down and you attempt to reauthenticate the port, the switch tries server 2. That is when it fails.
It is more likely that server 2 simply isn't configured correctly. Please post full debug (run "radiusd -X") output for the working (initial) request on server 1 and the failing (subsequent) request on server 2.
participants (4)
-
Alan DeKok -
John Paul -
Phil Mayers -
tnt＠kalik.co.yu