I'm hoping somebody can shed a little light on this, I found it very strange. btw: we're talking about freeradius version 1.1.3 here. We currently run some lesser radius server on our network, and I have been engaged in converting the environment to freeradius (yea!). I believe we have worked out (lab and pilot-site testing) all the kinks for our authentication needs, and today we attempted to implement the change across our whole network. Our lesser radius server lives on two physical boxes and listens on ports 1645/1646 AND 1812/1813 (can freeradius mimic this and listen on both sets of ports?) Our plan has been to make sure appropriate radius client devices were all pointing to the 1812/1813 authentication ports prior to the change. The change involved stopping the lesser server and bringing up freeradius on the ports 1812/1813 on both of the two physical servers. The idea was that the clients (chiefly Cisco WAPs) would have a seemless conversion and continue to authenticate using the same IP and ports they were previously using. This however was not the observed result. What happened was almost universal failure. What we saw were requests coming into freeradius, being processed as expected, and returning the appropriate response - many Accept responses clearly visible in the logs. The radius clients however did not accept these responses and treated them as authentication failure. The pilot: We ran freeradius on ports 11812/11813 on these same physical servers, side-by-side with the lesser radius server. We then pointed the client devices to the alternate port combination. They were successful and behaved exactly as expected. Log (debug) output from freeradius for the successful pilots and the failed implementation today are identical, however, with the new ports today 1812/1813 the pilot sites who had previously been working also failed. Does anyone have an idea what could have happened here? If a radius client was talking to server X, and then suddenly recieves a response from server Y on the same IP / port combination... is there some way that the client can tell the server has changed, and thus reject any responses from it as invalid? Does the radius client bind in some way to the server (on the application layer I would assume)? Shared secrets are the same and the request is clearly visible and processed by freeradius. debug output for a request looks like the following: freeradius does it's job but the client device doesn't authenticate the user. Please note that LDAP should fail in this instance. LDAP is used for another kind of authentication. The users file which matches DEFAULT at line 7798 is where the authentication comes from. Nov 29 10:58:48 rad_recv: Access-Request packet from host 10.32.251.10:32768, id=105, length=183 Nov 29 10:58:48 User-Name = "0014a4858ad0" Nov 29 10:58:48 Called-Station-Id = "00-15-62-a9-cc-50:guest" Nov 29 10:58:48 Calling-Station-Id = "00-14-a4-85-8a-d0" Nov 29 10:58:48 NAS-Port = 29 Nov 29 10:58:48 NAS-IP-Address = 10.32.251.10 Nov 29 10:58:48 NAS-Identifier = "co-lp-wlc01" Nov 29 10:58:48 Airespace-Wlan-Id = 7 Nov 29 10:58:48 User-Password = "********" Nov 29 10:58:48 Service-Type = Call-Check Nov 29 10:58:48 Framed-MTU = 1300 Nov 29 10:58:48 NAS-Port-Type = Wireless-802.11 Nov 29 10:58:48 Tunnel-Type:0 = VLAN Nov 29 10:58:48 Tunnel-Medium-Type:0 = IEEE-802 Nov 29 10:58:48 Tunnel-Private-Group-Id:0 = "2250" Nov 29 10:58:48 Processing the authorize section of radiusd.conf Nov 29 10:58:48 modcall: entering group authorize for request 0 Nov 29 10:58:48 hints: Matched DEFAULT at 39 Nov 29 10:58:48 modcall[authorize]: module "preprocess" returns ok for request 0 Nov 29 10:58:48 rlm_realm: No '@' in User-Name = "0014a4858ad0", looking up realm NULL Nov 29 10:58:48 rlm_realm: No such realm "NULL" Nov 29 10:58:48 modcall[authorize]: module "suffix" returns noop for request 0 Nov 29 10:58:48 users: Matched entry DEFAULT at line 7798 Nov 29 10:58:48 modcall[authorize]: module "files" returns ok for request 0 Nov 29 10:58:48 radius_xlat: '/usr/local/freeradius/etc/scripts/wireless.atz 0014a4858ad0' Nov 29 10:58:48 Exec-Program: /usr/local/freeradius/etc/scripts/wireless.atz 0014a4858ad0 Nov 29 10:58:48 Exec-Program output: Requested WLAN is not restricted, deferring authentication. Nov 29 10:58:48 Exec-Program-Wait: plaintext: Requested WLAN is not restricted, deferring authentication. Nov 29 10:58:48 Exec-Program: returned: 0 Nov 29 10:58:48 modcall[authorize]: module "wireless" returns ok for request 0 Nov 29 10:58:48 rlm_ldap: - authorize Nov 29 10:58:48 rlm_ldap: performing user authorization for 0014a4858ad0 Nov 29 10:58:48 radius_xlat: '(&(uid=0014a4858ad0)(accesslist=)(isaccountenabled=true))' Nov 29 10:58:48 radius_xlat: 'o=xyz' Nov 29 10:58:48 rlm_ldap: ldap_get_conn: Checking Id: 0 Nov 29 10:58:48 rlm_ldap: ldap_get_conn: Got Id: 0 Nov 29 10:58:48 rlm_ldap: attempting LDAP reconnection Nov 29 10:58:48 rlm_ldap: (re)connect to ldapvip.co.ihc.com:389, authentication 0 Nov 29 10:58:48 rlm_ldap: bind as appl=VPN Radius Server, ou=applications, o=xyz/******** to ldapvip.xyz.com:389 Nov 29 10:58:48 rlm_ldap: waiting for bind result ... Nov 29 10:58:48 rlm_ldap: Bind was successful Nov 29 10:58:48 rlm_ldap: performing search in o=xyz, with filter (&(uid=0014a4858ad0)(accesslist=)(isaccountenabled=true)) Nov 29 10:58:48 rlm_ldap: object not found or got ambiguous search result Nov 29 10:58:48 rlm_ldap: search failed Nov 29 10:58:48 rlm_ldap: ldap_release_conn: Release Id: 0 Nov 29 10:58:48 modcall[authorize]: module "ldap" returns notfound for request 0 Nov 29 10:58:48 modcall: leaving group authorize (returns ok) for request 0 Nov 29 10:58:48 rad_check_password: Found Auth-Type Accept Nov 29 10:58:48 rad_check_password: Auth-Type = Accept, accepting the user Nov 29 10:58:48 Sending Access-Accept of id 105 to 10.32.251.10 port 32768 Nov 29 10:58:48 Finished request 0 Thanks, Lin
Lin Richardson wrote:
Our lesser radius server lives on two physical boxes and listens on ports 1645/1646 AND 1812/1813 (can freeradius mimic this and listen on both sets of ports?)
Yes. See "listen" in radiusd.conf.
What we saw were requests coming into freeradius, being processed as expected, and returning the appropriate response - many Accept responses clearly visible in the logs. The radius clients however did not accept these responses and treated them as authentication failure.
See the FAQ. Do you have multiple IP's on the machine?
Does anyone have an idea what could have happened here? If a radius client was talking to server X, and then suddenly recieves a response from server Y on the same IP / port combination...
Huh? What does that mean? "Suddenly", as in... what, exactly? If you shut down the old machine, and start a new machine with the same IP, then RADIUS should work as before, assuming the server configuration is the same.
Nov 29 10:58:48 rad_check_password: Found Auth-Type Accept Nov 29 10:58:48 rad_check_password: Auth-Type = Accept, accepting the user Nov 29 10:58:48 Sending Access-Accept of id 105 to 10.32.251.10 <http://10.32.251.10> port 32768 Nov 29 10:58:48 Finished request 0
The Access-Accept contains no attributes. Are you sure you want to do that? The request contained VLAN attributes, so I presume you want to put the user in a VLAN. i.e. Are you sure that you have configured FreeRADIUS to return the SAME response as your old server? If the old server returns a bunch of attributes, and FreeRADIUS doesn't... then the configurations aren't identical, and the clients will behave differently. Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
Hi Alan, Thanks for taking a look at this. A quick clarification. The configuration of freeradius in this situation has already been tested and is not really the basis for my question. It is not a standard configuration, but it does work. Freeradius is installed on the same physical machine and listens on the same IP address as another vended radius solution. Freeradius is configured to use nonstandard ports (11812/11813), while the vended solution is using both sets of common ports - (same IP address, same physical machine). We have moved entire pilot facilities to authenticate on the nonstandard ports using freeradius and everything works swimmingly. Huge kudos to freeradius on that count. We have chosen to use freeradius not for economy, but rather for flexibility - hence the nonstandard configuration. We want to move all facilities to freeradius now for production use. We would like to use the standard ports of 1812/1813 in an effort to keep things somewhat industry standard, and also because all the clients are already configured for those ports. We would have to manually change the configuration of hundreds of WAPs to the nonstandard ports if we intended to continue to use them. The concept was simple - shut down the vended radius solution, thus freeing up the desired ports. Change the "listen" directive in radiusd.conf to use 1812/1813, and restart freeradius. All the clients that WERE using the vended solution on the given ports should continue on as if nothing had happened, only now they would be talking to freeradius. Bear in mind that literally the same configuration and installation of freeradius with only a different listening port have already been in limited production use with the very devices in question. I did read the FAQ which talked about needing to pass the same/correct attributes back to the client as a previous/other working radius solution in order for freeradius to work. I strongly believe that this is not the problem here as it has already been working without any problems on a different port. I almost wonder if power cycling the client device would resolve the problem... once freeradius had assumed the new ports.
What we saw were requests coming into freeradius, being processed as
expected, and returning the appropriate response - many Accept responses clearly visible in the logs. The radius clients however did not accept these responses and treated them as authentication failure.
See the FAQ. Do you have multiple IP's on the machine?
No multiple IPs
Does anyone have an idea what could have happened here? If a radius
client was talking to server X, and then suddenly recieves a response from server Y on the same IP / port combination...
Huh? What does that mean? "Suddenly", as in... what, exactly?
Just an expression meaning no other event to mark the change. Really more unexpectedly than suddenly. If you shut down the old machine, and start a new machine with the
same IP, then RADIUS should work as before, assuming the server configuration is the same.
So... even if they are on the same machine... killing the vended radius solution and starting freeradius on the same ports the vended solution was using should work seamlessly? Clients won't know or care, provided their shared secret and attributes are the same? Could a radius response with spoofed source IP and port be accepted as valid by a radius client simply based on the IP and port... and of course the coherence of the reply to a corresponding request?
Nov 29 10:58:48 rad_check_password: Found Auth-Type Accept
Nov 29 10:58:48 rad_check_password: Auth-Type = Accept, accepting the user Nov 29 10:58:48 Sending Access-Accept of id 105 to 10.32.251.10 <http://10.32.251.10> port 32768 Nov 29 10:58:48 Finished request 0
The Access-Accept contains no attributes. Are you sure you want to do that? The request contained VLAN attributes, so I presume you want to put the user in a VLAN.
The WAP controls this, and I have been forbidden to specify anything to do with VLANs. Which of the request attributes do/don't flow through to the reply? i.e. Are you sure that you have configured FreeRADIUS to return the
SAME response as your old server? If the old server returns a bunch of attributes, and FreeRADIUS doesn't... then the configurations aren't identical, and the clients will behave differently.
Because this is such a logical explanation I will also revisit this possibility. For a different type of radius client we do pass some things back, but the WAPs have always worked without anything else. Again, thanks for taking time to kick this around, I am truely at a loss. Regards, Lin
Lin Richardson wrote:
We want to move all facilities to freeradius now for production use. We would like to use the standard ports of 1812/1813 in an effort to keep things somewhat industry standard, and also because all the clients are already configured for those ports. We would have to manually change the configuration of hundreds of WAPs to the nonstandard ports if we intended to continue to use them.
Or... IP re-write rules on the RADIUS server. There's more than one way to solve a problem.
Bear in mind that literally the same configuration and installation of freeradius with only a different listening port have already been in limited production use with the very devices in question.
OK.
I almost wonder if power cycling the client device would resolve the problem... once freeradius had assumed the new ports.
I don't see why. RADIUS doesn't maintain sessions across states, and NASes are pretty dumb. This kind of behavior is a little surprising.
So... even if they are on the same machine... killing the vended radius solution and starting freeradius on the same ports the vended solution was using should work seamlessly? Clients won't know or care, provided their shared secret and attributes are the same?
Yes. The clients know nothing more than RADIUS requests and RADIUS responses. Same request/response == same behavior.
Could a radius response with spoofed source IP and port be accepted as valid by a radius client simply based on the IP and port... and of course the coherence of the reply to a corresponding request?
No, you need the shared secret, too.
The WAP controls this, and I have been forbidden to specify anything to do with VLANs. Which of the request attributes do/don't flow through to the reply?
None flow through to the reply. RADIUS clients can request anything they want, and you can't control that. So sane RADIUS servers don't let the clients control the reply. They let the *administrator* control the reply.
Because this is such a logical explanation I will also revisit this possibility. For a different type of radius client we do pass some things back, but the WAPs have always worked without anything else.
Even for the old server? An empty Access-Accept? That sounds a little odd. Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
participants (2)
-
Alan DeKok -
Lin Richardson