inline ..<br><br><div><span class="gmail_quote">On 9/25/07, <b class="gmail_sendername">Alan DeKok</b> <<a href="mailto:aland@deployingradius.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">aland@deployingradius.com
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Vinay Wagh wrote:<br>> On looking deeper in to the problem I saw that the reason the<br>> rbtree_insert in eaplist_add() fails is because there is already a node<br>> in the tree with the same state (PW_STATE), this node is for a different
<br>> request altogether and so the insert in to the tree fails. Since I send<br>> multiple requests from the same gateway and the radius client on the<br>> gateway opens up 4 source ports to the radius server we have can send 4
<br>> requests in the same second that have the same ID and source IP address.<br><br> Hmm... that may be allowed by the spec, but it seems broken to me.<br>The code in rlm_eap does NOT use the source port to distinguish
<br>requests, because some NASes send EAP requests for one session via<br>multiple source ports. It just depends on how busy the NAS is.</blockquote><div><br>We need to use more than one source port on the client because the client supports only 255 requests per instance of radius client and we need about 1000 requests per second. Each EAP session will always use the same port while it is being authenticated but a new request can pick up a different Id on a different source port.
<br> </div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">> The eap_handler_cmp() function uses the STATE to distinguish requests if
<br>> the id and source IP address are the same, but in my case the state is<br>> also the same. Here is the relevant debug log for it<br><br> If the state is the same, then something is going seriously wrong<br>inside of the server. The state that is generated SHOULD be random, and
<br>unique per request.</blockquote><div><br><br>Which is why it took me some time to figure this out. What I did was added debug code in rbtree_insert to print out contents of the node if a duplicate node existed. In the logs I saw that the node had the same state but a different identity.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">> Wed Sep 19 22:36:37 2007 : Info: STATE: Added state 82 f1 8b 49 6f bf b9
<br>
> a4 ee 2a d3 d5 ef f8 ec 3b for {am= <a href="mailto:1%7D201myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">1}201myid@mynet.net</a><br><br> Can you instrument the code to see when the state is generated? My
<br>suspicion is that a new State attribute is NOT being generated.
i.e.<br>the server sees the "same" request, and re-uses the same State.<br><br> That shouldn't be happening, though. On a new EAP-Identity packet, a<br>new State is allocated, and added to the list. Since the State is
<br>generated from random numbers, it SHOULD be unique every time.</blockquote><div><br>I have already added debug code to print out the state every time the radius server generates it. Out of a thousand requests you see atleast 4-5 requests with duplicate state info, the others have unique states. The debug above 'Added state ..." gets printed after a call to generate_state().
<br><br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">> I am guessing this can cause other problems, not sure if this can result
<br>> in sending Access-Accept to users who should not be authenticated but it<br>> definitely rejects users who should be authenticated. Another problem I<br>> see which seems related to this is that I receive Access-Accept on the
<br>> gateway for a valid user but the reply has an invalid message<br>> authenticator. I think this is probably because it finds a different<br>> handler instead, but not sure about that.<br><br> The calculation of the Message-Authenticator has nothing to do with
<br>EAP, or with the State. If the Message-Authenticator is invalid, then<br>something is seriously wrong.</blockquote><div><br><br>The reason I started debugging this problem is because I started getting Access-Reject without RADIUS_ATTR_MESSAGE_AUTHENTICATOR which is a seperate attribute in the radius message. I also observed that some reply's from the radius server had this field but it did not match the authenticator in the original request. Then I tried to link this to the problem I found and I think it is possible if we generate the same state. Assume that after the server sends the access challenge the radius server fails to insert the handler because there is already a duplicate and then before it gets rid of the duplicate handler the client replies. In this case the radius server will try to look for the handler and actually find it since the id, ip addr and state is the same but the identity is different. It can then use that context to reply to this request in which case the fields may not match. If the radius server had already replied to the duplicate handler then it will not find the handler that for our current request and send an Access-Reject.
<br><br>So whether we get an Access-Reject or a reply with invalid message authenticator depends on the timing of whether the radius server still has the duplicate context or not. Is that possible though ?<br> </div><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> Is this a known bug ? If yes, is it fixed and in what release ? For now,<br>> I am changing the eap_handler_cmp() function to compare the identity if<br>> the state is the same, since in our lab setup we also have a unique
<br>> identity for each user, do you see a problem with doing this.<br><br> No. I would suggest in your situation also adding a comparison on the<br>source port. That should work ard the problem.<br><br> I'm hesitant to add that comparison to the main code, though. It may
<br>cause other peoples networks to break.</blockquote><div><br><br>Thats true, but how do we deal with this as a general solution ? If the product we build needs to sit in a network where they have deployed free radius we cannot modify their code and will need a solution from the freeradius community. I seems like we need to investigate the generate_state() function and see if we really generate random states.
<br> </div>I have uploaded the log file you can download it at <font style="color: rgb(0, 0, 0); font-size: 11px;"> <a href="http://download.yousendit.com/D1B4E30A06784505" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
http://download.yousendit.com/D1B4E30A06784505</a></font> (its 3MB and I was not sure if I would get flamed for attaching it, it will be available for 14 days at this location). Here are the important line numbers<br><br>
Line 23472: Access-Request from <a href="mailto:201myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
201myid@mynet.net</a> <br>Line 23521: A new state is generated for
<a href="mailto:210myid@myid.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">210myid@myid.net</a> and Access-Challenge is sent<br>Line 27092: Access-Request from <a href="mailto:245myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
245myid@mynet.net</a><br>Line 27141: State generate for <a href="mailto:245myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
245myid@mynet.net</a> but rbtree_insert fails due to duplicate state (same as <a href="mailto:201myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">201myid@mynet.net</a>)<br> Access-Challenge sent to
<a href="mailto:245myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">245myid@mynet.net
</a><br>Line 43911: Response to Access-Challenge from <a href="mailto:201myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">201myid@mynet.net</a> <br>Line 43964: Freeing handler for
<a href="mailto:201myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">201myid@mynet.net</a> and sending Access-Accept.
<br>Line 49522: Access-Request from <a href="mailto:245myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">245myid@mynet.net</a>, this is a response to the Access-Challenge<br>Lin3 49569: eaplist_find() fails to find the handler for
<a href="mailto:245myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
245myid@mynet.net</a> since we had already sent the reply to <a href="mailto:201myid@mynet.net" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">201myid@mynet.net</a><br><br><br>Thanks for your time,
<br>Vinay<br><br><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Alan DeKok.<br>-<br>List info/subscribe/unsubscribe? See <a href="http://www.freeradius.org/list/users.html" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://www.freeradius.org/list/users.html
</a><br></blockquote></div><br>