generate_state() in rlm_eap module generates duplicate state ?

[Vinay Wagh]

inline ..

On 9/25/07, Alan DeKok <aland at deployingradius.com > wrote:
>
> Vinay Wagh wrote:
> > On looking deeper in to the problem I saw that the reason the
> > rbtree_insert in eaplist_add() fails is because there is already a node
> > in the tree with the same state (PW_STATE), this node is for a different
>
> > request altogether and so the insert in to the tree fails. Since I send
> > multiple requests from the same gateway and the radius client on the
> > gateway opens up 4 source ports to the radius server we have can send 4
> > requests in the same second that have the same ID and source IP address.
>
>   Hmm... that may be allowed by the spec, but it seems broken to me.
> The code in rlm_eap does NOT use the source port to distinguish
> requests, because some NASes send EAP requests for one session via
> multiple source ports.  It just depends on how busy the NAS is.


We need to use more than one source port on the client because the client
supports only 255 requests per instance of radius client and we need about
1000 requests per second. Each EAP session will always use the same port
while it is being authenticated but a new request can pick up a different Id
on a different source port.


> The eap_handler_cmp() function uses the STATE to distinguish requests if
> > the id and source IP address are the same, but in my case the state is
> > also the same. Here is the relevant debug log for it
>
>   If the state is the same, then something is going seriously wrong
> inside of the server.  The state that is generated SHOULD be random, and
> unique per request.



Which is why it took me some time to figure this out. What I did was added
debug code in rbtree_insert to print out contents of the node if a duplicate
node existed. In the logs I saw that the node had the same state but a
different identity.

> Wed Sep 19 22:36:37 2007 : Info: STATE: Added state 82 f1 8b 49 6f bf b9
> > a4 ee 2a d3 d5 ef f8 ec 3b for {am= 1}201myid at mynet.net<1%7D201myid at mynet.net>
>
>   Can you instrument the code to see when the state is generated?  My
> suspicion is that a new State attribute is NOT being generated.   i.e.
> the server sees the "same" request, and re-uses the same State.
>
>   That shouldn't be happening, though.  On a new EAP-Identity packet, a
> new State is allocated, and added to the list.  Since the State is
> generated from random numbers, it SHOULD be unique every time.


I have already added debug code to print out the state every time the radius
server generates it. Out of a thousand requests you see atleast 4-5 requests
with duplicate state info, the others have unique states. The debug above
'Added state ..." gets printed after a call to generate_state().


> I am guessing this can cause other problems, not sure if this can result
> > in sending Access-Accept to users who should not be authenticated but it
> > definitely rejects users who should be authenticated. Another problem I
> > see which seems related to this is that I receive Access-Accept on the
> > gateway for a valid user but the reply has an invalid message
> > authenticator. I think this is probably because it finds a different
> > handler instead, but not sure about that.
>
>   The calculation of the Message-Authenticator has nothing to do with
> EAP, or with the State.  If the Message-Authenticator is invalid, then
> something is seriously wrong.



The reason I started debugging this problem is because I started getting
Access-Reject without RADIUS_ATTR_MESSAGE_AUTHENTICATOR which is a seperate
attribute in the radius message. I also observed that some reply's from the
radius server had this field but it did not match the authenticator in the
original request. Then I tried to link this to the problem I found and I
think it is possible if we generate the same state. Assume that after the
server sends the access challenge the radius server fails to insert the
handler because there is already a duplicate and then before it gets rid of
the duplicate handler the client replies. In this case the radius server
will try to look for the handler and actually find it since the id, ip addr
and state is the same but the identity is different. It can then use that
context to reply to this request in which case the fields may not match. If
the radius server had already replied to the duplicate handler then it will
not find the handler that for our current request and send an Access-Reject.


So whether we get an Access-Reject or a reply with invalid message
authenticator depends on the timing of whether the radius server still has
the duplicate context or not. Is that possible though ?


 > Is this a known bug ? If yes, is it fixed and in what release ? For now,
> > I am changing the eap_handler_cmp() function to compare the identity if
> > the state is the same, since in our lab setup we also have a unique
> > identity for each user, do you see a problem with doing this.
>
>   No.  I would suggest in your situation also adding a comparison on the
> source port.  That should work ard the problem.
>
>   I'm hesitant to add that comparison to the main code, though.  It may
> cause other peoples networks to break.



Thats true, but how do we deal with this as a general solution ? If the
product we build needs to sit in a network where they have deployed free
radius we cannot modify their code and will need a solution from the
freeradius community.  I seems like we need to investigate the
generate_state() function and see if we really generate random states.

I have uploaded the log file you can download it at
http://download.yousendit.com/D1B4E30A06784505 (its 3MB and I was not sure
if I would get flamed for attaching it, it will be available for 14 days at
this location). Here are the important line numbers

Line 23472:    Access-Request from 201myid at mynet.net
Line 23521:    A new state is generated for 210myid at myid.net and
Access-Challenge is sent
Line 27092:    Access-Request from 245myid at mynet.net
Line 27141:    State generate for 245myid at mynet.net but rbtree_insert fails
due to duplicate state (same as 201myid at mynet.net)
                     Access-Challenge sent to 245myid at mynet.net
Line 43911:    Response to Access-Challenge from 201myid at mynet.net
Line 43964:    Freeing handler for 201myid at mynet.net and sending
Access-Accept.
Line 49522:    Access-Request from 245myid at mynet.net, this is a response to
the Access-Challenge
Lin3 49569:    eaplist_find() fails to find the handler for
245myid at mynet.net since we had already sent the reply to 201myid at mynet.net


Thanks for your time,
Vinay


  Alan DeKok.
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freeradius.org/pipermail/freeradius-users/attachments/20070925/bd071bf3/attachment.html>