Debugging "No EAP session matching the State variable"

[John Douglass]

I run two freeradius servers (both 2.2.0 x86_64) with MySQL backends 
doing ntlm_auth (RHEL 6 Samba 3.6.9) for EAP-PEAP-MSChapV2 for our 
client devices.

I have enabled the server debug using radmin (the debug file is HUUUUUGE 
so that is why I am not posting it along with). I have googled and read 
and analyzed as much as I can so I am
looking to the list to see if anyone has experienced this problem.

I was concentrating on a single user mhaley:

Sep 16 08:40:33 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:40:42 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:40:42 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:40:54 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:40:54 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:41:22 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:41:22 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:41:33 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:41:33 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:41:49 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:41:49 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:42:08 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:42:08 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:42:12 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:42:12 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 08:42:15 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 08:42:15 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client resnet1-WiSM-A port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 09:57:56 newdvlanb radiusd[15211]: Invalid user: [mhaley7] (from 
client Rich-core-WiSM-E port 29 cli 3c:e0:72:a5:b7:81)
Sep 16 09:58:01 newdvlanb radiusd[15211]: Invalid user: [mhaley7] (from 
client Rich-core-WiSM-E port 29 cli 3c:e0:72:a5:b7:81)
Sep 16 09:58:57 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-E port 29 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 09:58:57 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-E port 29 cli 3c:e0:72:a5:b7:81)
Sep 16 10:03:42 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 10:03:42 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 10:03:49 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 10:03:54 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 10:06:09 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 10:06:09 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81)
Sep 16 10:36:10 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81 via TLS tunnel)
Sep 16 10:36:10 newdvlanb radiusd[15211]: Login OK: [mhaley7] (from 
client Rich-core-WiSM-C port 13 cli 3c:e0:72:a5:b7:81)

Around there (without the OK's, I am seeing many of this style of message):

Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [jwalters38] 
(from client resnet1-WiSM-A port 13 cli a8:26:d9:34:bc:5f)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [arogers44] 
(from client Rich-core-WiSM-E port 29 cli a8:06:00:cc:6b:29)
Sep 16 09:57:56 newdvlanb radiusd[15211]: Invalid user: [mhaley7] (from 
client Rich-core-WiSM-E port 29 cli 3c:e0:72:a5:b7:81)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [bboggess3] 
(from client Rich-core-WiSM-E port 29 cli 40:a6:d9:9a:9a:53)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [cparker31] 
(from client Rich-core-WiSM-E port 29 cli 88:53:95:79:ea:0c)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [djohnson77] 
(from client Rich-core-WiSM-E port 29 cli 60:45:bd:f2:7e:a8)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [lnichols3] 
(from client Rich-core-WiSM-E port 29 cli e0:75:7d:4e:97:bb)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [oanachebe3] 
(from client Rich-core-WiSM-E port 29 cli 98:d6:f7:5f:aa:cf)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [bmcgowan6] 
(from client Rich-core-WiSM-E port 29 cli c8:aa:21:39:7e:32)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.
Sep 16 09:57:56 newdvlanb radiusd[15211]: Login incorrect: [yyu98] (from 
client Rich-core-WiSM-E port 29 cli 9c:3a:af:60:ed:bc)
Sep 16 09:57:56 newdvlanb radiusd[15211]: rlm_eap: No EAP session 
matching the State variable.

I need some guidance on what to enable, what to look for, etc. to fix 
this. I will be glad to post a full debug log (this server is very busy, 
but it's beefy beefy so should be handling things). I'll gladly post the 
multi megabyte debug log somewhere with a date/time of when things are 
occurring. Within the debug mode, I didn't see a way for me to follow a 
given thread of authentication. It looks like (forgive me if I am 
misreading) the debug messages are interleaved. There appears to be a 
process ID (5357?) but that same guide number style doesn't appear in 
the debug (allowing me to focus in on that one authentication session).

It appears to be doing ok, but these failed auth's may appear to the end 
user as a wireless session drop so I am very concerned.

[root at newdvlana 2013]# /services/snacks/lawn/util/radius-server-status.sh
Received response ID 28, code 2, length = 140
     FreeRADIUS-Total-Access-Requests = 14103212
     FreeRADIUS-Total-Access-Accepts = 2072612
     FreeRADIUS-Total-Access-Rejects = 132162
     FreeRADIUS-Total-Access-Challenges = 11896299
     FreeRADIUS-Total-Auth-Responses = 14101073
     FreeRADIUS-Total-Auth-Duplicate-Requests = 430
     FreeRADIUS-Total-Auth-Malformed-Requests = 0
     FreeRADIUS-Total-Auth-Invalid-Requests = 0
     FreeRADIUS-Total-Auth-Dropped-Requests = 1824
     FreeRADIUS-Total-Auth-Unknown-Types = 0

After finding some messages on the devel list, I saw some reference to 
memory clean up but that was a while ago so not sure how valid that 
comment/problem is in the 2.2.0 version.

How should I approach this problem?

- John Douglass, Sr. Systems IT/Architect