Talloc sanity error (3.0 release branch, reproxying from PEAP inner tunnel)
I finally got around to trying some RC code (the release_branch_3.0.0 on github) on our production configurations, after a bit of massaging got them looking like they were working, but not so much the one that re-proxies the inner tunnel contents to an internal server after unwrapping EAP-PEAP: peap { default_eap_type = mschapv2 proxy_tunneled_request_as_eap = yes copy_request_to_tunnel = no use_tunneled_reply = yes tls = eduroam-eap-tls } Any request that tries to go to the proxy causes this to happen: Wed Aug 7 11:57:35 2013 : Debug: (5) - entering if ("%{FreeRADIUS-Proxied-To}" == 127.0.0.1) {...} Wed Aug 7 11:57:35 2013 : Debug: (5) update control { Wed Aug 7 11:57:35 2013 : Debug: (5) Proxy-To-Realm := "idpi" ... Wed Aug 7 11:57:35 2013 : Debug: (5) } # update control = ok Wed Aug 7 11:57:35 2013 : Debug: (5) - if ("%{FreeRADIUS-Proxied-To}" == 127.0.0.1) returns ok Wed Aug 7 11:57:35 2013 : Debug: (5) ... skipping else for request 5: Preceding "if" was taken } # server eduroam_idp Wed Aug 7 11:57:35 2013 : Debug: (5) eap_peap : Got tunneled reply code 0 Wed Aug 7 11:57:35 2013 : Debug: PEAP: Tunneled authentication will be proxied to idpi Wed Aug 7 11:57:35 2013 : Info: talloc: access after free error - first free may be at src/main/util.c:230 Wed Aug 7 11:57:35 2013 : Info: Bad talloc magic value - access after free ... I don't know if this is of any use, being so far removed from the free(): Program received signal SIGABRT, Aborted. [Switching to Thread 0x7ffff5eb4700 (LWP 27579)] 0x0000003fe54328a5 in raise () from /lib64/libc.so.6 ... (gdb) bt #0 0x0000003fe54328a5 in raise () from /lib64/libc.so.6 #1 0x0000003fe5434085 in abort () from /lib64/libc.so.6 #2 0x00007ffff7782c3c in ?? () from /usr/lib64/libtalloc.so.2 #3 0x00007ffff7782dd8 in talloc_get_name () from /usr/lib64/libtalloc.so.2 #4 0x00007ffff77857eb in _talloc_get_type_abort () from /usr/lib64/libtalloc.so.2 #5 0x00007ffff7bb4d95 in pairnext (cursor=0x7ffff5eb2950) at src/lib/valuepair.c:290 #6 0x00007ffff7bb4b42 in pairfind (vp=0x7fffe8007d80, attr=80, vendor=0, tag=-128 '\200') at src/lib/valuepair.c:209 #7 0x00007ffff6f58d45 in mod_authenticate (instance=0x7f8b30, request=0x844e40) at src/modules/rlm_eap/rlm_eap.c:360 #8 0x0000000000421812 in call_modsingle (component=0, sp=0x81ce30, request=0x844e40) at src/main/modcall.c:311 #9 0x0000000000422f93 in modcall (component=0, c=0x81cf30, request=0x844e40) at src/main/modcall.c:782 #10 0x000000000041f4c6 in indexed_modcall (comp=0, idx=6, request=0x844e40) at src/main/modules.c:758 #11 0x0000000000421127 in process_authenticate (auth_type=6, request=0x844e40) at src/main/modules.c:1648 #12 0x000000000040c910 in rad_check_password (request=0x844e40) at src/main/auth.c:252 #13 0x000000000040cee4 in rad_authenticate (request=0x844e40) ---Type <return> to continue, or q <return> to quit--- at src/main/auth.c:490 #14 0x0000000000430b79 in request_running (request=0x844e40, action=1) at src/main/process.c:1185 #15 0x000000000042d02e in request_handler_thread (arg=0x8397c0) at src/main/threads.c:685 #16 0x0000003fe5c07851 in start_thread () from /lib64/libpthread.so.0 #17 0x0000003fe54e811d in clone () from /lib64/libc.so.6 (gdb) (gdb) up #1 0x0000003fe5434085 in abort () from /lib64/libc.so.6 (gdb) up #2 0x00007ffff7782c3c in ?? () from /usr/lib64/libtalloc.so.2 (gdb) up #3 0x00007ffff7782dd8 in talloc_get_name () from /usr/lib64/libtalloc.so.2 (gdb) up #4 0x00007ffff77857eb in _talloc_get_type_abort () from /usr/lib64/libtalloc.so.2 (gdb) up #5 0x00007ffff7bb4d95 in pairnext (cursor=0x7ffff5eb2950) at src/lib/valuepair.c:290 290 VERIFY_VP(cursor->current); (gdb) list 285 */ 286 VALUE_PAIR *pairnext(vp_cursor_t *cursor) 287 { 288 cursor->current = cursor->next; 289 if (cursor->current) { 290 VERIFY_VP(cursor->current); 291 292 /* 293 * Set this now in case 'current' gets freed before 294 * pairnext is called again. (gdb) print cursor->current $1 = (VALUE_PAIR *) 0x7fffe8007820 (gdb) print cursor->current->da $2 = (const DICT_ATTR *) 0x6c6c617420646142 (gdb) print *cursor->current->da Cannot access memory at address 0x6c6c617420646142 (gdb) up #6 0x00007ffff7bb4b42 in pairfind (vp=0x7fffe8007d80, attr=80, vendor=0, tag=-128 '\200') at src/lib/valuepair.c:209 209 i = pairnext(&cursor)) { (gdb) list 204 vp_cursor_t cursor; 205 VALUE_PAIR *i; 206 207 for (i = paircursor(&cursor, &vp); 208 i; 209 i = pairnext(&cursor)) { 210 VERIFY_VP(i); 211 if ((i->da->attr == attr) && (i->da->vendor == vendor) 212 && ((tag == TAG_ANY) || (i->da->flags.has_tag && 213 (i->tag == tag)))) { (gdb) print attr $3 = 80 (gdb) print vendor $4 = 0 (gdb) print tag $5 = -128 '\200' (gdb) print i->da->name $6 = "U" (gdb) print *(char *)i->da->name $7 = 85 'U' (gdb) print *(char *)i->da->name + 1 $8 = 86 (gdb) print *(char *)(i->da->name + 1) $9 = 115 's' (gdb) print *(char *)(i->da->name + 2) $10 = 101 'e' (gdb) print *(char *)(i->da->name + 3) $11 = 114 'r' (gdb) print *(char *)(i->da->name + 4) $12 = 45 '-' (gdb) print *(char *)(i->da->name + 5) $13 = 78 'N' (gdb) up #7 0x00007ffff6f58d45 in mod_authenticate (instance=0x7f8b30, request=0x844e40) at src/modules/rlm_eap/rlm_eap.c:360 360 vp = pairfind(request->proxy->vps, PW_MESSAGE_AUTHENTICATOR, 0, TAG_ANY); (gdb) print request $14 = (REQUEST *) 0x844e40 (gdb) print request->proxy $15 = (RADIUS_PACKET *) 0x7fffe8007bc0 (gdb) print request->proxy->vps $16 = (VALUE_PAIR *) 0x7fffe8007d80 (gdb) print request->proxy->vps->da $17 = (const DICT_ATTR *) 0x66dbf0 (gdb) print request->proxy->vps->da->name $18 = "E" (gdb) print (char *)request->proxy->vps->da->name $19 = 0x66dc00 "EAP-Message" (gdb) print (char *)request->proxy->vps->da->type $20 = 0x6 <Address 0x6 out of bounds> (gdb) print request->proxy->vps->da->type $21 = PW_TYPE_OCTETS (gdb) print (char *)request->proxy->vps->next->da->name $22 = 0x6d8b08 "FreeRADIUS-Proxied-To" (gdb) print (char *)request->proxy->vps->next->next->da->name $23 = 0x66c280 "User-Name" (gdb) print (char *)request->proxy->vps->next->next->next->da->name $24 = 0x6c6c617420646152 <Address 0x6c6c617420646152 out of bounds> (gdb) up #8 0x0000000000421812 in call_modsingle (component=0, sp=0x81ce30, request=0x844e40) at src/main/modcall.c:311 311 myresult = sp->modinst->entry->module->methods[component]( (gdb) list 306 /* 307 * For logging unresponsive children. 308 */ 309 request->module = sp->modinst->name; 310 311 myresult = sp->modinst->entry->module->methods[component]( 312 sp->modinst->insthandle, request); 313 314 request->module = ""; 315 safe_unlock(sp->modinst); (gdb) The good news is the outbound instance for RadSec proxy seems to be holding up so far. I've got another few weeks to test that before EDUROAM-US retracts RadSec service pending getting their PKI in order, so I'll try to bang on it a bit stress and endurance-wise before it goes away.
Hi,
peap { default_eap_type = mschapv2 proxy_tunneled_request_as_eap = yes copy_request_to_tunnel = no use_tunneled_reply = yes tls = eduroam-eap-tls }
okay
Any request that tries to go to the proxy causes this to happen: Wed Aug 7 11:57:35 2013 : Debug: (5) - entering if ("%{FreeRADIUS-Proxied-To}" == 127.0.0.1) {...} Wed Aug 7 11:57:35 2013 : Debug: (5) update control { Wed Aug 7 11:57:35 2013 : Debug: (5) Proxy-To-Realm := "idpi" Wed Aug 7 11:57:35 2013 : Debug: (5) } # update control = ok Wed Aug 7 11:57:35 2013 : Debug: (5) - if ("%{FreeRADIUS-Proxied-To}" == 127.0.0.1) returns ok Wed Aug 7 11:57:35 2013 : Debug: (5) ... skipping else for request 5: Preceding "if" was taken } # server eduroam_idp Wed Aug 7 11:57:35 2013 : Debug: (5) eap_peap : Got tunneled reply code 0 Wed Aug 7 11:57:35 2013 : Debug: PEAP: Tunneled authentication will be proxied to idpi Wed Aug 7 11:57:35 2013 : Info: talloc: access after free error - first free may be at src/main/util.c:230 Wed Aug 7 11:57:35 2013 : Info: Bad talloc magic value - access after free
this sample doesnt show enough of the process.. how did you configure the server...from scratch or copy pasting bits over from a 2.x ? does this 'eap' module use its own virtual_server or does it inherit the virtual_server that instigated it (you have no 'virtual_server = "blah"' line in your peap{} section...so i assume its using eduroam_idp VS for the unwrapping?) alan
A.L.M.Buxey@lboro.ac.uk [A.L.M.Buxey@lboro.ac.uk] wrote:
how did you configure the server...from scratch or copy pasting bits over from a 2.x ?
It's a mongrel, not an alteration of fresh 3.0. It was working on a pre-talloc 3.0 development branch.
does this 'eap' module use its own virtual_server or does it inherit the virtual_server that instigated it (you have no 'virtual_server = "blah"' line in your peap{} section...so i assume its using eduroam_idp VS for the unwrapping?)
There's only one incestuous server clause, and only one EAP configuration block, yes. I tried to replicate on a test server with lightly modified 3.0 stock configs. The error only happens when everything is running through the same server/eap instances, so good instincts there. Replicating it is easy: just uncomment the peap virtual-server directive and add at the top of authorize: if (Freeradius-Proxied-To == "127.0.0.1") { update control { Proxy-To-Realm = example.com } } ...and it doesn't matter that example.com defaults to home_server localhost, it does not get that far. I believe it is the way it is because at some point we were having trouble using outer.request and such between virtual servers. I'll have to test those and see if that limitation is still in effect.
Brian Julin wrote:
I tried to replicate on a test server with lightly modified 3.0 stock configs. The error only happens when everything is running through the same server/eap instances, so good instincts there. Replicating it is easy: just uncomment the peap virtual-server directive and add at the top of authorize:
if (Freeradius-Proxied-To == "127.0.0.1") { update control { Proxy-To-Realm = example.com } }
That doesn't make much sense. If it's in the "default" virtual server, the FreeRADIUS-Proxied-To attribute will never exist. If it's in the "inner-tunnel" virtual server, it will always exist, and always have that value.
...and it doesn't matter that example.com defaults to home_server localhost, it does not get that far.
Well... I tried it, and I didn't see any errors. Can you check that you're really running a *stock* binary, and a *stock* configuration?
I believe it is the way it is because at some point we were having trouble using outer.request and such between virtual servers. I'll have to test those and see if that limitation is still in effect.
All that should work... Alan DeKok.
Alan DeKok wrote: Brian Julin wrote:
I tried to replicate on a test server with lightly modified 3.0 stock configs. The error only happens when everything is running through the same server/eap instances, so good instincts there. Replicating it is easy: just uncomment the peap virtual- server directive and add at the top of authorize:
if (Freeradius-Proxied-To == "127.0.0.1") { update control { Proxy-To-Realm = example.com } }
That doesn't make much sense. If it's in the "default" virtual server, the FreeRADIUS-Proxied-To attribute will never exist. If it's in the "inner-tunnel" virtual server, it will always exist, and always have that value.
Only if you send it there with a virtual_server="inner-tunnel" statement in the peap block. This happens if you do not, as documented in the comments for that option. Ah -- maybe to replicate you can't have inner-tunnel in sites-enabled, since it has that loopback listen directive. I had swapped in proxy-inner-tunnel at some point, it appears, which does not have it.
...and it doesn't matter that example.com defaults to home_server localhost, it does not get that far.
Well... I tried it, and I didn't see any errors.
Can you check that you're really running a *stock* binary, and a *stock* configuration?
I will -- should I preferably be testing against the release git branch, or against a release tag in master, BTW?
I believe it is the way it is because at some point we were having trouble using outer.request and such between virtual servers. I'll have to test those and see if that limitation is still in effect.
All that should work...
Good.
...and it doesn't matter that example.com defaults to home_server localhost, it does not get that far.
Well... I tried it, and I didn't see any errors.
Can you check that you're really running a *stock* binary, and a *stock* configuration?
I will -- should I preferably be testing against the release git branch, or against a release tag in master, BTW?
release git branch. It contains many fixes since rc1. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team
Alan DeKok wrote:
Well... I tried it, and I didn't see any errors.
Can you check that you're really running a *stock* binary, and a *stock* configuration?
Attached is a recipe for how I replicated it (and another doublefree) on a clean system.
On 9 Aug 2013, at 16:14, Brian Julin <BJulin@clarku.edu> wrote:
Alan DeKok wrote:
Well... I tried it, and I didn't see any errors.
Can you check that you're really running a *stock* binary, and a *stock* configuration?
Attached is a recipe for how I replicated it (and another doublefree) on a clean system.
With which version of the server? Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team
On 9 Aug 2013, at 16:27, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
On 9 Aug 2013, at 16:14, Brian Julin <BJulin@clarku.edu> wrote:
Alan DeKok wrote:
Well... I tried it, and I didn't see any errors.
Can you check that you're really running a *stock* binary, and a *stock* configuration?
Attached is a recipe for how I replicated it (and another doublefree) on a clean system.
With which version of the server?
Never mind, release_branch_3_0_0 Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team
Brian Julin wrote:
Alan DeKok wrote:
Well... I tried it, and I didn't see any errors.
Can you check that you're really running a *stock* binary, and a *stock* configuration?
Attached is a recipe for how I replicated it (and another doublefree) on a clean system.
I've pushed a fix, thanks. Alan DeKok.
participants (4)
-
A.L.M.Buxey＠lboro.ac.uk -
Alan DeKok -
Arran Cudbard-Bell -
Brian Julin