On 18/06/14 18:03, Arran Cudbard-Bell wrote:
On 18 Jun 2014, at 16:45, Phil Mayers <p.mayers@IMPERIAL.AC.UK> wrote:
So run under valgrind, I'm reliably seeing use-after-free errors like this:
Invalid read of size 4 at 0x36AD402D84: talloc_get_name (talloc.c:349) by 0x36AD4057EA: _talloc_get_type_abort (talloc.c:1206) by 0x4E470EC: fr_verify_vp (debug.c:829)
Git pull.
Set envvar TALLOC_FREE_FILL=B
Talloc should now abort a little more gracefully.
If it doesn't then lib/debug.c:828
Sorry, none of that seems to work; I just get: Wed Jun 18 19:04:16 2014 : Info: talloc: access after free error - first free may be at src/lib/valuepair.c:171 Wed Jun 18 19:04:16 2014 : Info: Bad talloc magic value - access after free Wed Jun 18 19:04:16 2014 : Info: talloc abort: Bad talloc magic value - access after free Wed Jun 18 19:04:16 2014 : Info: CAUGHT SIGNAL: Aborted Wed Jun 18 19:04:16 2014 : Info: Backtrace of last 17 frames: /opt/fr3/lib/libfreeradius-radius.so(fr_fault+0xd2)[0x7f19f465d674] /opt/fr3/lib/libfreeradius-radius.so(+0xa9af)[0x7f19f465d9af] /usr/lib64/libtalloc.so.2(talloc_get_name+0x58)[0x36ad402dd8] /usr/lib64/libtalloc.so.2(_talloc_get_type_abort+0x2b)[0x36ad4057eb] /opt/fr3/lib/libfreeradius-radius.so(fr_verify_vp+0xb1)[0x7f19f465e162] /opt/fr3/lib/libfreeradius-radius.so(_fr_cursor_init+0x67)[0x7f19f465c91f] /opt/fr3/lib/libfreeradius-radius.so(fr_verify_list+0x2e)[0x7f19f465e602] /opt/fr3/lib/libfreeradius-server.so(+0x20a7d)[0x7f19f48b4a7d] /opt/fr3/lib/libfreeradius-server.so(verify_request+0xd4)[0x7f19f48b4b5b] /opt/fr3/sbin/radiusd[0x4354b9] /opt/fr3/sbin/radiusd[0x433251] /opt/fr3/lib/libfreeradius-radius.so(fr_event_run+0x142)[0x7f19f46803d9] /opt/fr3/lib/libfreeradius-radius.so(fr_event_loop+0x509)[0x7f19f4680ce6] /opt/fr3/sbin/radiusd(radius_event_process+0x26)[0x43d488] /opt/fr3/sbin/radiusd(main+0xbf5)[0x42a1a5] /lib64/libc.so.6(__libc_start_main+0xfd)[0x379dc1ed1d] /opt/fr3/sbin/radiusd[0x40cd09] ,..and then it calls panic_action and aborts. I can reliably reproduce it now; the trick seems to be to set a bunch of eapol_test running in a loop, stop the server and start it. I think this means it's concurrency/racey. I have some circumstantial evidence that eap_ttls is implicated, and that it might be related to the handling of the fake requests for the inner tunnel - but it's very circumstantial. The heap corruption makes it really hard to be sure of anything - *someone* is trampling over memory they shouldn't, but valgrind seems to get very very confused when this happens, and swamps me with messages.