On Tue, 21 Feb 2012 01:09:19 -0800 Alan DeKok <aland@deployingradius.com> wrote:
From a quick inspection, the cause is likely that a connection is still in use by the module, but in another thread. When it releases a connection that's still in use, bad things happen.
Would that still be possible if I was starting with -t? This is in a test environment, so I'm certain there weren't any clients being processed at the time of restart.
I've committed a fix which avoids the problem. The side effect is that the server exits while LDAP connections are still open. It's imperfect, but there are few better choices.
Hmm, I pulled the two relevant patches but I still get the segfault: (gdb) run -t -f -d /etc/raddb Starting program: /tmp/radiusd/sbin/radiusd -t -f -d /etc/raddb [Thread debugging using libthread_db enabled] [New Thread 0x7ffff2eeb700 (LWP 1219)] [Thread 0x7ffff2eeb700 (LWP 1219) exited] Program received signal SIGTERM, Terminated. 0x00007ffff6694f13 in select () from /lib64/libc.so.6 (gdb) continue Continuing. Program received signal SIGSEGV, Segmentation fault. 0x00007ffff3e0f480 in ?? () (gdb) bt #0 0x00007ffff3e0f480 in ?? () #1 0x00007ffff71d9dea in ?? () from /usr/lib64/libcrypto.so.1.0.0 #2 0x00007ffff755a12d in SSL_free () from /usr/lib64/libssl.so.1.0.0 #3 0x00007ffff548b1da in ?? () from /usr/lib64/libldap_r-2.4.so.2 #4 0x00007ffff524df59 in ber_sockbuf_remove_io () from /usr/lib64/liblber-2.4.so.2 #5 0x00007ffff524dfed in ber_int_sb_destroy () from /usr/lib64/liblber-2.4.so.2 #6 0x00007ffff524e06c in ber_sockbuf_free () from /usr/lib64/liblber-2.4.so.2 #7 0x00007ffff546fb2c in ldap_ld_free () from /usr/lib64/libldap_r-2.4.so.2 #8 0x00007ffff56a50eb in ldap_detach (instance=<value optimized out>) at rlm_ldap.c:2590 #9 0x00000000004186fe in module_instance_free (data=<value optimized out>) at modules.c:383 #10 0x00007ffff7bcee4d in FreeWalker (tree=0x7973e0, X=0x79a000) at rbtree.c:63 #11 0x00007ffff7bcee8e in rbtree_free (tree=0x9b1e10) at rbtree.c:74 #12 0x0000000000419480 in detach_modules () at modules.c:431 #13 0x000000000041d17c in main (argc=<value optimized out>, argv=<value optimized out>) at radiusd.c:456 Interestingly, I tried running it with -X and it fails elsewhere (I didn't try this before the patches so it might be the same, sorry): (gdb) run -X -t -f -d /etc/raddb ... Program received signal SIGTERM, Terminated. 0x00007ffff6694f13 in select () from /lib64/libc.so.6 (gdb) continue Continuing. Polling for detail file /var/log/radius/radacct/forward/detail-* Detail listener /var/log/radius/radacct/forward/detail-* state unopened signalled 0 waiting 0.800610 sec Waking up in 0.8 seconds. Signalled to terminate Exiting normally. Program received signal SIGSEGV, Segmentation fault. rbtree_find (tree=0x7bfbc0, Data=0x9c1ac0) at rbtree.c:479 479 int result = tree->Compare(Data, Current->Data); (gdb) bt #0 rbtree_find (tree=0x7bfbc0, Data=0x9c1ac0) at rbtree.c:479 #1 0x00007ffff7bcf0f9 in rbtree_deletebydata (tree=0x7bfbc0, data=0x9c1ac0) at rbtree.c:457 #2 0x00007ffff462a894 in eap_handler_free (inst=0x7a0030, handler=0x9c1ac0) at mem.c:154 #3 0x00007ffff462afc4 in eaplist_free (inst=0x7a0030) at mem.c:266 #4 0x00007ffff4628cbf in eap_detach (instance=<value optimized out>) at rlm_eap.c:66 #5 0x00000000004186fe in module_instance_free (data=<value optimized out>) at modules.c:383 #6 0x00007ffff7bcee4d in FreeWalker (tree=0x7972d0, X=0x7bc660) at rbtree.c:63 #7 0x00007ffff7bcee2d in FreeWalker (tree=0x7972d0, X=0x799ef0) at rbtree.c:60 #8 0x00007ffff7bcee8e in rbtree_free (tree=0x7bfbc0) at rbtree.c:74 #9 0x0000000000419480 in detach_modules () at modules.c:431 #10 0x000000000041d17c in main (argc=<value optimized out>, argv=<value optimized out>) at radiusd.c:456 I tried it again with "run -X -t -f -d /etc/raddb" and it gave a backtrace that pointed at LDAP again, but it was also able to shut down all of the rlm_sqsl_mysql handles before segfaulting. Is there any other output/debug stuff that would be helpful?