I am running FreeRADIUS 1.1.1 on a Fedora Core 4 server (kernel 2.6.11-1.1369_FC4smp) to authenticate using EAP-SIM. After ~400 successful auths at 20 requests/second the radiusd service encounters a segmentation fault. The output of the gdb dump is as follows: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1208572224 (LWP 9805)] 0x0072ec33 in _int_malloc () from /lib/libc.so.6 (gdb) bt #0 0x0072ec33 in _int_malloc () from /lib/libc.so.6 #1 0x00730792 in malloc () from /lib/libc.so.6 #2 0x005007e4 in eap_compose (handler=0x8fb8220) at eap.c:395 #3 0x004ffa94 in eap_authenticate (instance=0x8f8e4b8, request=0x8fbe648) at rlm_eap.c:341 #4 0x08053009 in modcall () #5 0x0805351d in modcall () #6 0x0805312d in modcall () #7 0x080525ba in find_module_instance () #8 0x0804c532 in rad_check_password () #9 0x0804cb03 in rad_authenticate () #10 0x08054c0a in rad_respond () #11 0x08056287 in main () (gdb) I have another FreeRADIUS 1.0.3 server running on a Red Hat 9 server (kernel 2.4.20-8) that has had no problems running this kind of traffic. Any help in diagnosing the reason why I'm encountering a fault in malloc would be much appreciated. Thanks, Nik
Nikolas Thoman <nikthoman@yahoo.com> wrote:
Any help in diagnosing the reason why I'm encountering a fault in malloc would be much appreciated.
It usually happens because something else in the code is over-writing a buffer, or writing to free'd memory. Run the server under valgrind to see what's going on. You'll have to pass special options to work around the infinite SSL warnings, but those warnings can be ignored. Alan DeKok.
Alan DeKok wrote:
Nikolas Thoman <nikthoman@yahoo.com> wrote:
Any help in diagnosing the reason why I'm encountering a fault in malloc would be much appreciated.
It usually happens because something else in the code is over-writing a buffer, or writing to free'd memory.
Run the server under valgrind to see what's going on. You'll have to pass special options to work around the infinite SSL warnings, but those warnings can be ignored.
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Think i have the same problem. I normally use EAP-PEAP but i couldnt get the server to segfault in valgrind with that. Think it was openssl that grinded it to a halt. Tried with EAP-MD5 instead and it produced the "desired" result. Attached the output from valgrind. Bjarni Hardarson ==15822== Memcheck, a memory error detector. ==15822== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al. ==15822== Using LibVEX rev 1575, a library for dynamic binary translation. ==15822== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP. ==15822== Using valgrind-3.1.1, a dynamic binary instrumentation framework. ==15822== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al. ==15822== --15822-- Command line --15822-- /usr/local/sbin/radiusd --15822-- -f --15822-- -i --15822-- 127.0.0.1 --15822-- Startup, with flags: --15822-- -v --15822-- --error-limit=no --15822-- --leak-check=full --15822-- --show-reachable=yes --15822-- --trace-children=yes --15822-- Contents of /proc/version: --15822-- Linux version 2.6.5-7.252-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Tue Feb 14 11:11:04 UTC 2006 --15822-- Arch and subarch: X86, x86-sse2 --15822-- Valgrind library directory: /usr/local/lib/valgrind --15822-- Reading syms from /lib/ld-2.3.3.so (0x4000000) --15822-- Reading syms from /usr/local/sbin/radiusd (0x8048000) --15822-- Reading syms from /usr/local/lib/valgrind/x86-linux/memcheck (0xB0000000) --15822-- object doesn't have a dynamic symbol table --15822-- Reading suppressions file: /usr/local/lib/valgrind/default.supp --15822-- REDIR: 0x4012CF0 (index) redirected to 0xB001B3CE (vgPlain_x86_linux_REDIR_FOR_index) --15822-- Reading syms from /usr/local/lib/valgrind/x86-linux/vgpreload_core.so (0x4018000) --15822-- Reading syms from /usr/local/lib/valgrind/x86-linux/vgpreload_memcheck.so (0x401A000) --15822-- REDIR: 0x4012E90 (strlen) redirected to 0x401D210 (strlen) --15822-- Reading syms from /lib/libnsl.so.1 (0x4027000) --15822-- Reading syms from /lib/libresolv.so.2 (0x403B000) --15822-- Reading syms from /lib/tls/libpthread.so.0 (0x404E000) --15822-- Reading syms from /usr/local/lib/libradius-1.1.1.so (0x405E000) --15822-- Reading syms from /lib/libcrypt.so.1 (0x4074000) --15822-- Reading syms from /usr/lib/libsnmp.so.5.1.3 (0x40A5000) --15822-- Reading syms from /usr/lib/libcrypto.so.0.9.7 (0x4142000) --15822-- Reading syms from /usr/lib/libltdl.so.3.1.0 (0x4232000) --15822-- Reading syms from /lib/libdl.so.2 (0x423A000) --15822-- Reading syms from /lib/tls/libc.so.6 (0x423D000) --15822-- REDIR: 0x4000970 (_dl_sysinfo_int80) redirected to 0xB001B3CB (???) --15822-- REDIR: 0x42A50D0 (rindex) redirected to 0x401CE70 (rindex) --15822-- REDIR: 0x42A1980 (malloc) redirected to 0x401B5D2 (malloc) --15822-- REDIR: 0x42A63F0 (memcpy) redirected to 0x401D550 (memcpy) --15822-- REDIR: 0x42A4D10 (strlen) redirected to 0x401D1F0 (strlen) --15822-- REDIR: 0x42A4630 (index) redirected to 0x401CF60 (index) --15822-- REDIR: 0x429FAA0 (free) redirected to 0x401C0FB (free) --15822-- REDIR: 0x42A4810 (strcpy) redirected to 0x401D250 (strcpy) --15822-- REDIR: 0x42A47A0 (strcmp) redirected to 0x401D4C0 (strcmp) --15822-- REDIR: 0x42A4DC0 (strnlen) redirected to 0x401D1C0 (strnlen) --15822-- REDIR: 0x42A4480 (strcat) redirected to 0x401CFF0 (strcat) Wed May 3 22:48:16 2006 : Info: Starting - reading configuration files ... --15822-- REDIR: 0x42A59D0 (memchr) redirected to 0x401D520 (memchr) ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x4067500: lrad_rand_seed (radius.c:2403) ==15822== by 0x4061E11: my_dict_init (dict.c:890) ==15822== by 0x4062C16: dict_init (dict.c:1111) ==15822== by 0x8051965: read_radius_conf_file (mainconfig.c:1239) ==15822== by 0x80519F9: read_mainconfig (mainconfig.c:1272) ==15822== by 0x805661D: main (radiusd.c:936) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x4067500: lrad_rand_seed (radius.c:2403) ==15822== by 0x4061E11: my_dict_init (dict.c:890) ==15822== by 0x40622A6: my_dict_init (dict.c:930) ==15822== by 0x4062C16: dict_init (dict.c:1111) ==15822== by 0x8051965: read_radius_conf_file (mainconfig.c:1239) ==15822== by 0x80519F9: read_mainconfig (mainconfig.c:1272) ==15822== by 0x805661D: main (radiusd.c:936) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x4067500: lrad_rand_seed (radius.c:2403) ==15822== by 0x4061E11: my_dict_init (dict.c:890) ==15822== by 0x40622A6: my_dict_init (dict.c:930) ==15822== by 0x40622A6: my_dict_init (dict.c:930) ==15822== by 0x4062C16: dict_init (dict.c:1111) ==15822== by 0x8051965: read_radius_conf_file (mainconfig.c:1239) ==15822== by 0x80519F9: read_mainconfig (mainconfig.c:1272) ==15822== by 0x805661D: main (radiusd.c:936) --15822-- REDIR: 0x42A6C70 (rawmemchr) redirected to 0x401DA70 (rawmemchr) --15822-- REDIR: 0x42A4F10 (strncmp) redirected to 0x401D460 (strncmp) --15822-- REDIR: 0x42A5020 (strncpy) redirected to 0x401D320 (strncpy) --15822-- REDIR: 0x42A6D40 (strchrnul) redirected to 0x401DA50 (strchrnul) --15822-- REDIR: 0x42A60C0 (stpcpy) redirected to 0x401D720 (stpcpy) --15822-- REDIR: 0x42A2820 (calloc) redirected to 0x401C8B7 (calloc) --15822-- Reading syms from /lib/libnss_files.so.2 (0x4560000) --15822-- REDIR: 0x42A1ED0 (realloc) redirected to 0x401C962 (realloc) --15822-- Reading syms from /usr/local/lib/rlm_exec-1.1.1.so (0x4020000) --15822-- REDIR: 0x4013260 (stpcpy) redirected to 0x401D800 (stpcpy) --15822-- Reading syms from /usr/local/lib/rlm_expr-1.1.1.so (0x4023000) --15822-- Reading syms from /usr/local/lib/rlm_always-1.1.1.so (0x4025000) --15822-- Reading syms from /usr/local/lib/rlm_pap-1.1.1.so (0x4558000) --15822-- Reading syms from /usr/local/lib/rlm_mschap-1.1.1.so (0x4569000) --15822-- Reading syms from /usr/local/lib/rlm_ldap-1.1.1.so (0x456F000) --15822-- Reading syms from /usr/lib/libldap_r.so.199.3.24 (0x4582000) --15822-- Reading syms from /usr/lib/liblber.so.199.3.24 (0x45BB000) --15822-- Reading syms from /usr/lib/libsasl2.so.2.0.18 (0x45C8000) --15822-- Reading syms from /usr/lib/libssl.so.0.9.7 (0x45DD000) --15822-- Reading syms from /usr/local/lib/rlm_eap-1.1.1.so (0x457A000) --15822-- Reading syms from /usr/local/lib/libeap-1.1.1.so (0x460D000) --15822-- Reading syms from /usr/local/lib/rlm_eap_leap-1.1.1.so (0x455B000) --15822-- Reading syms from /usr/local/lib/rlm_eap_md5-1.1.1.so (0x4615000) --15822-- Reading syms from /usr/local/lib/rlm_eap_tls-1.1.1.so (0x4618000) --15822-- REDIR: 0x42A5ED0 (memset) redirected to 0x401D9C0 (memset) --15822-- Reading syms from /usr/local/lib/rlm_eap_ttls-1.1.1.so (0x471C000) --15822-- Reading syms from /usr/local/lib/rlm_eap_mschapv2-1.1.1.so (0x4721000) --15822-- Reading syms from /usr/local/lib/rlm_eap_peap-1.1.1.so (0x4724000) --15822-- Reading syms from /usr/local/lib/rlm_preprocess-1.1.1.so (0x4729000) --15822-- Reading syms from /usr/local/lib/rlm_detail-1.1.1.so (0x472C000) --15822-- Reading syms from /usr/local/lib/rlm_files-1.1.1.so (0x472F000) --15822-- Reading syms from /usr/local/lib/rlm_acct_unique-1.1.1.so (0x4732000) --15822-- Reading syms from /usr/local/lib/rlm_unix-1.1.1.so (0x4735000) --15822-- Reading syms from /usr/local/lib/rlm_radutmp-1.1.1.so (0x473A000) --15822-- REDIR: 0x42A5E60 (memmove) redirected to 0x401D9F0 (memmove) ==15822== ==15822== Thread 8: ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x406765B: lrad_rand (radius.c:2434) ==15822== by 0x457DBD4: generate_state (state.c:126) ==15822== by 0x457D866: eaplist_add (mem.c:183) ==15822== by 0x457BD45: eap_authenticate (rlm_eap.c:365) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x457C24F: eap_handler_cmp (rlm_eap.c:86) ==15822== by 0x406A9D7: rbtree_insert (rbtree.c:253) ==15822== by 0x457D904: eaplist_add (mem.c:217) ==15822== by 0x457BD45: eap_authenticate (rlm_eap.c:365) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x406A9DD: rbtree_insert (rbtree.c:254) ==15822== by 0x457D904: eaplist_add (mem.c:217) ==15822== by 0x457BD45: eap_authenticate (rlm_eap.c:365) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x406A9E7: rbtree_insert (rbtree.c:271) ==15822== by 0x457D904: eaplist_add (mem.c:217) ==15822== by 0x457BD45: eap_authenticate (rlm_eap.c:365) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x457C24F: eap_handler_cmp (rlm_eap.c:86) ==15822== by 0x406AA33: rbtree_insert (rbtree.c:287) ==15822== by 0x457D904: eaplist_add (mem.c:217) ==15822== by 0x457BD45: eap_authenticate (rlm_eap.c:365) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x406AA39: rbtree_insert (rbtree.c:287) ==15822== by 0x457D904: eaplist_add (mem.c:217) ==15822== by 0x457BD45: eap_authenticate (rlm_eap.c:365) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== ==15822== Syscall param socketcall.sendto(msg) points to uninitialised byte(s) ==15822== at 0x4058AE8: sendto (in /lib/tls/libpthread.so.0) ==15822== by 0x805600E: rad_respond (radiusd.c:1788) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== by 0x42F721D: clone (in /lib/tls/libc.so.6) ==15822== by 0x5544BAF: ??? ==15822== Address 0x471A644 is 4 bytes inside a block of size 64 alloc'd ==15822== at 0x401B651: malloc (vg_replace_malloc.c:149) ==15822== by 0x4069A05: rad_encode (radius.c:704) ==15822== by 0x406A0EE: rad_send (radius.c:872) ==15822== by 0x805600E: rad_respond (radiusd.c:1788) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== by 0x42F721D: clone (in /lib/tls/libc.so.6) ==15822== by 0x5544BAF: ??? ==15822== ==15822== Thread 5: ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x457C24F: eap_handler_cmp (rlm_eap.c:86) ==15822== by 0x406A35D: rbtree_find (rbtree.c:458) ==15822== by 0x457D68B: eaplist_find (mem.c:306) ==15822== by 0x457C2F7: eap_handler (eap.c:993) ==15822== by 0x457BC11: eap_authenticate (rlm_eap.c:230) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x406A365: rbtree_find (rbtree.c:460) ==15822== by 0x457D68B: eaplist_find (mem.c:306) ==15822== by 0x457C2F7: eap_handler (eap.c:993) ==15822== by 0x457BC11: eap_authenticate (rlm_eap.c:230) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== ==15822== More than 100 errors detected. Subsequent errors ==15822== will still be recorded, but in less detail than before. ==15822== ==15822== Conditional jump or move depends on uninitialised value(s) ==15822== at 0x406A369: rbtree_find (rbtree.c:463) ==15822== by 0x457D68B: eaplist_find (mem.c:306) ==15822== by 0x457C2F7: eap_handler (eap.c:993) ==15822== by 0x457BC11: eap_authenticate (rlm_eap.c:230) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== ==15822== Thread 1: ==15822== Syscall param socketcall.sendto(msg) points to uninitialised byte(s) ==15822== at 0x4058AE8: sendto (in /lib/tls/libpthread.so.0) ==15822== by 0x8057B52: main (radiusd.c:546) ==15822== Address 0x6A42D2C is 4 bytes inside a block of size 64 alloc'd ==15822== at 0x401B651: malloc (vg_replace_malloc.c:149) ==15822== by 0x4069A05: rad_encode (radius.c:704) ==15822== by 0x406A0EE: rad_send (radius.c:872) ==15822== by 0x805600E: rad_respond (radiusd.c:1788) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== by 0x42F721D: clone (in /lib/tls/libc.so.6) ==15822== by 0x4B3FBAF: ??? ==15822== ==15822== Thread 24: ==15822== Invalid write of size 4 ==15822== at 0x457D797: eaplist_find (mem.c:332) ==15822== by 0x457C2F7: eap_handler (eap.c:993) ==15822== by 0x457BC11: eap_authenticate (rlm_eap.c:230) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== Address 0x5A17BEC is 4 bytes inside a block of size 72 free'd ==15822== at 0x401C178: free (vg_replace_malloc.c:235) ==15822== by 0x457D564: eap_handler_free (mem.c:142) ==15822== by 0x457BF3E: eap_authenticate (rlm_eap.c:268) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== ==15822== Thread 1: ==15822== Invalid read of size 4 ==15822== at 0x8058536: request_cmp (request_list.c:175) ==15822== by 0x406A35D: rbtree_find (rbtree.c:458) ==15822== by 0x8058FCE: rl_delete (request_list.c:472) ==15822== by 0x80591F9: refresh_request (request_list.c:1052) ==15822== by 0x80597BE: rl_clean_list (request_list.c:1402) ==15822== by 0x80573B3: main (radiusd.c:1462) ==15822== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==15822== ==15822== Process terminating with default action of signal 11 (SIGSEGV) ==15822== Access not within mapped region at address 0x0 ==15822== at 0x8058536: request_cmp (request_list.c:175) ==15822== by 0x406A35D: rbtree_find (rbtree.c:458) ==15822== by 0x8058FCE: rl_delete (request_list.c:472) ==15822== by 0x80591F9: refresh_request (request_list.c:1052) ==15822== by 0x80597BE: rl_clean_list (request_list.c:1402) ==15822== by 0x80573B3: main (radiusd.c:1462) --15822-- discard syms at 0x4560000-0x4569000 in /lib/libnss_files.so.2 due to munmap() ==15822== ==15822== ERROR SUMMARY: 385659 errors from 104 contexts (suppressed: 79 from 2) ==15822== ==15822== 1 errors in context 1 of 104: ==15822== Invalid read of size 4 ==15822== at 0x8058536: request_cmp (request_list.c:175) ==15822== by 0x406A35D: rbtree_find (rbtree.c:458) ==15822== by 0x8058FCE: rl_delete (request_list.c:472) ==15822== by 0x80591F9: refresh_request (request_list.c:1052) ==15822== by 0x80597BE: rl_clean_list (request_list.c:1402) ==15822== by 0x80573B3: main (radiusd.c:1462) ==15822== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Bjarni Hardarson <freeradius@hardarson.se> wrote:
Think i have the same problem. I normally use EAP-PEAP but i couldnt get the server to segfault in valgrind with that. Think it was openssl that grinded it to a halt. Tried with EAP-MD5 instead and it produced the "desired" result.
I'm running FC4 on one of my machines, and I don't see the same problem with the CVS head of 1.1 (i.e. 1.1.1 + a few patches that shouldn't affect this). And looking at what valgrind *is* complaining about, I can't see for the life of me why the code depends on uninitialized values. If I walk back up the callstack valgrind shows, the code clearly initializes the data structures. My next question is where did you get the server binary from? Source? An RPM? If you could build & install the server from source, that may help. Maybe there's a problem with the RPM. Alan DeKok.
Alan DeKok wrote:
I'm running FC4 on one of my machines, and I don't see the same problem with the CVS head of 1.1 (i.e. 1.1.1 + a few patches that shouldn't affect this).
I forgot to mention that i am not running FC4. Sorry about that. Here is my info from /proc/version Linux version 2.6.5-7.252-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Tue Feb 14 11:11:04 UTC 2006
And looking at what valgrind *is* complaining about, I can't see for the life of me why the code depends on uninitialized values. If I walk back up the callstack valgrind shows, the code clearly initializes the data structures.
My next question is where did you get the server binary from? Source? An RPM?
I built the server binary from the released 1.1.1 source. ./configure --with-edir --enable-developer Bjarni Hardarson
Bjarni Hardarson <freeradius@hardarson.se> wrote:
I built the server binary from the released 1.1.1 source. ./configure --with-edir --enable-developer
The only difference I can see is that I'm using gcc version 4. I have another machine with 3.2, I'll see if I can try that. Maybe it's a gcc bug? Alan DeKok.
Alan DeKok wrote:
The only difference I can see is that I'm using gcc version 4. I have another machine with 3.2, I'll see if I can try that.
Nope. Doesn't seem to make any difference.
Tried with 1.0.5 and it handled the load just fine. I will try to get FC4 and run the same stress test on that. Bjarni Hardarson
On Fri, 2006-05-05 at 14:00 -0400, Alan DeKok wrote:
Bjarni Hardarson <freeradius@hardarson.se> wrote:
Think i have the same problem. I normally use EAP-PEAP but i couldnt get the server to segfault in valgrind with that. Think it was openssl that grinded it to a halt. Tried with EAP-MD5 instead and it produced the "desired" result.
I'm running FC4 on one of my machines, and I don't see the same problem with the CVS head of 1.1 (i.e. 1.1.1 + a few patches that shouldn't affect this).
And looking at what valgrind *is* complaining about, I can't see for the life of me why the code depends on uninitialized values. If I walk back up the callstack valgrind shows, the code clearly initializes the data structures.
In general, a lot of 'unitialized value' errors that valgrind complains about come from the network libs. If you ignore the 'unitialized value' errors in the valgrind log then you come to the real errors, 'Invalid Write', 'Invalid Read' to/from memory areas that aren't part of the server or were previously freed. The first real error I can see in the log is ==15822== Thread 24: ==15822== Invalid write of size 4 ==15822== at 0x457D797: eaplist_find (mem.c:332) ==15822== by 0x457C2F7: eap_handler (eap.c:993) ==15822== by 0x457BC11: eap_authenticate (rlm_eap.c:230) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) ==15822== Address 0x5A17BEC is 4 bytes inside a block of size 72 free'd ==15822== at 0x401C178: free (vg_replace_malloc.c:235) ==15822== by 0x457D564: eap_handler_free (mem.c:142) ==15822== by 0x457BF3E: eap_authenticate (rlm_eap.c:268) ==15822== by 0x8054775: modcall (modcall.c:236) ==15822== by 0x8054CF8: call_one (modcall.c:269) ==15822== by 0x80549B9: modcall (modcall.c:324) ==15822== by 0x8052D6F: indexed_modcall (modules.c:469) ==15822== by 0x804CC27: rad_check_password (auth.c:367) ==15822== by 0x804D16B: rad_authenticate (auth.c:662) ==15822== by 0x8055D05: rad_respond (radiusd.c:1642) ==15822== by 0x805C5B6: request_handler_thread (threads.c:517) ==15822== by 0x4053CB6: start_thread (in /lib/tls/libpthread.so.0) which is a write to some memory that we previously held, but have since free'd. Regards, Stuart Auchterlonie ======================================================================= Homechoice is a trading name of Video Networks Limited of 205 Holland Park Avenue, London W11 4XB and registered in England and Wales (No. 2740910). This email may contain confidential and privileged information and is intended for the named or authorised recipients only. If you are not the named or authorised recipient of this email, please note that any copying, distribution, disclosure or use of its contents is strictly prohibited. If you have received this email in error please notify the sender immediately and then destroy it. The views expressed in this email are not necessarily those held by Video Networks Limited and we do not accept any liability for any action taken in reliance on the contents of this message. We do not guarantee that the integrity of this email has been maintained, nor that it is free of viruses, interceptions or interference. _______________________________________________________________________ This email has been scanned for all known viruses by the MessageLabs Email Security System. _______________________________________________________________________
Stuart Auchterlonie <stuart.auchterlonie@homechoice.net> wrote:
If you ignore the 'unitialized value' errors in the valgrind log then you come to the real errors, 'Invalid Write', 'Invalid Read' to/from memory areas that aren't part of the server or were previously freed.
Ah, OK. That looks like it's a bug that's been there a while. It only happens when TLS is being used inside of PEAP, apparently. Try this patch. If it works, I'll add it into 1.1.2 Alan DeKok. -------------- Index: src/modules/rlm_eap/eap.c =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/eap.c,v retrieving revision 1.52.4.1 diff -u -r1.52.4.1 eap.c --- src/modules/rlm_eap/eap.c 6 Feb 2006 16:23:49 -0000 1.52.4.1 +++ src/modules/rlm_eap/eap.c 8 May 2006 17:13:30 -0000 @@ -1116,7 +1116,7 @@ if (handler->eap_ds == NULL) { free(*eap_packet_p); *eap_packet_p = NULL; - eap_handler_free(handler); + eaplist_delete(handler); return NULL; } Index: src/modules/rlm_eap/eap.h =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/eap.h,v retrieving revision 1.27.4.1 diff -u -r1.27.4.1 eap.h --- src/modules/rlm_eap/eap.h 6 Feb 2006 16:23:50 -0000 1.27.4.1 +++ src/modules/rlm_eap/eap.h 8 May 2006 17:13:31 -0000 @@ -121,6 +121,9 @@ int status; int stage; + + int in_list; + void *inst; } EAP_HANDLER; /* Index: src/modules/rlm_eap/mem.c =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/mem.c,v retrieving revision 1.14.4.1 diff -u -r1.14.4.1 mem.c --- src/modules/rlm_eap/mem.c 6 Feb 2006 16:23:51 -0000 1.14.4.1 +++ src/modules/rlm_eap/mem.c 8 May 2006 17:13:31 -0000 @@ -111,8 +111,8 @@ { EAP_HANDLER *handler; - handler = rad_malloc(sizeof(EAP_HANDLER)); - memset(handler, 0, sizeof(EAP_HANDLER)); + handler = rad_malloc(sizeof(*handler)); + memset(handler, 0, sizeof(*handler)); return handler; } @@ -121,6 +121,12 @@ if (!handler) return; + /* + * This is a hack to work around request_data_add not + * taking an "inst" pointer + */ + if (handler->inst) return eaplist_delete(handler->inst, handler); + if (handler->identity) { free(handler->identity); handler->identity = NULL; @@ -195,6 +201,7 @@ */ handler->timestamp = handler->request->timestamp; handler->status = 1; + handler->inst = inst; memcpy(handler->state, state->strvalue, sizeof(handler->state)); handler->src_ipaddr = handler->request->packet->src_ipaddr; @@ -216,6 +223,8 @@ */ status = rbtree_insert(inst->session_tree, handler); + handler->in_list = 1; + if (status) { EAP_HANDLER *prev; @@ -243,6 +252,40 @@ return 1; } + +static void eaplist_delete_internal(rlm_eap_t *inst, EAP_HANDLER *handler) +{ + + rbtree_deletebydata(inst->session_tree, handler); + + inst->session_head = handler->next; + if (handler->next) handler->next->prev = NULL; + handler->inst = NULL; + eap_handler_free(handler); +} + + +/* + * Delete a handler from the list. + */ +void eaplist_delete(rlm_eap_t *inst, EAP_HANDLER *handler) +{ + if (!handler->in_list || !handler->inst) { + handler->inst = NULL; + eap_handler_free(handler); + return; + } + + /* + * Playing with a data structure shared among threads + * means that we need a lock, to avoid conflict. + */ + pthread_mutex_lock(&(inst->session_mutex)); + eaplist_delete_internal(inst, handler); + pthread_mutex_unlock(&(inst->session_mutex)); +} + + /* * Find a a previous EAP-Request sent by us, which matches * the current EAP-Response. @@ -292,13 +335,7 @@ handler = inst->session_head; if (handler && ((request->timestamp - handler->timestamp) > inst->timer_limit)) { - node = rbtree_find(inst->session_tree, handler); - rad_assert(node != NULL); - rbtree_delete(inst->session_tree, node); - - inst->session_head = handler->next; - if (handler->next) handler->next->prev = NULL; - eap_handler_free(handler); + eaplist_delete(inst, handler); } } Index: src/modules/rlm_eap/rlm_eap.c =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/rlm_eap.c,v retrieving revision 1.26.2.1.2.1 diff -u -r1.26.2.1.2.1 rlm_eap.c --- src/modules/rlm_eap/rlm_eap.c 6 Feb 2006 16:23:52 -0000 1.26.2.1.2.1 +++ src/modules/rlm_eap/rlm_eap.c 8 May 2006 17:13:31 -0000 @@ -244,7 +244,7 @@ case PW_EAP_PEAP: DEBUG2(" rlm_eap: Unable to tunnel TLS inside of TLS"); eap_fail(handler); - eap_handler_free(handler); + eaplist_delete(inst, handler); return RLM_MODULE_INVALID; break; @@ -265,7 +265,7 @@ */ if (rcode == EAP_INVALID) { eap_fail(handler); - eap_handler_free(handler); + eaplist_delete(inst, handler); DEBUG2(" rlm_eap: Failed in EAP select"); return RLM_MODULE_INVALID; } @@ -367,7 +367,7 @@ } else { DEBUG2(" rlm_eap: Freeing handler"); /* handler is not required any more, free it now */ - eap_handler_free(handler); + eaplist_delete(handler); } /* @@ -496,7 +496,7 @@ REQUEST_DATA_EAP_TUNNEL_CALLBACK); if (!data) { radlog(L_ERR, "rlm_eap: Failed to retrieve callback for tunneled session!"); - eap_handler_free(handler); + eaplist_delete(handler); return RLM_MODULE_FAIL; } @@ -507,7 +507,7 @@ free(data); if (rcode == 0) { eap_fail(handler); - eap_handler_free(handler); + eaplist_delete(handler); return RLM_MODULE_REJECT; } @@ -528,7 +528,7 @@ } else { /* couldn't have been LEAP, there's no tunnel */ DEBUG2(" rlm_eap: Freeing handler"); /* handler is not required any more, free it now */ - eap_handler_free(handler); + eaplist_delete(handler); } /* Index: src/modules/rlm_eap/rlm_eap.h =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/rlm_eap.h,v retrieving revision 1.13.4.1 diff -u -r1.13.4.1 rlm_eap.h --- src/modules/rlm_eap/rlm_eap.h 6 Feb 2006 16:23:53 -0000 1.13.4.1 +++ src/modules/rlm_eap/rlm_eap.h 8 May 2006 17:13:31 -0000 @@ -101,6 +101,7 @@ EAP_HANDLER *eaplist_find(rlm_eap_t *inst, REQUEST *request, eap_packet_t *eap_packet); void eaplist_free(rlm_eap_t *inst); +void eaplist_delete(rlm_eap_t *inst, EAP_HANDLER *handler); /* State */ void generate_key(void);
Alan DeKok wrote:
Ah, OK. That looks like it's a bug that's been there a while. It only happens when TLS is being used inside of PEAP, apparently.
I got the output from valgrind using EAP-MD5.
Try this patch. If it works, I'll add it into 1.1.2
Tried the patch but the build fails with the following errors. rlm_eap.c: In function 'eap_authenticate': rlm_eap.c:370: error: too few arguments to function 'eaplist_delete' rlm_eap.c:499: warning: passing argument 1 of 'eaplist_delete' from incompatible pointer type rlm_eap.c:499: error: too few arguments to function 'eaplist_delete' rlm_eap.c:510: warning: passing argument 1 of 'eaplist_delete' from incompatible pointer type rlm_eap.c:510: error: too few arguments to function 'eaplist_delete' rlm_eap.c:531: warning: passing argument 1 of 'eaplist_delete' from incompatible pointer type rlm_eap.c:531: error: too few arguments to function 'eaplist_delete' Wish i could help you with the code but i dont speak C. Bjarni Hardarson
Bjarni Hardarson <freeradius@hardarson.se> wrote:
Tried the patch but the build fails with the following errors.
Sorry, sent the wrong patch. Alan DeKok ---- Index: src/modules/rlm_eap/eap.h =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/eap.h,v retrieving revision 1.27.4.1 diff -u -r1.27.4.1 eap.h --- src/modules/rlm_eap/eap.h 6 Feb 2006 16:23:50 -0000 1.27.4.1 +++ src/modules/rlm_eap/eap.h 8 May 2006 21:39:57 -0000 @@ -121,6 +121,9 @@ int status; int stage; + + int in_list; + void *inst; } EAP_HANDLER; /* Index: src/modules/rlm_eap/mem.c =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/mem.c,v retrieving revision 1.14.4.1 diff -u -r1.14.4.1 mem.c --- src/modules/rlm_eap/mem.c 6 Feb 2006 16:23:51 -0000 1.14.4.1 +++ src/modules/rlm_eap/mem.c 8 May 2006 21:39:57 -0000 @@ -111,8 +111,8 @@ { EAP_HANDLER *handler; - handler = rad_malloc(sizeof(EAP_HANDLER)); - memset(handler, 0, sizeof(EAP_HANDLER)); + handler = rad_malloc(sizeof(*handler)); + memset(handler, 0, sizeof(*handler)); return handler; } @@ -121,6 +121,12 @@ if (!handler) return; + /* + * This is a hack to work around request_data_add not + * taking an "inst" pointer + */ + if (handler->inst) return eaplist_delete(handler->inst, handler); + if (handler->identity) { free(handler->identity); handler->identity = NULL; @@ -195,6 +201,7 @@ */ handler->timestamp = handler->request->timestamp; handler->status = 1; + handler->inst = inst; memcpy(handler->state, state->strvalue, sizeof(handler->state)); handler->src_ipaddr = handler->request->packet->src_ipaddr; @@ -216,6 +223,8 @@ */ status = rbtree_insert(inst->session_tree, handler); + handler->in_list = 1; + if (status) { EAP_HANDLER *prev; @@ -243,6 +252,42 @@ return 1; } + +static void eaplist_delete_internal(rlm_eap_t *inst, EAP_HANDLER *handler) +{ + + rbtree_deletebydata(inst->session_tree, handler); + + inst->session_head = handler->next; + if (handler->next) handler->next->prev = NULL; + handler->inst = NULL; + handler->in_list = 0; + eap_handler_free(handler); +} + + +/* + * Delete a handler from the list. + */ +void eaplist_delete(rlm_eap_t *inst, EAP_HANDLER *handler) +{ + if (!handler->in_list || !handler->inst) { + handler->inst = NULL; + handler->in_list = 0; + eap_handler_free(handler); + return; + } + + /* + * Playing with a data structure shared among threads + * means that we need a lock, to avoid conflict. + */ + pthread_mutex_lock(&(inst->session_mutex)); + eaplist_delete_internal(inst, handler); + pthread_mutex_unlock(&(inst->session_mutex)); +} + + /* * Find a a previous EAP-Request sent by us, which matches * the current EAP-Response. @@ -292,13 +337,7 @@ handler = inst->session_head; if (handler && ((request->timestamp - handler->timestamp) > inst->timer_limit)) { - node = rbtree_find(inst->session_tree, handler); - rad_assert(node != NULL); - rbtree_delete(inst->session_tree, node); - - inst->session_head = handler->next; - if (handler->next) handler->next->prev = NULL; - eap_handler_free(handler); + eaplist_delete(inst, handler); } } @@ -339,6 +378,7 @@ inst->session_tail = NULL; } handler->prev = handler->next = NULL; + handler->in_list = 0; } } Index: src/modules/rlm_eap/rlm_eap.h =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/rlm_eap.h,v retrieving revision 1.13.4.1 diff -u -r1.13.4.1 rlm_eap.h --- src/modules/rlm_eap/rlm_eap.h 6 Feb 2006 16:23:53 -0000 1.13.4.1 +++ src/modules/rlm_eap/rlm_eap.h 8 May 2006 21:39:57 -0000 @@ -101,6 +101,7 @@ EAP_HANDLER *eaplist_find(rlm_eap_t *inst, REQUEST *request, eap_packet_t *eap_packet); void eaplist_free(rlm_eap_t *inst); +void eaplist_delete(rlm_eap_t *inst, EAP_HANDLER *handler); /* State */ void generate_key(void);
"Alan DeKok" <aland@nitros9.org> wrote:
Sorry, sent the wrong patch.
With a lock bug. Dang. I'll get it right one of these days. OK, This should work. Alan DeKok. Index: src/modules/rlm_eap/eap.h =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/eap.h,v retrieving revision 1.27.4.1 diff -u -r1.27.4.1 eap.h --- src/modules/rlm_eap/eap.h 6 Feb 2006 16:23:50 -0000 1.27.4.1 +++ src/modules/rlm_eap/eap.h 8 May 2006 21:53:07 -0000 @@ -121,6 +121,9 @@ int status; int stage; + + int in_list; + void *inst; } EAP_HANDLER; /* Index: src/modules/rlm_eap/mem.c =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/mem.c,v retrieving revision 1.14.4.1 diff -u -r1.14.4.1 mem.c --- src/modules/rlm_eap/mem.c 6 Feb 2006 16:23:51 -0000 1.14.4.1 +++ src/modules/rlm_eap/mem.c 8 May 2006 21:53:08 -0000 @@ -25,6 +25,10 @@ static const char rcsid[] = "$Id: mem.c,v 1.14.4.1 2006/02/06 16:23:51 nbk Exp $"; +static void eaplist_delete_locked(rlm_eap_t *inst, EAP_HANDLER *handler); +static void eaplist_delete(rlm_eap_t *inst, EAP_HANDLER *handler); + + /* * Allocate a new EAP_PACKET */ @@ -111,8 +115,8 @@ { EAP_HANDLER *handler; - handler = rad_malloc(sizeof(EAP_HANDLER)); - memset(handler, 0, sizeof(EAP_HANDLER)); + handler = rad_malloc(sizeof(*handler)); + memset(handler, 0, sizeof(*handler)); return handler; } @@ -121,6 +125,12 @@ if (!handler) return; + /* + * This is a hack to work around request_data_add not + * taking an "inst" pointer + */ + if (handler->inst) return eaplist_delete(handler->inst, handler); + if (handler->identity) { free(handler->identity); handler->identity = NULL; @@ -195,6 +205,7 @@ */ handler->timestamp = handler->request->timestamp; handler->status = 1; + handler->inst = inst; memcpy(handler->state, state->strvalue, sizeof(handler->state)); handler->src_ipaddr = handler->request->packet->src_ipaddr; @@ -216,6 +227,8 @@ */ status = rbtree_insert(inst->session_tree, handler); + handler->in_list = 1; + if (status) { EAP_HANDLER *prev; @@ -244,6 +257,56 @@ } /* + * Delete, assuming that the mutex is locked. + */ +static void eaplist_delete_locked(rlm_eap_t *inst, EAP_HANDLER *handler) +{ + rbtree_deletebydata(inst->session_tree, handler); + + /* + * And unsplice it from the linked list. + */ + if (handler->prev) { + handler->prev->next = handler->next; + } else { + inst->session_head = NULL; + } + if (handler->next) { + handler->next->prev = handler->prev; + } else { + inst->session_tail = NULL; + } + handler->prev = handler->next = NULL; + + handler->inst = NULL; + handler->in_list = 0; + eap_handler_free(handler); +} + + +/* + * Delete a handler from the list. + */ +static void eaplist_delete(rlm_eap_t *inst, EAP_HANDLER *handler) +{ + if (!handler->in_list || !handler->inst) { + handler->inst = NULL; + handler->in_list = 0; + eap_handler_free(handler); + return; + } + + /* + * Playing with a data structure shared among threads + * means that we need a lock, to avoid conflict. + */ + pthread_mutex_lock(&(inst->session_mutex)); + eaplist_delete_locked(inst, handler); + pthread_mutex_unlock(&(inst->session_mutex)); +} + + +/* * Find a a previous EAP-Request sent by us, which matches * the current EAP-Response. * @@ -292,13 +355,7 @@ handler = inst->session_head; if (handler && ((request->timestamp - handler->timestamp) > inst->timer_limit)) { - node = rbtree_find(inst->session_tree, handler); - rad_assert(node != NULL); - rbtree_delete(inst->session_tree, node); - - inst->session_head = handler->next; - if (handler->next) handler->next->prev = NULL; - eap_handler_free(handler); + eaplist_delete_locked(inst, handler); } } @@ -320,25 +377,7 @@ if (verify_state(state, handler->timestamp) != 0) { handler = NULL; } else { - /* - * It's OK, delete it from the tree. - */ - rbtree_delete(inst->session_tree, node); - - /* - * And unsplice it from the linked list. - */ - if (handler->prev) { - handler->prev->next = handler->next; - } else { - inst->session_head = NULL; - } - if (handler->next) { - handler->next->prev = handler->prev; - } else { - inst->session_tail = NULL; - } - handler->prev = handler->next = NULL; + eaplist_delete_locked(inst, handler); } }
Alan DeKok wrote:
With a lock bug. Dang. I'll get it right one of these days.
Now the server segfaults at the first Access-Request with EAP. Attached the output from valgrind. (not the whole thing this time :) Bjarni Hardarson ==18068== ==18068== Invalid read of size 4 ==18068== at 0x457E7C1: eap_ds_free (mem.c:101) ==18068== by 0x457EAED: eaplist_find (mem.c:407) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427) ==18068== Address 0x46C69F0 is 48 bytes inside a block of size 80 free'd ==18068== at 0x401C178: free (vg_replace_malloc.c:235) ==18068== by 0x457E8BE: eap_handler_free (mem.c:152) ==18068== by 0x457E975: eaplist_delete_locked (mem.c:283) ==18068== by 0x457EABE: eaplist_find (mem.c:380) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== ==18068== Invalid read of size 4 ==18068== at 0x457EAEE: eaplist_find (mem.c:408) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427) ==18068== Address 0x46C69F4 is 52 bytes inside a block of size 80 free'd ==18068== at 0x401C178: free (vg_replace_malloc.c:235) ==18068== by 0x457E8BE: eap_handler_free (mem.c:152) ==18068== by 0x457E975: eaplist_delete_locked (mem.c:283) ==18068== by 0x457EABE: eaplist_find (mem.c:380) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== ==18068== Invalid write of size 4 ==18068== at 0x457EAF1: eaplist_find (mem.c:409) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427) ==18068== Address 0x46C69F4 is 52 bytes inside a block of size 80 free'd ==18068== at 0x401C178: free (vg_replace_malloc.c:235) ==18068== by 0x457E8BE: eap_handler_free (mem.c:152) ==18068== by 0x457E975: eaplist_delete_locked (mem.c:283) ==18068== by 0x457EABE: eaplist_find (mem.c:380) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== ==18068== Invalid write of size 4 ==18068== at 0x457EAF8: eaplist_find (mem.c:408) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427) ==18068== Address 0x46C69F0 is 48 bytes inside a block of size 80 free'd ==18068== at 0x401C178: free (vg_replace_malloc.c:235) ==18068== by 0x457E8BE: eap_handler_free (mem.c:152) ==18068== by 0x457E975: eaplist_delete_locked (mem.c:283) ==18068== by 0x457EABE: eaplist_find (mem.c:380) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== ==18068== Invalid read of size 4 ==18068== at 0x457D443: eap_handler (eap.c:1049) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427) ==18068== Address 0x46C69EC is 44 bytes inside a block of size 80 free'd ==18068== at 0x401C178: free (vg_replace_malloc.c:235) ==18068== by 0x457E8BE: eap_handler_free (mem.c:152) ==18068== by 0x457E975: eaplist_delete_locked (mem.c:283) ==18068== by 0x457EABE: eaplist_find (mem.c:380) ==18068== by 0x457D3EB: eap_handler (eap.c:993) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== ==18068== Invalid read of size 1 ==18068== at 0x401D47D: strncmp (mac_replace_strmem.c:311) ==18068== by 0x457D44D: eap_handler (eap.c:1049) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427) ==18068== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==18068== ==18068== Process terminating with default action of signal 11 (SIGSEGV): dumping core ==18068== Access not within mapped region at address 0x0 ==18068== at 0x401D47D: strncmp (mac_replace_strmem.c:311) ==18068== by 0x457D44D: eap_handler (eap.c:1049) ==18068== by 0x457CC37: eap_authenticate (rlm_eap.c:230) ==18068== by 0x8054C2A: modcall (modcall.c:236) ==18068== by 0x8055256: call_one (modcall.c:269) ==18068== by 0x8054E16: modcall (modcall.c:324) ==18068== by 0x805315D: indexed_modcall (modules.c:469) ==18068== by 0x804CD1D: rad_check_password (auth.c:367) ==18068== by 0x804D1C4: rad_authenticate (auth.c:662) ==18068== by 0x8056284: rad_respond (radiusd.c:1642) ==18068== by 0x8057E51: main (radiusd.c:1427)
Bjarni Hardarson <freeradius@hardarson.se> wrote:
Now the server segfaults at the first Access-Request with EAP.
That isn't vey helpful.
Attached the output from valgrind. (not the whole thing this time :)
It looks like it's now too aggressive about destroying handles. Ok, how do you reproduce it? I can't reproduce it here, so it's a little difficult for me to know what's going wrong, or if a patch will help. Alan DeKok.
Bjarni Hardarson <freeradius@hardarson.se> wrote: ... After a bit of looking, it appears there are other problems. Here's a patch that may help. No idea if it solves this problem, but the current code looks buggy. Alan DeKok. Index: src/modules/rlm_eap/mem.c =================================================================== RCS file: /source/radiusd/src/modules/rlm_eap/mem.c,v retrieving revision 1.14.4.1 diff -u -r1.14.4.1 mem.c --- src/modules/rlm_eap/mem.c 6 Feb 2006 16:23:51 -0000 1.14.4.1 +++ src/modules/rlm_eap/mem.c 9 May 2006 18:24:19 -0000 @@ -223,8 +223,11 @@ if (prev) { prev->next = handler; handler->prev = prev; + handler->next = NULL; + inst->session_tail = handler; } else { inst->session_head = inst->session_tail = handler; + handler->next = handler->prev = NULL; } } @@ -331,12 +334,12 @@ if (handler->prev) { handler->prev->next = handler->next; } else { - inst->session_head = NULL; + inst->session_head = handler->next; } if (handler->next) { handler->next->prev = handler->prev; } else { - inst->session_tail = NULL; + inst->session_tail = handler->prev; } handler->prev = handler->next = NULL; }
Alan DeKok wrote:
After a bit of looking, it appears there are other problems. Here's a patch that may help. No idea if it solves this problem, but the current code looks buggy.
That fixed it :) Been stressing the server for 20 minutes now. Doing EAP-PEAP, EAP-TTLS and EAP-MD5 at 30 authentications/sec Thank you Alan. Bjarni Hardarson
participants (4)
-
Alan DeKok -
Bjarni Hardarson -
Nikolas Thoman -
Stuart Auchterlonie