Alan DeKok <aland@deployingradius.com> writes:
Bjørn Mork wrote:
I am now seeing this very same problem, and strongly suspect it to be related to dead proxy home servers. I was able to provoke the "Exiting normally" on a server with *no* traffic at all, by doing a couple of requests for a realm with dead home servers and then waiting:
Wed Nov 25 18:03:56 2009 : Error: PROXY: Marking home server 88.a.b.158 port 1812 as zombie (it looks like it is dead). Wed Nov 25 18:04:35 2009 : Error: PROXY: Marking home server 84.c.d.222 port 1812 as zombie (it looks like it is dead). Wed Nov 25 19:38:13 2009 : Info: Exiting normally.
No requests at all were sent to this server between the two last log lines.
Hmm... the "exiting normally" means that it received a signal to exit (internal or external). Otherwise, it just keeps running.
Try using gdb, and:
(gdb) break event_loop_exit
I don't have that symbol. Did you mean fr_event_loop_exit? Anyway, I ran the server (this time a lab-/test-server with some traffic but nothing near any real load) using Breakpoint 1, radius_signal_self (flag=2) at event.c:3733 3733 event.c: No such file or directory. in event.c (gdb) cont Continuing. Breakpoint 2, fr_event_loop_exit (el=0x8e3c4d0, code=2) at event.c:309 309 event.c: No such file or directory. in event.c This is still based on the stable tree with commit 2df19cf0024fd23d2906c13c0b01067076540871 as the last one. And I got: Program received signal SIGTERM, Terminated. 0x000000390d8306f7 in kill () from /lib64/libc.so.6 (gdb) thread apply all bt full Thread 22 (Thread 0x4d189940 (LWP 23437)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035f830) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4080d8 <rad_accounting> self = (THREAD_HANDLE *) 0x1035f830 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 20 (Thread 0x4bd87940 (LWP 23435)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035f530) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4094c7 <rad_authenticate> self = (THREAD_HANDLE *) 0x1035f530 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 17 (Thread 0x49f84940 (LWP 23432)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035f0b0) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4094c7 <rad_authenticate> self = (THREAD_HANDLE *) 0x1035f0b0 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 14 (Thread 0x48181940 (LWP 23429)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035ec30) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4080d8 <rad_accounting> self = (THREAD_HANDLE *) 0x1035ec30 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 12 (Thread 0x46d7f940 (LWP 23427)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035e930) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4080d8 <rad_accounting> self = (THREAD_HANDLE *) 0x1035e930 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 10 (Thread 0x4597d940 (LWP 23425)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035e630) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4094c7 <rad_authenticate> self = (THREAD_HANDLE *) 0x1035e630 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 3 (Thread 0x41d77940 (LWP 23418)): #0 0x000000390e40c9b1 in sem_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x000000000042632e in request_handler_thread (arg=0x1035dcf0) at threads.c:453 fun = (RAD_REQUEST_FUNP) 0x4094c7 <rad_authenticate> self = (THREAD_HANDLE *) 0x1035dcf0 #2 0x000000390e4064a7 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #3 0x000000390d8d3c2d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 1 (Thread 0x2ad516d82e70 (LWP 23315)): #0 0x000000390d8306f7 in kill () from /lib64/libc.so.6 No symbol table info available. #1 0x0000000000423b6a in main (argc=4, argv=0x7fffecacbef8) at radiusd.c:419 rcode = 0 argval = -1 spawn_flag = 1 dont_fork = 1 flag = 0 act = {__sigaction_handler = {sa_handler = 0x423d41 <sig_fatal>, sa_sigaction = 0x423d41 <sig_fatal>}, sa_mask = {__val = { 0 <repeats 16 times>}}, sa_flags = 0, sa_restorer = 0} (gdb) (gdb) cont Continuing. [Thread 0x4bd87940 (LWP 23435) exited] [Thread 0x49f84940 (LWP 23432) exited] [Thread 0x48181940 (LWP 23429) exited] [Thread 0x46d7f940 (LWP 23427) exited] [Thread 0x4597d940 (LWP 23425) exited] [Thread 0x41d77940 (LWP 23418) exited] [Thread 0x4d189940 (LWP 23437) exited] Program exited with code 0377. Don't know if that helped anything at all? Bjørn