Hi, finally I found the time to build the recent freeradius with address sanitizer support. I built with gcc, that works fine, no clang needed. RTLD_DEEPBIND in src/main/modules.c seems to break asan-support? https://github.com/google/sanitizers/issues/611 The changelog says, trouble with oracle was the cause for RTLD_DEEPBIND, would it be possible to couple RTLD_DEEPBIND via configure to oracle support? The debian package has no oracle support, so I used this patch: --- freeradius-3.0.20+dfsg.orig/src/main/modules.c +++ freeradius-3.0.20+dfsg/src/main/modules.c @@ -168,10 +168,11 @@ fr_dlhandle fr_dlopenext(char const *nam } else #endif flags |= RTLD_LOCAL; +/* #ifdef RTLD_DEEPBIND flags |= RTLD_DEEPBIND; #endif - +*/ The first run crashed after a few minutes (see AddressSanitizer output below), a second run with higher max_requests still runs without a crash for a few hours now. I guess there shouldn't be a crash with double-free, even if max_requests is to low? Here's the asan-Log: ================================================================= ==115708==ERROR: AddressSanitizer: attempting double-free on 0x60b0001bbab0 in thread T0: #0 0x7f7ee2477fb0 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0) #1 0x7f7ee1e825d2 in _tc_free_internal ../talloc.c:1201 #2 0x7f7ee1e8243f in _tc_free_children_internal ../talloc.c:1646 #3 0x7f7ee1e8243f in _tc_free_internal ../talloc.c:1163 #4 0x7f7ee1e8243f in _tc_free_children_internal ../talloc.c:1646 #5 0x7f7ee1e8243f in _tc_free_internal ../talloc.c:1163 #6 0x7f7ee1e8243f in _tc_free_children_internal ../talloc.c:1646 #7 0x7f7ee1e8243f in _tc_free_internal ../talloc.c:1163 #8 0x7f7ee1e7d347 in _tc_free_children_internal ../talloc.c:1646 #9 0x7f7ee1e7d347 in _tc_free_internal ../talloc.c:1163 #10 0x7f7ee1e7d347 in _talloc_free_internal ../talloc.c:1227 #11 0x7f7ee1e7d347 in _talloc_free ../talloc.c:1769 #12 0x5594dc01a3cd in request_free src/main/process.c:602 #13 0x5594dc01a3cd in request_free src/main/process.c:587 #14 0x5594dc0206fc in request_done src/main/process.c:897 #15 0x5594dc024ac0 in request_receive src/main/process.c:1765 #16 0x5594dbfe7c3b in auth_socket_recv src/main/listen.c:1597 #17 0x5594dc01a775 in event_socket_handler src/main/process.c:4869 #18 0x7f7ee22b1988 in fr_event_loop src/lib/event.c:649 #19 0x5594dc031734 in radius_event_process src/main/process.c:5954 #20 0x5594dbfc7d33 in main src/main/radiusd.c:626 #21 0x7f7ee197209a in __libc_start_main ../csu/libc-start.c:308 #22 0x5594dbfc8979 in _start (/usr/sbin/freeradius+0x57979) 0x60b0001bbab0 is located 0 bytes inside of 111-byte region [0x60b0001bbab0,0x60b0001bbb1f) freed by thread T0 here: ==115708==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_stackdepotbase.h:140 "((id & (((u32)-1) >> kReservedBits))) == ((id))" (0x6a15fa73, 0xea15fa73) #0 0x7f7ee2482fa5 (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xf3fa5) #1 0x7f7ee249df39 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10ef39) #2 0x7f7ee24982af (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x1092af) #3 0x7f7ee23bc1ec (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x2d1ec) #4 0x7f7ee23bd60d (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x2e60d) #5 0x7f7ee23bdc2f (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x2ec2f) #6 0x7f7ee247fee3 (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xf0ee3) #7 0x7f7ee23bacc4 (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x2bcc4) #8 0x7f7ee2477f8a in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8f8a) #9 0x7f7ee1e825d2 in _tc_free_internal ../talloc.c:1201 #10 0x7f7ee1e8243f in _tc_free_children_internal ../talloc.c:1646 #11 0x7f7ee1e8243f in _tc_free_internal ../talloc.c:1163 #12 0x7f7ee1e8243f in _tc_free_children_internal ../talloc.c:1646 #13 0x7f7ee1e8243f in _tc_free_internal ../talloc.c:1163 #14 0x7f7ee1e8243f in _tc_free_children_internal ../talloc.c:1646 #15 0x7f7ee1e8243f in _tc_free_internal ../talloc.c:1163 #16 0x7f7ee1e7d347 in _tc_free_children_internal ../talloc.c:1646 #17 0x7f7ee1e7d347 in _tc_free_internal ../talloc.c:1163 #18 0x7f7ee1e7d347 in _talloc_free_internal ../talloc.c:1227 #19 0x7f7ee1e7d347 in _talloc_free ../talloc.c:1769 #20 0x5594dc01a3cd in request_free src/main/process.c:602 #21 0x5594dc01a3cd in request_free src/main/process.c:587 #22 0x5594dc0206fc in request_done src/main/process.c:897 #23 0x5594dc024ac0 in request_receive src/main/process.c:1765 #24 0x5594dbfe7c3b in auth_socket_recv src/main/listen.c:1597 #25 0x5594dc01a775 in event_socket_handler src/main/process.c:4869 #26 0x7f7ee22b1988 in fr_event_loop src/lib/event.c:649 #27 0x5594dc031734 in radius_event_process src/main/process.c:5954 #28 0x5594dbfc7d33 in main src/main/radiusd.c:626 #29 0x7f7ee197209a in __libc_start_main ../csu/libc-start.c:308 #30 0x5594dbfc8979 in _start (/usr/sbin/freeradius+0x57979) Am 06.03.19 um 23:17 schrieb Alan DeKok:
Another alternative is to use LLVM. If you install the LLVM RPM, you can then build FreeRADIUS with some more options.
1) install LLVM 2) grab the 3.0.18 source (or the source from GitHub) 3) do: CC=clang ./configure --prefix=/opt/freeradius ... args ...
I think the name of the compiler on Suse is "clang". If not, you'll have to find that out
You're best to put this build into a unique directory, so it's easy to get rid of it later.
4) Edit "Make.inc", and look for the line "CFLAGS = ..." add this:
-fsanitize=address -fno-omit-frame-pointer -fno-optimize-sibling-calls -fsanitize-address-use-after-scope
5) run "make; make install"
You can then run that version with some extra environment variables:
export ASAN_SYMBOLIZER_PATH="/usr/local/opt/llvm/bin/llvm-symbolizer"
You'll have to find out where that executable is located...
export ASAN_OPTIONS="malloc_context_size=50 detect_leaks=1 symbolize=1"
Then run FreeRADIUS: /usr/freeradius/sbin/radiusd -f -d /etc/raddb
With some luck, the address sanitizer may find the issue. Note that the server will run much slower than normal, but that's likely fine.
These issues are very, very, difficult to track down without tools like the address sanitizer. We're using that in the v4 / master branch, and it's a huge help.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- Daniel Feuchtinger Leibniz-Rechenzentrum Boltzmannstraße 1 D-85748 Garching b. München Tel. 089 35831 8820