FreeRADIUS 3.0: Segfault in rlm_detail
Hi, I'm trying to upgrade my config to FreeRADIUS 3.0, but I get reproducable segfaults in the `rlm_detail' module. I send a simple packet (User-Name, User-Password) via `radclient', and the server immediately crashes: $ bin/radiusd -xx -f -l /dev/stderr -t ... (0) auth_log : expand: "/server/devel/server/log/client/%{Client-IP-Address}/auth-%Y%m%d" -> '/server/devel/server/log/client/XXX.XXX.XX.XX/auth-20140304' (0) auth_log : /server/devel/server/log/client/%{Client-IP-Address}/auth-%Y%m%d expands to /server/devel/server/log/client/XXX.XXX.XX.XX/auth-20140304 (0) auth_log : Acquired filelock, tried 1 time(s) (0) auth_log : expand: "%S" -> '2014-03-04 14:41:14' Bad talloc magic value - unknown value [Backtrace follows] The auth_log is defined in the detail module config: $ cat etc/mods-enabled/detail ... detail auth_log { filename = ${radacctdir}/%{Client-IP-Address}/auth-%Y%m%d permissions = 0600 dir_permissions = 0700 locking = yes header = "%S" log_packet_header = yes suppress { User-Password } } ... Interesting part of GDB backtrace: #0 0x00007ffff5d891b5 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007ffff5d8bfc0 in *__GI_abort () at abort.c:92 #2 0x00007ffff71380bc in ?? () from /usr/lib/libtalloc.so.2 #3 0x00007ffff7138221 in talloc_get_name () from /usr/lib/libtalloc.so.2 #4 0x00007ffff713b56b in _talloc_get_type_abort () from /usr/lib/libtalloc.so.2 #5 0x00007ffff79913b8 in vp_prints (out=0x7fffffffa751 "", outlen=1023, vp=0x7fffffffac20) at src/lib/print.c:868 #6 0x00007ffff799154a in vp_print (fp=0xd31870, vp=0x7fffffffac20) at src/lib/print.c:908 #7 0x00007ffff3523e7e in detail_write (out=0xd31870, inst=0xd16a50, request=0xd31060, packet=0xd30ee0, compat=false) at src/modules/rlm_detail/rlm_detail.c:249 #8 0x00007ffff35248c3 in detail_do (instance=0xd16a50, request=0xd31060, packet=0xd30ee0, compat=false) at src/modules/rlm_detail/rlm_detail.c:484 #9 0x00007ffff3524a5a in mod_authorize (instance=0xd16a50, request=0xd31060) at src/modules/rlm_detail/rlm_detail.c:533 #10 0x0000000000423915 in call_modsingle (component=RLM_COMPONENT_AUTZ, sp=0xd18bb0, request=0xd31060) at src/main/modcall.c:311 ... The file /server/devel/server/log/client/XXX.XXX.XX.XX/auth-20140304 is actually being created, but contains only this: 2014-03-04 14:41:14 Packet-Type = Access-Request So presumably FreeRADIUS crashes while trying to print out the Packet-Src-IP-Address...? Thanks in advance, Julius P.S.: I'm using the v3.0.x branch from a few days ago, namely 80eee20b9. Compiling against the tip of v3.0.x, eb3bfda, is not possible any more: $ make CC src/lib/cbuff.c In file included from <command-line>:0: /server/devel/src/freeradius/src/freeradius-devel/build.h:84:6: error: #error Failed determining endianness of system make: *** [build/objs/src/lib/cbuff.lo] Error 1
Interesting part of GDB backtrace:
#0 0x00007ffff5d891b5 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007ffff5d8bfc0 in *__GI_abort () at abort.c:92 #2 0x00007ffff71380bc in ?? () from /usr/lib/libtalloc.so.2 #3 0x00007ffff7138221 in talloc_get_name () from /usr/lib/libtalloc.so.2 #4 0x00007ffff713b56b in _talloc_get_type_abort () from /usr/lib/libtalloc.so.2 #5 0x00007ffff79913b8 in vp_prints (out=0x7fffffffa751 "", outlen=1023, vp=0x7fffffffac20) at src/lib/print.c:868 #6 0x00007ffff799154a in vp_print (fp=0xd31870, vp=0x7fffffffac20) at src/lib/print.c:908 #7 0x00007ffff3523e7e in detail_write (out=0xd31870, inst=0xd16a50, request=0xd31060, packet=0xd30ee0, compat=false) at src/modules/rlm_detail/rlm_detail.c:249 #8 0x00007ffff35248c3 in detail_do (instance=0xd16a50, request=0xd31060, packet=0xd30ee0, compat=false) at src/modules/rlm_detail/rlm_detail.c:484 #9 0x00007ffff3524a5a in mod_authorize (instance=0xd16a50, request=0xd31060) at src/modules/rlm_detail/rlm_detail.c:533 #10 0x0000000000423915 in call_modsingle (component=RLM_COMPONENT_AUTZ, sp=0xd18bb0, request=0xd31060) at src/main/modcall.c:311 ...
The file /server/devel/server/log/client/XXX.XXX.XX.XX/auth-20140304 is actually being created, but contains only this:
2014-03-04 14:41:14 Packet-Type = Access-Request
So presumably FreeRADIUS crashes while trying to print out the Packet-Src-IP-Address...?
It shouldn't be trying to print Packet-Src-IP-Address as it isn't a real attribute. Could you provide the contents of the request, I can't get it to crash on my system, and the backtrace provides no clue about which attribute it's failing at printing.
make: *** [build/objs/src/lib/cbuff.lo] Error 1
OK, what compiler/OS are you using. It means that the preprocessor macros in ./build.h couldn't figure out the endianess of the system, which is odd as the Macros should cover at least GCC and clang. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Hi, Arran! Additional info between the lines: * Arran Cudbard-Bell <a.cudbardb@freeradius.org> [2014-03-04 15:52]:
#5 0x00007ffff79913b8 in vp_prints (out=0x7fffffffa751 "", outlen=1023, vp=0x7fffffffac20) at src/lib/print.c:868 token = 0x0 start = 0x7fffffffa751 "" len = 4222451713 freespace = 1023 #6 0x00007ffff799154a in vp_print (fp=0xd31870, vp=0x7fffffffac20) at src/lib/print.c:908 buf = "\t\000Ë", '\000' <repeats 45 times>, [more stuff...] p = 0x7fffffffa751 "" len = 13058144 #7 0x00007ffff3523e7e in detail_write (out=0xd31870, inst=0xd16a50, request=0xd31060, packet=0xd30ee0, compat=false) at src/modules/rlm_detail/rlm_detail.c:249 src_vp = {da = 0x7e2260, next = 0x0, op = T_OP_EQ, tag = 0 '\000', value = {xlat = 0x0}, type = VT_NONE, length = 0, data = {strvalue = 0x130a8582 <Address 0x130a8582 out of bounds>, octets = 0x130a8582 <Address 0x130a8582 out of bounds>, ipaddr = {s_addr = 319456642}, ipv6addr = {__in6_u = {__u6_addr8 = "\202\205\n\023", '\000' <repeats 11 times>, __u6_addr16 = {34178, 4874, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {319456642, 0, 0, 0}}}, date = 319456642, integer = 319456642, sinteger = 319456642, integer64 = 319456642, filter = {319456642, 0, 0, 0}, ifid = "\202\205\n\023\000\000\000", ipv6prefix = "\202\205\n\023", '\000' <repeats 13 times>, ipv4prefix = "\202\205\n\023\000", ether = "\202\205\n\023\000", tlv = 0x130a8582 <Address 0x130a8582 out of bounds>, ptr = 0x130a8582}} dst_vp = {da = 0x7e2290, next = 0x0, op = T_OP_EQ, tag = 0 '\000', value = {xlat = 0x0}, type = VT_NONE, length = 0, data = {strvalue = 0x45028582 <Address 0x45028582 out of bounds>, octets = 0x45028582 <Address 0x45028582 out of bounds>, ipaddr = {s_addr = 1157793154}, ipv6addr = {__in6_u = {__u6_addr8 = "\202\205\002E", '\000' <repeats 11 times>, __u6_addr16 = {34178, 17666, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1157793154, 0, 0, 0}}}, date = 1157793154, integer = 1157793154, sinteger = 1157793154, integer64 = 1157793154, filter = {1157793154, 0, 0, 0}, ifid = "\202\205\002E\000\000\000", ipv6prefix = "\202\205\002E", '\000' <repeats 13 times>, ipv4prefix = "\202\205\002E\000", ether = "\202\205\002E\000", tlv = 0x45028582 <Address 0x45028582 out of bounds>, ptr = 0x45028582}} vp = 0xcbc980 timestamp = "2014-03-04 15:59:03\000" [more garbage] #8 0x00007ffff35248c3 in detail_do (instance=0xd16a50, request=0xd31060, packet=0xd30ee0, compat=false) at src/modules/rlm_detail/rlm_detail.c:484 outfd = 11 buffer = "/server/devel/server/log/client/130.133.10.19/auth-20140304\000\a\000" [more garbage...] p = 0x7fffffffae9d "/auth-20140304" st = {st_dev = 65024, st_ino = 1610613986, st_nlink = 1, st_mode = 33152, st_uid = 999, st_gid = 100, __pad0 = 0, st_rdev = 0, st_size = 150, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1393944516, tv_nsec = 33683970}, st_mtim = {tv_sec = 1393945050, tv_nsec = 117495477}, st_ctim = {tv_sec = 1393945050, tv_nsec = 117495477}, __unused = { 0, 0, 0}} locked = 1 lock_count = 0 tv = {tv_sec = 4156364544, tv_usec = 0} fsize = 150 outfp = 0xcbd750 gid = 0 grp = 0x3 endptr = 0x0 inst = 0xc74060
The file /server/devel/server/log/client/XXX.XXX.XX.XX/auth-20140304 is actually being created, but contains only this:
2014-03-04 14:41:14 Packet-Type = Access-Request
So presumably FreeRADIUS crashes while trying to print out the Packet-Src-IP-Address...?
It shouldn't be trying to print Packet-Src-IP-Address as it isn't a real attribute.
Ah, okay? My old 2.X instance is doing exactly that: It prints e.g. 2014-03-04 00:00:05 Packet-Type = Access-Request Packet-Src-IP-Address = ... Packet-Dst-IP-Address = ... Packet-Src-Port = 51585 Packet-Dst-Port = 1812 <read attributes follow> That's why I was thinking it something's not correct before printing the Packet-Src-IP-Address.
Could you provide the contents of the request,
As I said, I send a fairly simple packet. From the FreeRADIUS debug output, just before the detail module is called: Ready to process requests. rad_recv: Access-Request packet from host XXX.XXX.XX.XX port 42488, id=114, length=43 User-Name = 'feh' User-Password = 'foobarbaz'
make: *** [build/objs/src/lib/cbuff.lo] Error 1
OK, what compiler/OS are you using.
GCC 4.4.5 from Debian oldstable 6.0 / Squeeze.
It means that the preprocessor macros in ./build.h couldn't figure out the endianess of the system, which is odd as the Macros should cover at least GCC and clang.
Hmm. I don't know too much about this... but maybe this helps? $ gcc -E -dU src/include/build.h # 1 "src/include/build.h" # 1 "<built-in>" # 1 "<command-line>" # 1 "src/include/build.h" #undef __cplusplus #define __GNUC_MINOR__ 4 #define __GNUC__ 4 #undef __APPLE__ #undef __clang__ src/include/build.h:84:6: error: #error Failed determining endianness of system #undef __BIG_ENDIAN__ #undef __BYTE_ORDER__ #undef __LITTLE_ENDIAN__ #undef BIG_ENDIAN #undef LITTLE_ENDIAN #undef __SUNPRO_C Julius
Hi, Alan! * Alan DeKok <aland@deployingradius.com> [2014-03-04 16:39]:
OK. We have an idea as to what's going on. We're working on a fix.
I saw you just pushed a commit a537bd6035, "Wrapper to make vp_print() work with VPs on the stack". FreeRADIUS won't crash now, but I get this in the log files: 2014-03-05 14:27:47 Packet-Type = Access-Request Packet-Src-IP-Address <INVALID-TOKEN> 208.151.175.1 Packet-Dst-IP-Address <INVALID-TOKEN> 208.151.175.1 Packet-Src-Port <INVALID-TOKEN> 28284880 Packet-Dst-Port <INVALID-TOKEN> 28284880 User-Name = 'feh' Client-IP-Address = ... Julius
Hi, Arran! * Arran Cudbard-Bell <a.cudbardb@freeradius.org> [2014-03-04 15:52]:
make: *** [build/objs/src/lib/cbuff.lo] Error 1
OK, what compiler/OS are you using. It means that the preprocessor macros in ./build.h couldn't figure out the endianess of the system, which is odd as the Macros should cover at least GCC and clang.
I see you and Alan both use Macs. It seems GCC on OS X will actually define __LITTLE_ENDIAN__, but on Linux it doesn't: $ gcc -E -dM - < /dev/null | grep -i endian When I include <endian.h> just before the code block in question in build.h, then FreeRADIUS compiles correctly. Julius
Julius Plenz wrote:
Hi,
I'm trying to upgrade my config to FreeRADIUS 3.0, but I get reproducable segfaults in the `rlm_detail' module. I send a simple packet (User-Name, User-Password) via `radclient', and the server immediately crashes:
I've pushed a fix to the v3.0.x branch. Alan DeKok.
Hi, Alan! * Alan DeKok <aland@deployingradius.com> [2014-03-05 15:54]:
I'm trying to upgrade my config to FreeRADIUS 3.0, but I get reproducable segfaults in the `rlm_detail' module. I send a simple packet (User-Name, User-Password) via `radclient', and the server immediately crashes:
I've pushed a fix to the v3.0.x branch.
Thank you! Cherry-picking a537bd6 and b33b715 indeed fixes the problem. However the detection of the endian-ness is still not resolved, so the v3.0.x still fails to build on Linux with GCC. Julius
Hi, Alan! * Alan DeKok <aland@deployingradius.com> [2014-03-05 16:52]:
Thank you! Cherry-picking a537bd6 and b33b715 indeed fixes the problem. However the detection of the endian-ness is still not resolved, so the v3.0.x still fails to build on Linux with GCC.
I've pushed a fix.
Seeing that Arran reverted all the commits, I'm not sure what is supposed to happen now...? I tried your commits, and ./configure said: checking endian.h usability... yes checking endian.h presence... yes checking for endian.h... yes However, <endian.h> was not included. Now, with 25c344d89f, I get the following ./configure output: checking whether byte ordering is bigendian... no Yet compilation is still not working: /server/devel/src/freeradius/src/freeradius-devel/build.h:84:6: error: #error Failed determining endianness of system Julius
Julius Plenz wrote:
Seeing that Arran reverted all the commits, I'm not sure what is supposed to happen now...?
Arran should fix it.
I tried your commits, and ./configure said:
checking endian.h usability... yes checking endian.h presence... yes checking for endian.h... yes
However, <endian.h> was not included.
My commit included it in build.h. Arran reverted that because <endian.h> is apparently radically different across platforms. So... it's unusable.
Now, with 25c344d89f, I get the following ./configure output:
checking whether byte ordering is bigendian... no
Yet compilation is still not working:
/server/devel/src/freeradius/src/freeradius-devel/build.h:84:6: error: #error Failed determining endianness of system
I know who I'd blame... Alan DeKok.
Hi, Arran! * Arran Cudbard-Bell <a.cudbardb@freeradius.org> [2014-03-05 19:06]:
Fixed...
The issue is not really resolved yet... I'm using 0565937, "That's BIG_ENDIAN and LITTLE_ENDIAN", and I get: $ ./configure ... checking whether byte ordering is bigendian... no $ grep -i endian src/include/autoconf.h /* #undef BIG_ENDIAN */ #define LITTLE_ENDIAN 1 /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most # if defined __BIG_ENDIAN__ # define WORDS_BIGENDIAN 1 # ifndef WORDS_BIGENDIAN /* # undef WORDS_BIGENDIAN */ Yet still: /server/devel/src/freeradius/src/freeradius-devel/build.h:84:6: error: #error Failed determining endianness of system I don't know too much about autoconf handling, but how is build.h supposed to get the defs from autoconf.h? If I #include the autoconf.h in build.h, magically everything compiles... ;-) Julius
On 6 Mar 2014, at 11:58, Julius Plenz <plenz@cis.fu-berlin.de> wrote:
Hi, Arran!
* Arran Cudbard-Bell <a.cudbardb@freeradius.org> [2014-03-05 19:06]:
Fixed...
The issue is not really resolved yet... I'm using 0565937, "That's BIG_ENDIAN and LITTLE_ENDIAN", and I get:
$ ./configure ... checking whether byte ordering is bigendian... no
$ grep -i endian src/include/autoconf.h /* #undef BIG_ENDIAN */ #define LITTLE_ENDIAN 1 /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most # if defined __BIG_ENDIAN__ # define WORDS_BIGENDIAN 1 # ifndef WORDS_BIGENDIAN /* # undef WORDS_BIGENDIAN */
Yet still:
/server/devel/src/freeradius/src/freeradius-devel/build.h:84:6: error: #error Failed determining endianness of system
I don't know too much about autoconf handling, but how is build.h supposed to get the defs from autoconf.h?
-include
If I #include the autoconf.h in build.h, magically everything compiles... ;-)
Ah, yes might be an ordering thing. Try now... Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
On 5 Mar 2014, at 15:01, Julius Plenz <plenz@cis.fu-berlin.de> wrote:
Hi, Alan!
* Alan DeKok <aland@deployingradius.com> [2014-03-05 15:54]:
I'm trying to upgrade my config to FreeRADIUS 3.0, but I get reproducable segfaults in the `rlm_detail' module. I send a simple packet (User-Name, User-Password) via `radclient', and the server immediately crashes:
I've pushed a fix to the v3.0.x branch.
Thank you! Cherry-picking a537bd6 and b33b715 indeed fixes the problem. However the detection of the endian-ness is still not resolved, so the v3.0.x still fails to build on Linux with GCC.
Ok, put in a fix, re-run configure. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
participants (3)
-
Alan DeKok -
Arran Cudbard-Bell -
Julius Plenz