TMPL_TYPE_LITERAL has non-zero bytes in its data union
Hi, in our dev environment now with development version from upstream tree 3.0.x I see this error: CONSISTENCY CHECK FAILED src/main/map.c[1529]: TMPL_TYPE_LITERAL has non-zero bytes in its data union SOFT ASSERT FAILED src/main/tmpl.c[2155]: 0 CAUGHT SIGNAL: Aborted Backtrace of last 16 frames: /usr/lib64/libfreeradius-radius.so(fr_fault+0x1a7)[0x7f4f26716637] /usr/lib64/libfreeradius-radius.so(fr_assert_cond+0x50)[0x7f4f26716840] /usr/lib64/libfreeradius-server.so(tmpl_verify+0x1d0)[0x7f4f26773590] /usr/lib64/libfreeradius-server.so(map_prints+0x58)[0x7f4f26770e28] /usr/lib64/libfreeradius- server.so(fr_cond_sprint+0x1a9)[0x7f4f2676f069] /usr/sbin/radiusd(modcall_debug+0x27c)[0x55a776bfdffc] /usr/sbin/radiusd(modcall_debug+0xed)[0x55a776bfde6d] /usr/sbin/radiusd(modcall_debug+0xed)[0x55a776bfde6d] /usr/sbin/radiusd(modcall_debug+0xed)[0x55a776bfde6d] /usr/sbin/radiusd(+0x26c0a)[0x55a776bf5c0a] /usr/sbin/radiusd(+0x26e83)[0x55a776bf5e83] /usr/sbin/radiusd(virtual_servers_load+0xfd)[0x55a776bf77ad] /usr/sbin/radiusd(modules_init+0x90e)[0x55a776bf84fe] /usr/sbin/radiusd(main+0x5a8)[0x55a776be3a68] /lib64/libc.so.6(__libc_start_main+0xe7)[0x7f4f25af2c07] /usr/sbin/radiusd(_start+0x2a)[0x55a776be401a] No panic action set Aborted (core dumped) This is an error is related to some issue on linking of the binary between mismatched versioning for same libraries or it's related to a wrong configuration? In Sabayon we have upgraded to glibc-2.29 so I'm not sure if could be related to some issues of compatibility or whatever. Thanks in advance for any clarification g.
On Jun 21, 2019, at 4:03 AM, Geaaru <geaaru@gmail.com> wrote:
in our dev environment now with development version from upstream tree 3.0.x I see this error:
CONSISTENCY CHECK FAILED src/main/map.c[1529]: TMPL_TYPE_LITERAL has non-zero bytes in its data union SOFT ASSERT FAILED src/main/tmpl.c[2155]: 0 CAUGHT SIGNAL: Aborted
Weird.
Backtrace of last 16 frames: /usr/lib64/libfreeradius-radius.so(fr_fault+0x1a7)[0x7f4f26716637] /usr/lib64/libfreeradius-radius.so(fr_assert_cond+0x50)[0x7f4f26716840] /usr/lib64/libfreeradius-server.so(tmpl_verify+0x1d0)[0x7f4f26773590] /usr/lib64/libfreeradius-server.so(map_prints+0x58)[0x7f4f26770e28] /usr/lib64/libfreeradius- server.so(fr_cond_sprint+0x1a9)[0x7f4f2676f069] /usr/sbin/radiusd(modcall_debug+0x27c)[0x55a776bfdffc] /usr/sbin/radiusd(modcall_debug+0xed)[0x55a776bfde6d]
Hmm... it would be good to get debugging symbols, i.e. line numbers. That would let us quickly track down & fix the issue.
This is an error is related to some issue on linking of the binary between mismatched versioning for same libraries or it's related to a wrong configuration?
It shouldn't be either of those. Likely something in FreeRADIUS isn't initializing things properly at start-up.
In Sabayon we have upgraded to glibc-2.29 so I'm not sure if could be related to some issues of compatibility or whatever.
It won't be that. Alan DeKok.
I try to share debug symbols Tuesday or Wednesday. This Monday is a holiday day here. Thanks g. On Fri, 2019-06-21 at 09:56 -0400, Alan DeKok wrote:
On Jun 21, 2019, at 4:03 AM, Geaaru <geaaru@gmail.com> wrote:
in our dev environment now with development version from upstream tree3.0.x I see this error: CONSISTENCY CHECK FAILED src/main/map.c[1529]: TMPL_TYPE_LITERAL hasnon-zero bytes in its data unionSOFT ASSERT FAILED src/main/tmpl.c[2155]: 0CAUGHT SIGNAL: Aborted
Weird.
Backtrace of last 16 frames:/usr/lib64/libfreeradius- radius.so(fr_fault+0x1a7)[0x7f4f26716637]/usr/lib64/libfreeradius- radius.so(fr_assert_cond+0x50)[0x7f4f26716840]/usr/lib64/libfreerad ius- server.so(tmpl_verify+0x1d0)[0x7f4f26773590]/usr/lib64/libfreeradiu s- server.so(map_prints+0x58)[0x7f4f26770e28]/usr/lib64/libfreeradius- server.so(fr_cond_sprint+0x1a9)[0x7f4f2676f069]/usr/sbin/radiusd(mo dcall_debug+0x27c)[0x55a776bfdffc]/usr/sbin/radiusd(modcall_debug+0 xed)[0x55a776bfde6d]
Hmm... it would be good to get debugging symbols, i.e. line numbers. That would let us quickly track down & fix the issue.
This is an error is related to some issue on linking of the binarybetween mismatched versioning for same libraries or it's related to awrong configuration?
It shouldn't be either of those. Likely something in FreeRADIUS isn't initializing things properly at start-up.
In Sabayon we have upgraded to glibc-2.29 so I'm not sure if could berelated to some issues of compatibility or whatever.
It won't be that. Alan DeKok.
It seems not related to linking. I try to investigate a bit if could be related to my external module of OCI LIb or other. Program received signal SIGABRT, Aborted.0x00007ffff6eb86cb in raise () from /lib64/libpthread.so.0(gdb) bt#0 0x00007ffff6eb86cb in raise () from /lib64/libpthread.so.0#1 0x00007ffff7b43243 in fr_fault (sig=6) at /home/geaaru/Projects/freeradius- server/src/lib/debug.c:763#2 0x00007ffff7b43c2a in fr_assert_cond (file=0x7ffff7dc1cf0 "/home/geaaru/Projects/freeradius- server/src/main/tmpl.c", line=2155, expr=0x7ffff7dc1df0 "0", cond=false) at /home/geaaru/Projects/freeradius- server/src/lib/debug.c:1101#3 0x00007ffff7dac82c in tmpl_verify (file=0x7ffff7dc01a0 "/home/geaaru/Projects/freeradius- server/src/main/map.c", line=1529, vpt=0x555555a849f0) at /home/geaaru/Projects/freeradius- server/src/main/tmpl.c:2155#4 0x00007ffff7da7598 in map_prints (buffer=0x7fffffffbd10 "&CHAP-Challenge", bufsize=1023, map=0x555555a84d60) at /home/geaaru/Projects/freeradius- server/src/main/map.c:1529#5 0x00007ffff7d9fd82 in fr_cond_sprint (buffer=0x7fffffffbd10 "&CHAP-Challenge", bufsize=1024, in=0x555555a845a0) at /home/geaaru/Projects/freeradius- server/src/main/parser.c:93#6 0x000055555558c5a2 in modcall_debug (mc=0x555555d622d0, depth=5) at /home/geaaru/Projects/freeradius- server/src/main/modcall.c:3965#7 0x000055555558c616 in modcall_debug (mc=0x555555d621d0, depth=4) at /home/geaaru/Projects/freeradius- server/src/main/modcall.c:3968#8 0x000055555558c616 in modcall_debug (mc=0x555555d620d0, depth=3) at /home/geaaru/Projects/freeradius- server/src/main/modcall.c:3968#9 0x000055555558c52f in modcall_debug (mc=0x555555d61fd0, depth=2) at /home/geaaru/Projects/freeradius- server/src/main/modcall.c:3958#10 0x0000555555581f87 in load_component_section (cs=0x555555a76930, components=0x555555d5edb0, comp=MOD_AUTHORIZE) at /home/geaaru/Projects/freeradius- server/src/main/modules.c:1241#11 0x00005555555822f3 in load_byserver (cs=0x555555a765f0) at /home/geaaru/Projects/freeradius- server/src/main/modules.c:1334#12 0x00005555555828c2 in virtual_servers_load (config=0x5555557e9690) at /home/geaaru/Projects/freeradius-server/src/main/modules.c:1521#13 0x0000555555583e7f in modules_init (config=0x5555557e9690) at /home/geaaru/Projects/freeradius-server/src/main/modules.c:2151#14 0x000055555558d442 in main (argc=8, argv=0x7fffffffd1c8) at /home/geaaru/Projects/freeradius-server/src/main/radiusd.c:520(gdb) up#1 0x00007ffff7b43243 in fr_fault (sig=6) at /home/geaaru/Projects/freeradius-server/src/lib/debug.c:763763 raise(sig);(gdb) printThe history is empty.(gdb) On Fri, 2019-06-21 at 09:56 -0400, Alan DeKok wrote:
On Jun 21, 2019, at 4:03 AM, Geaaru <geaaru@gmail.com> wrote:
in our dev environment now with development version from upstream tree3.0.x I see this error: CONSISTENCY CHECK FAILED src/main/map.c[1529]: TMPL_TYPE_LITERAL hasnon-zero bytes in its data unionSOFT ASSERT FAILED src/main/tmpl.c[2155]: 0CAUGHT SIGNAL: Aborted
Weird.
Backtrace of last 16 frames:/usr/lib64/libfreeradius- radius.so(fr_fault+0x1a7)[0x7f4f26716637]/usr/lib64/libfreeradius- radius.so(fr_assert_cond+0x50)[0x7f4f26716840]/usr/lib64/libfreerad ius- server.so(tmpl_verify+0x1d0)[0x7f4f26773590]/usr/lib64/libfreeradiu s- server.so(map_prints+0x58)[0x7f4f26770e28]/usr/lib64/libfreeradius- server.so(fr_cond_sprint+0x1a9)[0x7f4f2676f069]/usr/sbin/radiusd(mo dcall_debug+0x27c)[0x55a776bfdffc]/usr/sbin/radiusd(modcall_debug+0 xed)[0x55a776bfde6d]
Hmm... it would be good to get debugging symbols, i.e. line numbers. That would let us quickly track down & fix the issue.
This is an error is related to some issue on linking of the binarybetween mismatched versioning for same libraries or it's related to awrong configuration?
It shouldn't be either of those. Likely something in FreeRADIUS isn't initializing things properly at start-up.
In Sabayon we have upgraded to glibc-2.29 so I'm not sure if could berelated to some issues of compatibility or whatever.
It won't be that. Alan DeKok.
On Jun 26, 2019, at 8:55 AM, Geaaru <geaaru@gmail.com> wrote:
It seems not related to linking. I try to investigate a bit if could be related to my external module of OCI LIb or other.
It shouldn't be.
#10 0x0000555555581f87 in load_component_section (cs=0x555555a76930, components=0x555555d5edb0, comp=MOD_AUTHORIZE) at /home/geaaru/Projects/freeradius-server/src/main/modules.c:1241
That's just debugging the things it loaded from the "authorize" section. My $0.02 is to comment out bits of the "authorize" section until it no longer crashes. That configuration, then, is the one causing the issue. Once we know what config causes the issue, the fix should be simple. Alan DeKok.
Yeah, I did that. Thanks. But the weird thing is that this happens in different configurations: This seems that first use case (if I drop all operations and I leave this): authorize { if (&Control:Auth-Type == "CHAP") { } } the exception is visible from my side. It seems related to Control:Auth-Type but for example this works: authorize { if (!&Control:Auth-Type || &Control:Auth-Type != "Accept") { } } and also this it works: authorize { if (!&Control:Auth-Type || &Control:Auth-Type != "Accept") { if (&Control:Auth-Type) { } elsif (&Control:Auth-Type) { } } } and also this it works: authorize { if (!&Control:Auth-Type || &Control:Auth-Type != "Accept") { if (&Control:Auth-Type == "Reject") { } elsif (&Control:Auth-Type == "Accept") { } } } It's weird. It seems related to the comparison of &Control:Auth-Type to string "CHAP" or "PAP". On Wed, 2019-06-26 at 09:29 -0400, Alan DeKok wrote:
On Jun 26, 2019, at 8:55 AM, Geaaru <geaaru@gmail.com> wrote:
It seems not related to linking. I try to investigate a bit if could be related to my external module of OCI LIb or other.
It shouldn't be.
#10 0x0000555555581f87 in load_component_section (cs=0x555555a76930, components=0x555555d5edb0, comp=MOD_AUTHORIZE) at /home/geaaru/Projects/freeradius-server/src/main/modules.c:1241
That's just debugging the things it loaded from the "authorize" section. My $0.02 is to comment out bits of the "authorize" section until it no longer crashes. That configuration, then, is the one causing the issue. Once we know what config causes the issue, the fix should be simple. Alan DeKok.
On Jun 26, 2019, at 10:00 AM, Geaaru <geaaru@gmail.com> wrote:
Yeah, I did that. Thanks. But the weird thing is that this happens in different configurations:
This seems that first use case (if I drop all operations and I leave this):
authorize {
if (&Control:Auth-Type == "CHAP") {
}
}
the exception is visible from my side.
OK. I see that too. Now that I can reproduce it here, it should be easier to fix. Alan DeKok.
Hi Alan, I saw your patch on upstream and I dunno if your commits cover completely this use case. But if could help, now I have a different stack for the exception with your last changes: CONSISTENCY CHECK FAILED /home/geaaru/Projects/freeradius- server/src/main/tmpl.c[1103]: TMPL_TYPE_LITERAL has non-zero bytes in its data unionSOFT ASSERT FAILED /home/geaaru/Projects/freeradius- server/src/main/tmpl.c[2158]: 0CAUGHT SIGNAL: AbortedBacktrace of last 16 frames:/tmp/radius/usr/lib/libfreeradius- radius.so(fr_fault+0x12c)[0x7ffff7b42f76]/tmp/radius/usr/lib/libfreerad ius- radius.so(fr_assert_cond+0x5b)[0x7ffff7b43c2a]/tmp/radius/usr/lib/libfr eeradius- server.so(tmpl_verify+0x1ef)[0x7ffff7dac8bc]/tmp/radius/usr/lib/libfree radius- server.so(tmpl_cast_in_place+0x116)[0x7ffff7daa369]/tmp/radius/usr/lib/ libfreeradius- server.so(+0x1a29a)[0x7ffff7da229a]/tmp/radius/usr/lib/libfreeradius- server.so(+0x18c79)[0x7ffff7da0c79]/tmp/radius/usr/lib/libfreeradius- server.so(fr_condition_tokenize+0x5e)[0x7ffff7da38e7]/tmp/radius/usr/li b/libfreeradius- server.so(+0xe9c7)[0x7ffff7d969c7]/tmp/radius/usr/lib/libfreeradius- server.so(+0xf906)[0x7ffff7d97906]/tmp/radius/usr/lib/libfreeradius- server.so(+0xe4fa)[0x7ffff7d964fa]/tmp/radius/usr/lib/libfreeradius- server.so(+0xf906)[0x7ffff7d97906]/tmp/radius/usr/lib/libfreeradius- server.so(cf_file_read+0x119)[0x7ffff7d97c6d]/tmp/radius/usr/sbin/radiu sd(main_config_init+0x4fa)[0x55555557e8eb]/tmp/radius/usr/sbin/radiusd( main+0x735)[0x55555558d18e]/lib64/libc.so.6(__libc_start_main+0xe7)[0x7 ffff6a63c07]/tmp/radius/usr/sbin/radiusd(_start+0x2a)[0x555555566a7a]No panic action set Program received signal SIGABRT, Aborted.0x00007ffff6eb86cb in raise () from /lib64/libpthread.so.0(gdb) bt#0 0x00007ffff6eb86cb in raise () from /lib64/libpthread.so.0#1 0x00007ffff7b43243 in fr_fault (sig=6) at /home/geaaru/Projects/freeradius- server/src/lib/debug.c:763#2 0x00007ffff7b43c2a in fr_assert_cond (file=0x7ffff7dc1d90 "/home/geaaru/Projects/freeradius- server/src/main/tmpl.c", line=2158, expr=0x7ffff7dc1e90 "0", cond=false) at /home/geaaru/Projects/freeradius- server/src/lib/debug.c:1101#3 0x00007ffff7dac8bc in tmpl_verify (file=0x7ffff7dc1d90 "/home/geaaru/Projects/freeradius- server/src/main/tmpl.c", line=1103, vpt=0x555555a772f0) at /home/geaaru/Projects/freeradius- server/src/main/tmpl.c:2158#4 0x00007ffff7daa369 in tmpl_cast_in_place (vpt=0x555555a772f0, type=PW_TYPE_INTEGER, enumv=0x55555598bb58) at /home/geaaru/Projects/freeradius- server/src/main/tmpl.c:1103#5 0x00007ffff7da229a in condition_tokenize (ctx=0x555555a69750, ci=0x555555a76b10, start=0x7ffffffe88de "&Control:Auth-Type == \"CHAP\") {", brace=true, pcond=0x555555a69760, error=0x7ffffffe87a0, flags=1) at /home/geaaru/Projects/freeradius- server/src/main/parser.c:1140#6 0x00007ffff7da0c79 in condition_tokenize (ctx=0x555555a76b10, ci=0x555555a76b10, start=0x7ffffffe88dd "(&Control:Auth-Type == \"CHAP\") {", brace=false, pcond=0x7ffffffe87b0, error=0x7ffffffe87a0, flags=1) at /home/geaaru/Projects/freeradius- server/src/main/parser.c:530#7 0x00007ffff7da38e7 in fr_condition_tokenize (ctx=0x555555a76b10, ci=0x555555a76b10, start=0x7ffffffe88dd "(&Control:Auth-Type == \"CHAP\") {", head=0x7ffffffe87b0, error=0x7ffffffe87a0, flags=1) at /home/geaaru/Projects/freeradius-server/src/main/parser.c:1754 On Wed, 2019-06-26 at 10:16 -0400, Alan DeKok wrote:
On Jun 26, 2019, at 10:00 AM, Geaaru <geaaru@gmail.com> wrote:
Yeah, I did that. Thanks. But the weird thing is that this happens in different configurations: This seems that first use case (if I drop all operations and I leave this): authorize { if (&Control:Auth-Type == "CHAP") { } } the exception is visible from my side.
OK. I see that too. Now that I can reproduce it here, it should be easier to fix. Alan DeKok.
On Jun 26, 2019, at 1:01 PM, Geaaru <geaaru@gmail.com> wrote:
I saw your patch on upstream and I dunno if your commits cover completely this use case.
It doesn't fix it.
But if could help, now I have a different stack for the exception with your last changes:
It now complains when the problem occurs, and not many functions later. I've tracked it down to exactly the situation that needs to be fixed. We should have a fix in a few days. Alan DeKok.
Hi, I had investigate a bit on this and it's very weird. I revert the code of the tag 3.0.17 that is now in production and where code with PAP/CHAP condition works perfectly, but now with the same 3.0.17 in upgraded rootfs I receive the same error. I fear that could be related to some library used by Freeradius. Between the differences I have: - glibc-2.28 (works) => glibc-2.29 (doesn't work)- json-c- 0.12 => json-c-0.13.1- talloc-2.1.14 => talloc- 2.2.0 Is there a specific library that is used by unlang parser? Could be related to glibc or talloc? On Wed, 2019-06-26 at 14:03 -0400, Alan DeKok wrote:
On Jun 26, 2019, at 1:01 PM, Geaaru <geaaru@gmail.com> wrote:
I saw your patch on upstream and I dunno if your commits cover completely this use case.
It doesn't fix it.
But if could help, now I have a different stack for the exception with your last changes:
It now complains when the problem occurs, and not many functions later. I've tracked it down to exactly the situation that needs to be fixed. We should have a fix in a few days. Alan DeKok.
Please, Share the radius -XXx output. And If you are getting some crash, please share the stack trace as you got previously. — Jorge Pereira
On 5 Jul 2019, at 14:34, Geaaru <geaaru@gmail.com> wrote:
On Jun 26, 2019, at 1:01 PM, Geaaru <geaaru@gmail.com> wrote:
I saw your patch on upstream and I dunno if your commits cover completely this use case.
It doesn't fix it.
But if could help, now I have a different stack for the exception with your last changes:
It now complains when the problem occurs, and not many functions later. I've tracked it down to exactly the situation that needs to be fixed. We should have a fix in a few days. Alan DeKok.
Hi, I had investigate a bit on this and it's very weird. I revert the code of the tag 3.0.17 that is now in production and where code with PAP/CHAP condition works perfectly, but now with the same 3.0.17 in upgraded rootfs I receive the same error. I fear that could be related to some library used by Freeradius. Between the differences I have: - glibc-2.28 (works) => glibc-2.29 (doesn't work)- json-c- 0.12 => json-c-0.13.1- talloc-2.1.14 => talloc- 2.2.0 Is there a specific library that is used by unlang parser? Could be related to glibc or talloc? On Wed, 2019-06-26 at 14:03 -0400, Alan DeKok wrote: - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
participants (3)
-
Alan DeKok -
Geaaru -
Jorge Pereira