Freeradius several segfaults at heavy load and startup ?
Hello, Wondered if anyone have any idea about below. If started with flag -X everything starts up ok but without -X then it crashes with these messages in the log.(atleast most of the time if one is persistent then it may well start up properly sometimes without the -X flag) As soon as it starts ok then there seems to be no problem whatsoever. It runs perfectly with the same config and perl hooks at a lab machine where there are no traffic. Searching through the web doesn't give much info? Any ideas would be appreciated. Thank you. Alex root@itop0-db0:/scripts# LD_PRELOAD=/usr/lib/libperl.so.5.10 /usr/sbin/freeradius root@itop0-db0:/scripts# ps -ef | grep freeradius root 30086 29914 0 08:10 pts/1 00:00:00 grep freeradius root@itop0-db0:/scripts#dmesg . .. ... [2233283.445884] freeradius[617]: segfault at 212 ip 00000000f72238d4 sp 00000000ffb85050 error 4 in rlm_perl-2.1.10.so[f7221000+5000] [2233430.045210] freeradius[2673]: segfault at 212 ip 00000000f72328d4 sp 00000000ffff4e50 error 4 in rlm_perl-2.1.10.so[f7230000+5000] [2233671.048040] freeradius[2730]: segfault at 212 ip 00000000f72928d4 sp 00000000fffe4370 error 4 in rlm_perl-2.1.10.so[f7290000+5000] [2233840.528299] freeradius[2776]: segfault at 212 ip 00000000f72248d4 sp 00000000ffb06020 error 4 in rlm_perl-2.1.10.so[f7222000+5000] [5223187.548213] freeradius[2380]: segfault at 1002 ip 00000000f71e88d4 sp 00000000ffe0dfc0 error 4 in rlm_perl-2.1.10.so[f71e6000+5000] [5887868.065533] freeradius[31931]: segfault at 1002 ip 00000000f71bb8d4 sp 00000000ffb46d50 error 4 in rlm_perl-2.1.10.so[f71b9000+5000] [5946015.698283] freeradius[1746]: segfault at 88 ip 00000000f6ed29d1 sp 00000000e2df5810 error 4 in libmysqlclient_r.so.16.0.0[f6e53000+1af000] [5951643.541290] freeradius[11237]: segfault at 15 ip 00000000f76aac1a sp 00000000f56287a0 error 4 in libperl.so.5.10.1[f7610000+14a000] [5951657.770507] freeradius[11386]: segfault at c ip 00000000f76d3a17 sp 00000000f66a98d0 error 4 in libperl.so.5.10.1[f768f000+14a000] [6026380.012188] freeradius[11693]: segfault at 4 ip 00000000f6f1252e sp 00000000f0ff77d0 error 6 in libmysqlclient_r.so.16.0.0[f6e94000+1af000] [6026442.934483] freeradius[5466]: segfault at 8 ip 00000000f767a611 sp 00000000f5e4c930 error 4 in libperl.so.5.10.1[f7633000+14a000] [6026462.851031] freeradius[5547]: segfault at 8 ip 00000000f75ec611 sp 00000000f5dbe930 error 4 in libperl.so.5.10.1[f75a5000+14a000] [6544711.773094] freeradius[29963]: segfault at 8 ip 00000000f7616611 sp 00000000f6e0e930 error 4 in libperl.so.5.10.1[f75cf000+14a000] [6544722.667507] freeradius[29984]: segfault at 8 ip 00000000f75e1611 sp 00000000f4db1930 error 4 in libperl.so.5.10.1[f759a000+14a000] [6544822.799431] freeradius[30080]: segfault at 8d160fc ip 00000000f76a81b1 sp 00000000f32fc540 error 4 in libperl.so.5.10.1[f75e6000+14a000] [6544885.122348] freeradius[30118]: segfault at 8 ip 00000000f75f7611 sp 00000000f5dc9930 error 4 in libperl.so.5.10.1[f75b0000+14a000] [6544895.128894] freeradius[30137]: segfault at 8 ip 00000000f76bc611 sp 00000000f6eb4930 error 4 in libperl.so.5.10.1[f7675000+14a000] [6544905.352401] freeradius[30152]: segfault at 8 ip 00000000f766d611 sp 00000000f6640930 error 4 in libperl.so.5.10.1[f7626000+14a000] [6544946.136776] freeradius[30196]: segfault at 8 ip 00000000f76a5611 sp 00000000f4e75930 error 4 in libperl.so.5.10.1[f765e000+14a000] [6544966.088368] freeradius[30212]: segfault at c ip 00000000f764aa17 sp 00000000f66208d0 error 4 in libperl.so.5.10.1[f7606000+14a000] ********* DISCLAIMER ********* This message and any attachment are confidential and may be privileged or otherwise protected from disclosure and may include proprietary information. If you are not the intended recipient, please telephone or email the sender and delete this message and any attachment from your system. If you are not the intended recipient you must not copy this message or attachment or disclose the contents to any other person
On 11/28/2012 04:28 AM, Alexander Silveröhrt wrote:
Hello,
Wondered if anyone have any idea about below. If started with flag –X everything starts up ok but without –X then it crashes with these messages in the log.(atleast most of the time if one is persistent then it may well start up properly sometimes without the –X flag) As soon as it starts ok then there seems to be no problem whatsoever.
Which version?
It runs perfectly with the same config and perl hooks at a lab machine where there are no traffic. Searching through the web doesn’t give much info?
Any ideas would be appreciated.
Thank you. Alex
root@itop0-db0:/scripts# LD_PRELOAD=/usr/lib/libperl.so.5.10
Why are you fiddling with LD_PRELOAD? Presumably the perl module is the problem, but you shouldn't need to PRELOAD anything (and in fact, shouldn't, as the ABI might have changed).
On 2012/11/28 11:50 AM, Phil Mayers wrote:
root@itop0-db0:/scripts# LD_PRELOAD=/usr/lib/libperl.so.5.10
Why are you fiddling with LD_PRELOAD?
On my debian boxes FR cannot run without preload. There is something on the mailing list about it a while back. -- Johan Meiring Cape PC Services CC Tel: (021) 883-8271 Fax: (021) 886-7782 -------------------- Before acting on this email or opening any attachments you should read Cape PC Service's email disclaimer at: http://www.pcservices.co.za/disclaimer.html
On 28/11/12 10:52, Johan Meiring wrote:
On 2012/11/28 11:50 AM, Phil Mayers wrote:
root@itop0-db0:/scripts# LD_PRELOAD=/usr/lib/libperl.so.5.10
Why are you fiddling with LD_PRELOAD?
On my debian boxes FR cannot run without preload.
Yuck. It's probably some libtool horror in 2.x. It might work in "master" / 3.x
There is something on the mailing list about it a while back.
Well, that ought to be fixed. But I don't use rlm_perl, and the code is incomprehensible gibberish (hey, it's perl - it's mandatory!) so I'm not going to touch it ;o)
Same here doesn't even start without LD_PRELOAD. -----Ursprungligt meddelande----- Från: freeradius-users-bounces+alexander.silverohrt=itux.se@lists.freeradius.org [mailto:freeradius-users-bounces+alexander.silverohrt=itux.se@lists.freeradius.org] För Johan Meiring Skickat: den 28 november 2012 11:52 Till: freeradius-users@lists.freeradius.org Ämne: Re: Freeradius several segfaults at heavy load and startup ? On 2012/11/28 11:50 AM, Phil Mayers wrote:
root@itop0-db0:/scripts# LD_PRELOAD=/usr/lib/libperl.so.5.10
Why are you fiddling with LD_PRELOAD?
On my debian boxes FR cannot run without preload. There is something on the mailing list about it a while back. -- Johan Meiring Cape PC Services CC Tel: (021) 883-8271 Fax: (021) 886-7782 -------------------- Before acting on this email or opening any attachments you should read Cape PC Service's email disclaimer at: http://www.pcservices.co.za/disclaimer.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html ********* DISCLAIMER ********* This message and any attachment are confidential and may be privileged or otherwise protected from disclosure and may include proprietary information. If you are not the intended recipient, please telephone or email the sender and delete this message and any attachment from your system. If you are not the intended recipient you must not copy this message or attachment or disclose the contents to any other person
-----Ursprungligt meddelande----- On my debian boxes FR cannot run without preload.
On Wed, Nov 28, 2012 at 03:57:34PM +0000, Alexander Silveröhrt wrote:
Same here doesn't even start without LD_PRELOAD.
What versions of Debian and FreeRADIUS are these reports referring to? Are you using FR compiled yourself, or own-built packages (from git?), or the standard Debian packages from their repo? Matthew -- Matthew Newton, Ph.D. <mcn4@le.ac.uk> Systems Architect (UNIX and Networks), Network Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk>
Versions: Linux version 2.6.32-5-amd64 (Debian 2.6.32-45) (dannf@debian.org) (gcc version 4.3.5 (Debian 4.3.5-4) freeradius: FreeRADIUS Version 2.1.10, for host i486-pc-linux-gnu, built on Nov 14 2010 at 20:41:03 perl v5.10.1 (*) built for i486-linux-gnu-thread-multi mysql Server version: 5.5.27-1~dotdeb.0-log (Debian) And again if not started with LD_PRELOAD then DBI will fail root@itop1-db1:~# freeradius Can't load '/usr/local/lib/perl/5.10.1/auto/DBI/DBI.so' for module DBI: /usr/local/lib/perl/5.10.1/auto/DBI/DBI.so: undefined symbol: PL_memory_wrap at /usr/lib/perl/5.10/DynaLoader.pm line 192. at /usr/local/lib/perl/5.10.1/DBI.pm line 266 BEGIN failed--compilation aborted at /usr/local/lib/perl/5.10.1/DBI.pm line 266. Compilation failed in require at /etc/freeradius/customhook.pl line 10. Just followed the solution from the mailing list and it worked to get DBI working. http://lists.freeradius.org/pipermail/freeradius-users/2008-September/031333... best regards Alex -----Ursprungligt meddelande----- Från: freeradius-users-bounces+alexander.silverohrt=itux.se@lists.freeradius.org [mailto:freeradius-users-bounces+alexander.silverohrt=itux.se@lists.freeradius.org] För Phil Mayers Skickat: den 28 november 2012 10:50 Till: freeradius-users@lists.freeradius.org Ämne: Re: Freeradius several segfaults at heavy load and startup ? On 11/28/2012 04:28 AM, Alexander Silveröhrt wrote:
Hello,
Wondered if anyone have any idea about below. If started with flag -X everything starts up ok but without -X then it crashes with these messages in the log.(atleast most of the time if one is persistent then it may well start up properly sometimes without the -X flag) As soon as it starts ok then there seems to be no problem whatsoever.
Which version?
It runs perfectly with the same config and perl hooks at a lab machine where there are no traffic. Searching through the web doesn't give much info?
Any ideas would be appreciated.
Thank you. Alex
root@itop0-db0:/scripts# LD_PRELOAD=/usr/lib/libperl.so.5.10
Why are you fiddling with LD_PRELOAD? Presumably the perl module is the problem, but you shouldn't need to PRELOAD anything (and in fact, shouldn't, as the ABI might have changed). - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html ********* DISCLAIMER ********* This message and any attachment are confidential and may be privileged or otherwise protected from disclosure and may include proprietary information. If you are not the intended recipient, please telephone or email the sender and delete this message and any attachment from your system. If you are not the intended recipient you must not copy this message or attachment or disclose the contents to any other person
Alexander Silveröhrt wrote:
And again if not started with LD_PRELOAD then DBI will fail root@itop1-db1:~# freeradius Can't load '/usr/local/lib/perl/5.10.1/auto/DBI/DBI.so' for module DBI: /usr/local/lib/perl/5.10.1/auto/DBI/DBI.so: undefined symbol: PL_memory_wrap at /usr/lib/perl/5.10/DynaLoader.pm line 192. at /usr/local/lib/perl/5.10.1/DBI.pm line 266 BEGIN failed--compilation aborted at /usr/local/lib/perl/5.10.1/DBI.pm line 266. Compilation failed in require at /etc/freeradius/customhook.pl line 10.
Debian-built perl (installed in /usr/lib/perl5) tries to load DBI module from locally built perl (installed in /usr/local/lib/perl). It's a bad idea. Try to install debian-built DBI module (libdbi-perl) and use it. I think the same error (mashup of multiple perl builds) was encountered by Johan Meiring. Debian-built freeradius uses debian-built perl. So only debian-built perl modules can be loaded by rlm_perl. You can easily test your debian-built perl supports loading of DBI (and so debian-built freeradius): /usr/bin/perl -e \ 'use DBI; print join("\n", DBI->available_drivers), "\n";'
Alexander Silveröhrt wrote:
Wondered if anyone have any idea about below. If started with flag –X everything starts up ok but without –X then it crashes with these messages in the log.
Older versions of rlm_perl didn't lock enough of the data structures. So it could have threading issues. Upgrade. Alan DeKok.
participants (6)
-
Alan DeKok -
Alexander Silveröhrt -
Iliya Peregoudov -
Johan Meiring -
Matthew Newton -
Phil Mayers