rlm_perl.c questions

Boian Jordanov bjordanov at orbitel.bg
Fri Jan 7 10:54:40 CET 2011 

Anatoly,

I still think this is thread issue. How do you use/initialize connection to DB is it a global variable ? 

Can you send me core #1 ?

 


On Jan 7, 2011, at 1:06 AM, Anatoly Ivanov wrote:

> Boian,
> 
> Thanks for the explanation.
> 
> I am rather limited in what I can do on a production system, so it is not
> (administratively) easy to re-compile and re-deploy the code.
> 
> Right now I have configured the server to never kill a thread from
> threadpool - by specifying max_spare_servers=max_servers. The server has not
> crashed for two days now, but that may be a coincidence - low load during
> the holidays helps with the stability. I should be able to tell for sure
> next week.
> 
> I have a bunch of backtraces. Please let me know if you see anything
> suspicious in there.
> bt #1:
> 
> #0  0xb47d1ab6 in mysql_st_execute (sth=0xaf13a1a8, imp_sth=0xadf537a8) at
> dbdimp.c:3209
> #1  0xb47da215 in XS_DBD__mysql__st_execute (my_perl=0xb831a08,
> cv=0xb1e16858) at mysql.xsi:588
> #2  0xb70740ab in XS_DBI_dispatch () from
> /opt/server/lib/perl5/i686-linux-thread-multi/auto/DBI/DBI.so
> #3  0xb76a841e in Perl_pp_entersub () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #4  0xb76a6841 in Perl_runops_standard () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #5  0xb7641500 in Perl_call_sv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #6  0xb7642284 in Perl_call_pv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #7  0xb72026ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so
> #8  0x08063524 in modcall ()
> #9  0x0805ffe7 in indexed_modcall ()
> #10 0x080602fc in module_accounting ()
> #11 0x0804e9d1 in rad_accounting ()
> #12 0x0806df65 in radius_handle_request ()
> #13 0x080661d0 in request_handler_thread ()
> #14 0xb759c4c0 in start_thread () from /lib/i686/cmov/libpthread.so.0
> #15 0xb734384e in clone () from /lib/i686/cmov/libc.so.6
> 
> bt #2:
> 
> #0  0xb7690c80 in Perl_pad_undef () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #1  0xb764db67 in Perl_cv_undef () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #2  0xb76dcf5a in Perl_sv_clear () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #3  0xb76dd288 in Perl_sv_free2 () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #4  0xb76dd391 in Perl_sv_free () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #5  0xb76dcd57 in Perl_sv_clear () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #6  0xb76dd288 in Perl_sv_free2 () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #7  0xb70af022 in XS_Sys__Syslog_closelog_xs () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Sys/Syslog/Syslog.so
> #8  0xb76c941e in Perl_pp_entersub () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #9  0xb76c7841 in Perl_runops_standard () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #10 0xb7662500 in Perl_call_sv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #11 0xb7663284 in Perl_call_pv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #12 0xb72236ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so
> #13 0x08063524 in modcall ()
> #14 0x0805ffe7 in indexed_modcall ()
> #15 0x0806037c in module_authenticate ()
> #16 0x0804f908 in rad_authenticate ()
> #17 0x0806df65 in radius_handle_request ()
> #18 0x080661d0 in request_handler_thread ()
> #19 0xb75bd4c0 in start_thread () from /lib/i686/cmov/libpthread.so.0
> #20 0xb736484e in clone () from /lib/i686/cmov/libc.so.6
> 
> 
> bt#3:
> 
> #0  0xb77a2424 in __kernel_vsyscall ()
> #1  0xb72d6640 in raise () from /lib/i686/cmov/libc.so.6
> #2  0xb72d8018 in abort () from /lib/i686/cmov/libc.so.6
> #3  0xb731348d in __libc_message () from /lib/i686/cmov/libc.so.6
> #4  0xb7319764 in malloc_printerr () from /lib/i686/cmov/libc.so.6
> #5  0xb731b966 in free () from /lib/i686/cmov/libc.so.6
> #6  0xb76d81b1 in Perl_safesysfree () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #7  0xb77040a8 in Perl_sv_clear () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #8  0xb7704288 in Perl_sv_free2 () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #9  0xb70d6022 in XS_Sys__Syslog_closelog_xs () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Sys/Syslog/Syslog.so
> #10 0xb76f041e in Perl_pp_entersub () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #11 0xb76ee841 in Perl_runops_standard () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #12 0xb7689500 in Perl_call_sv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #13 0xb768a284 in Perl_call_pv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #14 0xb724a6ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so
> #15 0x08063524 in modcall ()
> #16 0x0805ffe7 in indexed_modcall ()
> #17 0x0806037c in module_authenticate ()
> #18 0x0804f908 in rad_authenticate ()
> #19 0x0806df65 in radius_handle_request ()
> #20 0x080661d0 in request_handler_thread ()
> #21 0xb75e44c0 in start_thread () from /lib/i686/cmov/libpthread.so.0
> #22 0xb738b84e in clone () from /lib/i686/cmov/libc.so.6
> 
> bt #4
> 
> #0  0xb7673132 in Perl_av_fetch () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #1  0xb708c788 in DD_dump () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Data/Dumper/Dumper.so
> #2  0xb708d77f in DD_dump () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Data/Dumper/Dumper.so
> #3  0xb7090219 in XS_Data__Dumper_Dumpxs () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Data/Dumper/Dumper.so
> #4  0xb767541e in Perl_pp_entersub () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #5  0xb7673841 in Perl_runops_standard () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #6  0xb760e500 in Perl_call_sv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #7  0xb760f284 in Perl_call_pv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #8  0xb71cf6ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so
> #9  0x08063524 in modcall ()
> #10 0x0805ffe7 in indexed_modcall ()
> #11 0x0806037c in module_authenticate ()
> #12 0x0804f908 in rad_authenticate ()
> #13 0x0806df65 in radius_handle_request ()
> #14 0x080661d0 in request_handler_thread ()
> #15 0xb75694c0 in start_thread () from /lib/i686/cmov/libpthread.so.0
> #16 0xb731084e in clone () from /lib/i686/cmov/libc.so.6
> 
> bt #5:
> 
> #0  0xb771ef77 in Perl_newSVsv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #1  0xb7712020 in Perl_pp_aassign () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #2  0xb770a841 in Perl_runops_standard () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #3  0xb76a5500 in Perl_call_sv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #4  0xb76a6284 in Perl_call_pv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #5  0xb72666ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so
> #6  0x08063524 in modcall ()
> #7  0x0805ffe7 in indexed_modcall ()
> #8  0x0806037c in module_authenticate ()
> #9  0x0804f908 in rad_authenticate ()
> #10 0x0806df65 in radius_handle_request ()
> #11 0x080661d0 in request_handler_thread ()
> #12 0xb76004c0 in start_thread () from /lib/i686/cmov/libpthread.so.0
> #13 0xb73a784e in clone () from /lib/i686/cmov/libc.so.6
> 
> bt #6:
> #0  0xb76f4424 in __kernel_vsyscall ()
> #1  0xb7228640 in raise () from /lib/i686/cmov/libc.so.6
> #2  0xb722a018 in abort () from /lib/i686/cmov/libc.so.6
> #3  0xb726548d in __libc_message () from /lib/i686/cmov/libc.so.6
> #4  0xb726b764 in malloc_printerr () from /lib/i686/cmov/libc.so.6
> #5  0xb726d966 in free () from /lib/i686/cmov/libc.so.6
> #6  0xb762a1b1 in Perl_safesysfree () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #7  0xb76560a8 in Perl_sv_clear () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #8  0xb7656288 in Perl_sv_free2 () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #9  0xb7028022 in XS_Sys__Syslog_closelog_xs () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Sys/Syslog/Syslog.so
> #10 0xb764241e in Perl_pp_entersub () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #11 0xb7640841 in Perl_runops_standard () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #12 0xb75db500 in Perl_call_sv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #13 0xb75dc284 in Perl_call_pv () from
> /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so
> #14 0xb719c6ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so
> #15 0x08063524 in modcall ()
> #16 0x0805ffe7 in indexed_modcall ()
> #17 0x0806037c in module_authenticate ()
> #18 0x0804f908 in rad_authenticate ()
> #19 0x0806df65 in radius_handle_request ()
> #20 0x080661d0 in request_handler_thread ()
> #21 0xb75364c0 in start_thread () from /lib/i686/cmov/libpthread.so.0
> #22 0xb72dd84e in clone () from /lib/i686/cmov/libc.so.6
> 
> Thanks,
> Anatoly.
> 
> 
> On Thu, Jan 6, 2011 at 1:26 AM, Boian Jordanov <bjordanov at orbitel.bg> wrote:
> 
>> 
>> On Jan 6, 2011, at 1:51 AM, Anatoly Ivanov wrote:
>> 
>>> Hi everybody,
>>> 
>>> I have a couple of questions regarding modules/rlm_perl/rlm_perl.c:
>>> perl_detach(...):
>>> 
>>> 1.  Is it correct to call PERL_SYS_TERM in detach()?
>>> 
>>> I am experiencing quite random and unpredictable segfaults with radius
>> 2.1.8
>>> and rlm_perl, and it really looks like memory allocation/multithreading
>>> error. I have very similar system running radius 2.1.7, and it works just
>>> fine. The most significant difference in rlm_perl between 2.1.7 and 2.1.8
>> is
>>> that PERL_SYS_TERM call.
>>> 
>>> According to the documentation
>>> (http://perldoc.perl.org/perlembed.html), PERL_SYS_TERM
>>> should be called only once. In our case, it is called once per detaching
>>> thread.
>> 
>> In fact it is called only once in the end of life of module. You can see
>> that when a thread is destroyed is called
>> 
>> /* Create Key */
>> static void rlm_perl_make_key(pthread_key_t *key)
>> {
>>       pthread_key_create(key, rlm_destroy_perl);
>> }
>> 
>> rlm_destroy_perl which destroy only particular interp
>> 
>> So the best will be to produce gdb trace file as i already wrote in my
>> previous mail.
>> 
>> 
>>> 
>>> 2. This code:
>>> 
>>> embed = rad_malloc(4*(sizeof(char *)));
>>> memset(embed, 0, sizeof(4*(sizeof(char *))));
>>> 
>>> looks like a misprint to me. Not that it would cause any real errors, but
>> it
>>> is inaccurate anyway.
>>> 
>>> Thank you,
>>> Anatoly.
>>> -
>>> List info/subscribe/unsubscribe? See
>> http://www.freeradius.org/list/devel.html
>> 
>> Best Regards,
>> Boian Jordanov
>> Head of Voice Department
>> tel. +359 2 4004 723
>> tel. +359 2 4004 002
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> -
>> List info/subscribe/unsubscribe? See
>> http://www.freeradius.org/list/devel.html
>> 

Best Regards,
Boian Jordanov
Head of Voice Department
tel. +359 2 4004 723
tel. +359 2 4004 002

More information about the Freeradius-Devel mailing list