Freeradius crashes with SIGABRT

[Daniel Feuchtinger]

Thanks a lot for the help. 

Am 04.12.19 um 15:14 schrieb Alan DeKok:
> On Dec 4, 2019, at 8:24 AM, Daniel Feuchtinger <daniel.feuchtinger at lrz.de> wrote:

> 
>   The issue seems to be that the server is slow, and is receiving conflicting packets.  i.e. the NAS sends packet A, some time later gives up, and then send packet B.
> 
>   But the server is still running packet A.  Likely because it's blocked in a database.
It's mainly a Proxy, only a few perl / ldap skripts to determine a group on a small part of the NASes. 

> 
>   So it tries to cancel packet A.  But for some reason that cancellation is delayed.  During that delay, the NAS retransmits packet B.  And the server tries to cancel request A again.  And then things go boom.
> 
>   It's not clear to me *why* this is happening.  The issue is buried deep inside of some fairly complex state machine work in the server.
> 
>   I've pushed a patch which should help.  It updates the main "check for duplicate / conflicting packet" code.  When the *first* conflicting packet B comes in, packet A is now removed entirely from the conflicting packet list.  That means when the next packet B comes in, any lookup will find B not A, and there's no double free.
> 
>   One reason this has been so hard to track down is that it doesn't occur in most situations.  It looks like it happens only when (1) the back-end DB is very slow, and (2) the NAS retransmits very aggressively.
> 
>   Please test the latest code from GitHub:
> 
> https://github.com/FreeRADIUS/freeradius-server.git
I tried the Patch in commit 36656cc39f52dbc79f8b373d313281144eef205b
with the 3.0.20 code, that resulted in a lot of
memory leaks (109 of them within a few minutes): 

=================================================================
==33264==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 3168 byte(s) in 36 object(s) allocated from:
    #0 0x7efc8022b330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
    #1 0x7efc800f9ad8 in rad_malloc src/main/util.c:528
    #2 0x55fc419d8aef in server_pool_alloc src/main/realms.c:1016
    #3 0x55fc419d9016 in old_server_add src/main/realms.c:1628
    #4 0x55fc419d9d84 in old_realm_config src/main/realms.c:1706
    #5 0x55fc419e0404 in realm_add src/main/realms.c:2005
    #6 0x55fc419e0404 in realms_init src/main/realms.c:2223
    #7 0x55fc41997bb1 in main_config_init src/main/mainconfig.c:1141
    #8 0x55fc4196e5cb in main src/main/radiusd.c:341
    #9 0x7efc7f72509a in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: 3168 byte(s) leaked in 36 allocation(s).

The server did not crash but it stopped logging and
probably working while producing the leaks. 

I'll try again with the current git code tomorrow. 



> 
>   If that works, it should be the definitive fix.
> 
>   Alan DeKok.
> 
> 
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
> 



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6026 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.freeradius.org/pipermail/freeradius-users/attachments/20191204/0ec5be15/attachment.bin>