freeRadius leaking memory

[Nicolas Reich]

Hi,

I'm running some tests on the freeRadius server I'm working on (which 
uses only EAP-TLS) to see what sort of load it can handle, and to tune 
the configuration accordingly. Doing this, I've noticed that the memory 
used by the radiusd process continually increases, until the system is 
out of memory, a clear sign of memory leaking. So I kept experimenting 
and I got quite sure there is some memory leaking. So I tried with 
Valgrind, and here's what I got after 1 minute (approximately 75 access 
requests):

LEAK SUMMARY:
    definitely lost: 46,978 bytes in 327 blocks
    indirectly lost: 893,915 bytes in 5,922 blocks
      possibly lost: 2,228,808 bytes in 4,792 blocks
    still reachable: 1,599,763 bytes in 32,997 blocks
         suppressed: 0 bytes in 0 blocks
Reachable blocks (those to which a pointer was found) are not shown.
To see them, rerun with: --leak-check=full --show-leak-kinds=all

Those are the biggest for "definitely lost":

214,180 (7,676 direct, 206,504 indirect) bytes in 76 blocks are 
definitely lost in loss record 1,696 of 1,700
    at 0x4C2DB9D: malloc (vg_replace_malloc.c:299)
    by 0x5B9697F: _talloc_array (in /usr/lib64/libtalloc.so.2.1.9)
    by 0x50814BF: talloc_bstrndup (in 
/usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x509121F: value_data_from_str (in 
/usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x5082207: fr_pair_value_from_str (in 
/usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x50826A7: fr_pair_make (in 
/usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x148961: cbtls_verify (in /usr/sbin/radiusd)
    by 0x561AE84: ??? (in /usr/lib64/libcrypto.so.1.0.2m)
    by 0x561CE5F: X509_verify_cert (in /usr/lib64/libcrypto.so.1.0.2m)
    by 0x5964137: ssl_verify_cert_chain (in /usr/lib64/libssl.so.1.0.2m)
    by 0x593AF78: ssl3_get_client_certificate (in 
/usr/lib64/libssl.so.1.0.2m)
    by 0x593C7D7: ssl3_accept (in /usr/lib64/libssl.so.1.0.2m)

509,668 (26,928 direct, 482,740 indirect) bytes in 153 blocks are 
definitely lost in loss record 1,698 of 1,700
    at 0x4C2DB9D: malloc (vg_replace_malloc.c:299)
    by 0x5B95DFA: _talloc_zero (in /usr/lib64/libtalloc.so.2.1.9)
    by 0x5081901: ??? (in /usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x5081A12: fr_pair_afrom_da (in 
/usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x5082598: fr_pair_make (in 
/usr/lib64/freeradius/libfreeradius-radius.so)
    by 0x148961: cbtls_verify (in /usr/sbin/radiusd)
    by 0x561AE84: ??? (in /usr/lib64/libcrypto.so.1.0.2m)
    by 0x561CE5F: X509_verify_cert (in /usr/lib64/libcrypto.so.1.0.2m)
    by 0x5964137: ssl_verify_cert_chain (in /usr/lib64/libssl.so.1.0.2m)
    by 0x593AF78: ssl3_get_client_certificate (in 
/usr/lib64/libssl.so.1.0.2m)
    by 0x593C7D7: ssl3_accept (in /usr/lib64/libssl.so.1.0.2m)
    by 0x5948801: ssl3_read_bytes (in /usr/lib64/libssl.so.1.0.2m)

And those for "possibly lost":

567,552 bytes in 32 blocks are possibly lost in loss record 1,700 of 1,700
    at 0x4C2DB9D: malloc (vg_replace_malloc.c:299)
    by 0x55277E7: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.2m)
    by 0x5949CD2: ssl3_setup_read_buffer (in /usr/lib64/libssl.so.1.0.2m)
    by 0x5949E28: ssl3_setup_buffers (in /usr/lib64/libssl.so.1.0.2m)
    by 0x594B484: ssl23_get_client_hello (in /usr/lib64/libssl.so.1.0.2m)
    by 0x594BC5B: ssl23_accept (in /usr/lib64/libssl.so.1.0.2m)
    by 0x594CDC1: ssl23_read (in /usr/lib64/libssl.so.1.0.2m)
    by 0x14BE29: tls_handshake_recv (in /usr/sbin/radiusd)
    by 0x52B115E: eaptls_process (in 
/usr/lib64/freeradius/libfreeradius-eap.so)
    by 0xBBCCD0D: ??? (in /usr/lib64/freeradius/rlm_eap_tls.so)
    by 0xAFAB571: ??? (in /usr/lib64/freeradius/rlm_eap.so)
    by 0xAFABA99: eap_method_select (in /usr/lib64/freeradius/rlm_eap.so)

559,872 bytes in 32 blocks are possibly lost in loss record 1,699 of 1,700
    at 0x4C2DB9D: malloc (vg_replace_malloc.c:299)
    by 0x55277E7: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.2m)
    by 0x5949DCC: ssl3_setup_write_buffer (in /usr/lib64/libssl.so.1.0.2m)
    by 0x5949E37: ssl3_setup_buffers (in /usr/lib64/libssl.so.1.0.2m)
    by 0x594B484: ssl23_get_client_hello (in /usr/lib64/libssl.so.1.0.2m)
    by 0x594BC5B: ssl23_accept (in /usr/lib64/libssl.so.1.0.2m)
    by 0x594CDC1: ssl23_read (in /usr/lib64/libssl.so.1.0.2m)
    by 0x14BE29: tls_handshake_recv (in /usr/sbin/radiusd)
    by 0x52B115E: eaptls_process (in 
/usr/lib64/freeradius/libfreeradius-eap.so)
    by 0xBBCCD0D: ??? (in /usr/lib64/freeradius/rlm_eap_tls.so)
    by 0xAFAB571: ??? (in /usr/lib64/freeradius/rlm_eap.so)
    by 0xAFABA99: eap_method_select (in /usr/lib64/freeradius/rlm_eap.so)


I am unsure on how to proceed next though. Any advice? Could it be a 
misconfiguration? Could it be a problem in OpenSSL? I have freeRadius 
3.0.15 and OpenSSL 1.0.2m on Fedora 25.

Thanks in advance, and regards,
Nicolas