3.0.x crash in _talloc_free()

[Arran Cudbard-Bell]

> On 18 Nov 2015, at 07:47, Matthew Newton <mcn4 at leicester.ac.uk> wrote:
> 
> Hi,
> 
> Just put our first real FR3 deployment live yesterday (3.0.x HEAD
> 46fb0cd312f0) :) Then quickly hit a crash :(
> 
> Initial outputs (was running -X to start so these are single
> threaded) gave:
> 
> ...
> (294) eap_tls: ocsp: Cert status: good
> (294) eap_tls: ocsp: Certificate is valid
> (294) eap_tls: TLS_accept: SSLv3 read client certificate A
> (294) eap_tls: <<< recv TLS 1.0 Handshake [length 0046], ClientKeyExchange
> *** glibc detected *** freeradius: free(): invalid pointer: 0x0000000001c7a140 ***
> ======= Backtrace: =========
> ...
> 
> and
> 
> ...
> Waking up in 0.1 seconds.
> (4563) Cleaning up request packet ID 8 with timestamp +1917
> (4564) Cleaning up request packet ID 9 with timestamp +1917
> Waking up in 0.2 seconds.
> (4566) Cleaning up request packet ID 207 with timestamp +1917
> *** glibc detected *** /usr/sbin/freeradius: free(): invalid next size (fast): 0x00000000019f4830 ***
> ======= Backtrace: =========
> ...
> 
> Haven't manage to get a core dump yet, but the panic action is
> working. Three crashes so far are the same, one slightly
> different. All are in _talloc_free: The three same ones error in
> the request_free (process.c:806) at the end of request_done. The
> other is in eappeap_process (peap.c:828).
> 
> The fact it manages to get all the way through request_free
> without crashing, with all the checks that are done in there,
> before bombing out in the free seem slightly weird to me, as the
> data in *request must be sane.
> 
> System is Debian wheezy with talloc 2.0.7. I think my next plan is
> to upgrade the system to jessie which has version 2.1.1 available
> and see if it's a bug in that old version of talloc that's being
> hit. Weird nobody else has reported it before, though; it's not a
> busy server and crashed four times so far this morning (though not
> at all overnight when less lightly loaded).
> 
> Full startup output from -X and four gdb panic action dumps are at
> (yeah I know there are a few warnings in the config at the moment):
> 
>  https://gist.github.com/mcnewton/63a01685f3748e202e81
> 
> One snippet of panic_action output below.
> 
> Any ideas?

Valgrind :/ Sounds like malloc memory header has been corrupted.

-Arran

Arran Cudbard-Bell <a.cudbardb at freeradius.org>
FreeRADIUS development team

FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 872 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freeradius.org/pipermail/freeradius-devel/attachments/20151118/1ebe5e10/attachment.sig>