Seg-fault with proxy and fail-overs

[Alan DeKok]

Chris Moules wrote:
> The system seems to run fine for some time. Then I get a Seg-Fault. I
> was quickly able to isolate one clear cause for a
> seg-fault, which was my 'copy-acct-to-home-server' setup. Disabling the
> virtual server made the system run, apparently, fine.
> Setting up this virtual server independently by duplicating and suitably
> modifying the configuration and running with the '-d'
> option I was able to watch this system run. It was happily reading from
> the log file and sending the data to the home-server.
> After a number of records, (I noted 50-300 on various runs), the
> freeradius process would segfault. This happend, as far as I
> could see, always after the home-server failed to respond within the
> timeout (response_window) and so failed.

  Ugh.

  There are ways to track this down, but it involves re-building the
server with debugging symbols.

> I have been trying to get a core file from the system without success
> (very embarrassing):
> ulimit -c -> unlimited
> allow_core_dumps = yes
> cat /proc/sys/kernel/core_pattern -> core
> The freeradius user has write access to the location where I execute the
> program.

  Run it in a "screen" output, under "gdb".  See doc/bugs for instructions.

> Doing an strace of the process was not very insightful:

  That won't tell you much,

> Doing a 'kill -TERM' to the process when _not_ processing data produced
> this:
...
> *** glibc detected *** freeradius: corrupted double-linked list:
> 0x000000000083b1f0 ***
...
> /usr/lib/freeradius/libfreeradius-radius-2.1.4.so(fr_hash_table_free+0x93)[0x7fbe1c69db63]
> /usr/lib/freeradius/libfreeradius-radius-2.1.4.so(dict_free+0x7a)[0x7fbe1c69c1da]

  Hmm... that's not good.  It's not serious, but it's not good.

  The problem is I don't see that on my test systems.

  Some more output from "gdb" after a crash would be *enormously* helpful.

  Alan DeKok.