Radiusd hangs on redis cluster failover (sometimes)

[Milan Nikolic]

Hi,

I just wanted to send an update, maybe it will be useful for someone else
or for developers.

What I found out is that when radiusd goes to some mode
where ctrl+c/restart doesn't work and it must be killed, if I wait for at
least 15min it will recover and it works normally after that.
I first tried to change tcp_keepalive_intvl and tcp_keepalive_probes in the
kernel, and Redis keepalive but without success. Instead, what fixed it for
me was to change rediswho module pool settings,
start=0, min=0, lifetime=60, cleanup_interval=30 and idle_timeout=0.

Now everything works as expected and the Redis cluster and FreeRADIUS can
both survive force power off.

Regards,
Milan

On Fri, Aug 9, 2019 at 9:02 PM Milan Nikolic <gen2brain at gmail.com> wrote:

> Alan,
>
> Thank you for your time and patience.
> I understand that v4 is in active development and many things are missing
> or misbehave but I still need to use it.
>
> Milan
>
> On Fri, Aug 9, 2019 at 8:45 PM Alan DeKok <aland at deployingradius.com>
> wrote:
>
>> On Aug 9, 2019, at 1:27 PM, Milan Nikolic <gen2brain at gmail.com> wrote:
>> > So I now did exactly that and still have Debug lines in the log, see
>> here
>> > (you can see what command is started after script in the log)
>> > https://pastebin.com/2nWMBKRM , that is not normal or now there are
>> less
>> > Debug lines as you prefer?
>>
>>   The default configuration doesn't do that.  I'm running the server from
>> git HEAD, with the default configuration.  I've never seen to add the
>> "Debug :" output to every line.
>>
>> > If the path to directory or redirection of output can affect logging
>> than I
>> > must say is not usual behavior and can be one of the reasons why every
>> > thread has a couple of posts where just this is discussed and not the
>> > actual issue.
>>
>>   That is simply not true.  The majority of times that happens, it's for
>> v3, AND they include the full timestamp in the debug output.  Which means
>> that they have *deliberately* used "radiusd -Xx".
>>
>> > Is there anything that can be enabled in the config file to get similar
>> > output (for me, not to post to lists)? For such powerful software
>> debugging
>> > looks like the weakest thing. I tried debug_level 1, 2, 3, I was happy
>> that
>> > I can change level "on fly" with radmin tool but all useful information
>> I
>> > can only get with `-X` or `-Xx`. In my case, that means I must stop
>> > service, change config (I use some configs from ENV and it is not
>> enough to
>> > export those because service forks as a user), manually start, find the
>> > issue then change back and start service. Not practical at all in
>> > production.
>>
>>   This is where test systems come in handy.
>>
>> > Debug log is above, sorry, I had to recompile, I commented out that
>> line,
>> > because I don't want it in detail log, here is what I get:
>> >
>> > Fri Aug  9 15:34:48 2019
>> > Module-Failure-Message = "No 'accounting Start' section found: Ignoring
>> it."
>>
>>   Then you're not running the default configuration.  Which has an empty
>> "accounting start" section.
>>
>> > Should I add empty accounting Start {} section, is that a must now
>> (didn't
>> > try that, just cross my mind)? What about others, i.e. Stop/Alive, etc.
>> My
>> > "default" site config is very simple and basic, I only use
>> > Accounting-Request/Response and listen.
>>
>>   Ah.  So you deleted almost everything in the default configuration.
>> And then got surprised that the server produced a warning message.
>>
>>   That is not a productive thing to do.
>>
>>   If you're running v4, "good luck".  If it works, great.  If it doesn't,
>> submit a patch to fix it.
>>
>>   v4 is in active development, and you should NOT be running it unless
>> you know what you're doing.
>>
>>   Alan DeKok.
>>
>>
>> -
>> List info/subscribe/unsubscribe? See
>> http://www.freeradius.org/list/users.html
>
>