System frequently stops responding...

[Mohamed Lrhazi]

Hello all,

Still still trying to get to the bottom of this issue... to summarize:
- Wireless controllers log that RADIUS server (a load balanced VIP), did
not respond to a query. this is logged in clusters of dozen or so, several
times a day.
- Using docker containers.. so decided to try without them
- Built two VMs, RedHat Enterprise 7, running provided freeradius
RPMs. 3.0.4
- Sending the quarter of our traffic to this pool of two VMs.
- Issue still occurs on these VMs.
- I run radiusd in -Xx mode, on both of the RHEL7 VMs, also run a
continuous tcdpump, on each VM.

- Problem occurrences seem to reliably coincide with:
-- tcpdump shows all the requests logged by the controllers having been
resent few times (duplicates in wireshark).
-- radiusd goes silent (no log at all) for 30 seconds. after which it
resumes logging and I presume, working.
- radiusd logs a line for each missed query, I think, like so:
Error: (7719) Ignoring duplicate packet from client gu_net_10 port 3010 -
ID: 96 due to unfinished request in component <core> module

-- Spikes in CPU usage (as seen in sar output).

What can I do next? to further zoom in on the root cause? Or is this pretty
clearly CPU starvation? just add more VMs ?

Thanks a lot,
Mohamed.

On Fri, Jun 26, 2015 at 6:36 PM, Mohamed Lrhazi <
Mohamed.Lrhazi at georgetown.edu> wrote:

> Oh wait.. you meant the man page, that thing in the ASCII terminal :)
>
> On Fri, Jun 26, 2015 at 6:35 PM, Mohamed Lrhazi <
> Mohamed.Lrhazi at georgetown.edu> wrote:
>
>> Weird... both my chrome and Firefox show 1922, even in view source HTML
>> mode:
>>
>>
>> <dt>debug condition '((User-Name == bob) || (Packet-Src-IP-Address == 1922))'</dt>
>>
>>
>> This is from: http://freeradius.org/radiusd/man/radmin.html
>>
>>
>>
>> On Fri, Jun 26, 2015 at 12:05 PM, <A.L.M.Buxey at lboro.ac.uk> wrote:
>>
>>> Hi,
>>>
>>> > BTW, the man page says:
>>> >
>>> > debug condition '((User-Name == bob) || (Packet-Src-IP-Address ==
>>> 1922))'
>>> > A more complex condition that enables debugging output for requests
>>> > containing User-Name "bob", or requests that originate from source IP
>>> > address 1922.
>>> >
>>> > What's that address of 1922?
>>>
>>> umm.. not whats in the radmin man page
>>>
>>> the actual text says Packet-Src-IP-Address == 192.0.2.22
>>>
>>>
>>> which I think makes this quite sensible....debug for user-name of Bob if
>>> the
>>> client has address 192.0.2.22  :-)
>>>
>>> alan
>>> -
>>> List info/subscribe/unsubscribe? See
>>> http://www.freeradius.org/list/users.html
>>
>>
>>
>