fr_packet_cmp again

[Josip Almasi]

Alan DeKok wrote:
> Josip Almasi wrote:
>> Died 7 mins after with
>> Error: ASSERT FAILED threads.c[423]: request->magic == REQUEST_MAGIC
>> :/
> 
>   That's terrible.  But at least it helps track down where the problem is.

Care to elaborate?

But this terrible thing is up to RHEL 6, maybe kernel issue.
I mentioned it earlier as non-critical.
After a few hundered thousands threads, no new threads can start. Be it 
freeradius or java threads.
I kill radiusd and restart and it still cant start new threads - clone 
returns resource temporary unavailable.
Workaround is using fixed number of threads, as you explained sometimes.
To make it even wierder, if we set radiusd uid and gid to 0, it works:)
I have no clue how to set this up, sure it's not up to ulimit.

>> Could it be some sort of timing issue, i.e. I have faster box than you
>> so you cant reproduce it?
> 
>   Yes.

Want to try it on our boxes? We could set you up an openvpn account.
Or, whatever, it's lab anyway, we'll nat it to internet and give you ssh.

>> BTW I like what I saw in 3, response time under heavy load is way better.
> 
>   That's good to hear.  I'd like to see graphs, if at all possible...

Sorry no graphs.
But we narrowed it down to rlm_detail.
For auth requests, performance degrades less, to max 20 ms response 
time. Say, with about 2K auth req/s, it's still about 2ms, quite nice.
But for acct it can get to more than a second.
When we turn off detail, it's the same as for acct requests.
But when we detail to /dev/null, response grows 40 ms.
With detailing to files, we got additional 80-120 ms.
It was at about 5K acct req/s.
All this, IIRC. We may do it again if you're interested.

OTOH, FR 3 keeps response at 1-2ms at all times.

Regards...