[RE]Problem with proxied accounting

Wed Jul 29 17:19:14 CEST 2009

An HTML attachment was scrubbed...
URL: <http://lists.freeradius.org/pipermail/freeradius-users/attachments/20090729/275ebf16/attachment.html>
-------------- next part --------------

Thanks, Alan for the advice!

We have just upgraded to FreeRADIUS 2.1.6, but unfortunately the problem persists:

Server1/:
total 1008236
-rw-------  1 root root 714751062 Jul 29 11:53 detail-20090729
-rw-------  1 root root 316653344 Jul 29 07:29 detail.work <= stuck!

We got sometimes the following message for Server2:

Thu Jul 23 15:00:45 2009 : Proxy: No outstanding request was found for proxy reply from home server <Server2 IP address> port 1813 - ID 142
And several messages for Server1:

Wed Jul 29 11:36:53 2009 : Error: Rejecting request 3993531 due to lack of any response from home server <Server1 IP address> port 1813
Wed Jul 29 11:36:53 2009 : Error: PROXY: Marking home server <Server1 IP address> port 1813 as zombie (it looks like it is dead).
Wed Jul 29 11:37:28 2009 : Info: Suspicious proxy state... continuing
Wed Jul 29 11:37:30 2009 : Error: Rejecting request 3998634 due to lack of any response from home server <Server1 IP address> port 1813
<...>

In an attempt to force the revival of Server1, we scheduled the execution of the following command every minute:

radmin -e "set home_server state <Server1 IP address> 1813 alive"

But still, it seems to stuck ... I checked with "tcpdump port 1813 and host <Server1 IP address>" that even though the details.work for Server1 is freezed, FreeRADIUS is sending accounting requests to Server1 and it is receiving accounting responses.

# tcpdump port 1813 and host <Server1 IP address>
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
11:39:27.667255 IP <FreeRADIUS IP address>.1814 > Server1.radius-acct: RADIUS, Accounting Request (4), id: 0x0c length: 202
11:39:27.675969 IP Server1.radius-acct > <FreeRADIUS IP address>.1814: RADIUS, Accounting Response (5), id: 0x0c length: 20
<...>

I have many many questions of how FreeRADIUS proxy works. Could someone please help us understand what we doing wrong?

1) Do you think that Server1 detail.work get "stuck", because FreeRADIUS detected that some of the accouting requests in the detail.work didn't have a response from Server1?

2) FreeRADIUS is still sending accounting requests and receiving responses for Server1, just because we are setting Server1 alive? When we do this, it starts processing the details.work from the beginning of the file? After sometime, FreeRADIUS proxy stops completely. Is it because FreeRADIUS had too many accounting requests without responses?

3) Server1 receives all accounting requests received by FreeRADIUS, and Server2 receives accounting requests that matches a filter. I've noticed that Server1 and Server2 accounting responses for the same accounting requests have the same Packet Identifier, and that the NAS frequently reuses this Packet Identifier. Do you think that FreeRADIUS can get lost in this situation?

Please help us!

Thanks, Cristina Miyata

-----------------
Cristina Miyata wrote:
> We are using Freeradius 2.1.1 and we send accounting RADIUS to 2 different servers called Server1 and Server2. In order to do so, we created two proxy servers and 3 detailed accouting logs: detail (stored in the server), detail1 (processed by the proxy server that send accounting to Server1) and detail2 (processed by the proxy server that send accounting to Server2).

  I'd suggest upgrading to 2.1.6.  It fixes some issues when reading
from detail files.

  Alan DeKok.

---------[ Received Mail Content ]----------
Subject : Problem with proxied accounting
Date : Tue, 21 Jul 2009 21:28:58 -0400 (EDT)
>From : "Cristina Miyata" <cmiyata at lycos.com>
To : <freeradius-users at lists.freeradius.org>

 p {margin-top:0px;margin-bottom:0px;} 
Dear FreeRADIUS Users,

We are using Freeradius 2.1.1 and we send accounting RADIUS to 2 different servers called Server1 and Server2. In order to do so, we created two proxy servers and 3 detailed accouting logs: detail (stored in the server), detail1 (processed by the proxy server that send accounting to Server1) and detail2 (processed by the proxy server that send accounting to Server2).

For a while, the proxy serves works fine. Then one of them starts logging reject request due to lack of any response from home server <Server1 IP address> port 1813:

Tue Jul 21 22:03:29 2009 : Error: Rejecting request 38447540 due to lack of any response from home server <Server1 IP address> port 1813
Tue Jul 21 22:03:29 2009 : Error: PROXY: Marking home server <Server1 IP address> port 1813 as zombie (it looks like it is dead).

The proxy server for Server2 also stops working from time to time, but doesn't log any errors in "radius.log" file. The details file for the proxy gets larger and larger, and it seems to be consumed very very slowly (can see accounting being sent to Server2) or not consumed (simply stops sending accouting to Server2) by the proxy server:

total 4265476
-rw-------  1 root root 1246037041 Jul 20 23:59 detail-20090720
-rw-------  1 root root 3117400401 Jul 21 21:49 detail-20090721
-rw-------  1 root root     107664 Jul 20 16:59 detail.work

Server's CPU utilization is low and memory utilization is high (4GB in use and 33MB free).

Does anyone has a clue what is going on with our FreeRADIUS? Really don't know what to do.

Appreciate any help.

Thanks,

Cristina Miyata