adrian.p.smith@bt.com wrote:

...

We have noticed that the relay of packets from these detail files stopped working with no error reported in the radius log. A restart of the radius process made it all work again.

Is this a known issue and/or has it been addressed in 3.0.4?

I would wait for 3.0.5. I've made a number of improvements, including the ability to start continue reading the detail file after the server has been restarted.

We will probably release 3.0.5 next week.

Alan DeKok.

Thanks Alan. We have found this relay functionality in version 2 to be pretty bomb-proof and have not seen issues before. Was this part of the code re-written for version 3, thus making new issues likely? Regards, Adrian

adrian.p.smith@bt.com wrote:

...

Thanks Alan. We have found this relay functionality in version 2 to be pretty bomb-proof and have not seen issues before. Was this part >> of the code re-written for version 3, thus making new issues likely?

...

Yes. The code in v2 worked, but was very very slow. The new code is much faster, and can continue reading a detail file after >partially processing it.

...

Alan DeKok.

Hello again. For information we now seem to have a reproducible problem.... If we manually create a large detail file and place it in the radacct folder, start radius, we see the packets being sent, but the process stalls someway through and never recovers until restarted. Doing the same thing on a version 2 FreeRadius works fine and the file is processed in its entirety. This feels like a definite regression?

The server *is* supposed to work.

Of course!

Did you try the v3.0.x branch? Or are you just confirming that you still see issues with 3.0.2?

The problem is with 3.0.3 which was the latest and greatest when we went into production. We wanted to use 3.x.x because we like the rlm_rest module.

As I've said, there are fixes in the v3.0.x branch in git. Please try that. https://github.com/FreeRADIUS/freeradius-server/ And grab the v3.0.x branch. Build it. Install it. Test it. Please.

Hi Alan, We have made some more progress today. We have tried using the v3.0.x branch and got similar results which led us to try further tests. Because we see the debug output saying the remote server has stopped responding we tried sending the detail file packets back to our self on the local machine. This was a total success. Then suspecting that the network between the machines was a contributing factor, we tried another "closer" machine on the same subnet and this was also successful. Reverting back to the original remote machine shows the problem still exists on both 3.0.3 and v3.0.x and hopefully the debug extract below shows what happens. Once the Status-Server message has been answered the detail processing on the sending server never re-starts even though the remote server is marked as alive. The detail.work remains in the radacct folder and any new files appearing there are also ignored. It feels like even if a remote server or network is slow or un-reliable, Freeradius should be able to recover and continue processing the detail file. Waking up in 4.3 seconds. (1014) No proxy response, giving up on request and marking it done Marking home server 192.168.49.10 port 1813 as zombie (it has not responded in 20 seconds). PING: Waiting 4 seconds for response to ping Sending Status-Server Id 146 from 0.0.0.0:53177 to 192.168.49.10:1813 Message-Authenticator := 0x00 NAS-Identifier := 'Status Check 0. Are you alive?' PING: Next status packet in 30 seconds (1014) ERROR: Failing request - proxy ID 62, due to lack of any response from home server 192.168.49.10 port 1813 (1014) Cleaning up request packet ID 246 with timestamp +12 Waking up in 0.2 seconds. Received Accounting-Response Id 146 from 192.168.49.10:1813 to 192.168.19.20:53177 length 20 (1016) Received response to status check 1016 (1 in current sequence) (1016) Marking home server 192.168.49.10 port 1813 alive Waking up in 0.2 seconds. (1015) Cleaning up request packet ID 55 with timestamp +27 Ready to process requests. Any more help appreciated. Regards, Adrian

-----Original Message----- From: Smith,AP,Adrian,TNK6 R Sent: 24 October 2014 18:09 To: FreeRadius users mailing list Subject: RE: detail server stopped working silently

...

Reverting back to the original remote machine shows the problem still exists on both 3.0.3 and v3.0.x and hopefully the debug extract below shows what happens. Once the Status-Server >message has been answered the detail processing on the sending server never re-starts even though the remote server is marked as alive. The detail.work remains in the radacct folder and >any new files appearing there are also ignored. It feels like even if a remote server or network is slow or un-reliable, Freeradius should be able to recover and continue processing the >detail file.

Hmm... the detail reader is supposed to use "retry_interval" to retransmit packets. It shouldn't matter if the home server is down or not. Packets should be sent every >"retry_interval".

Maybe the rest of the debug output will help. You *should* see it reading packets from the detail file. When the home server fails, it should say "Detail - No response to request. >Will retry in %d seconds"

The important messages are the ones where it transitions from sending packets to not sending packets...

Alan, Ran another test and the full debug log file is attached. You will notice that we are running 2 detail servers and the other one continues on, although it's not done any significant work. Regards, Adrian Zip file is held at your end for moderation - let me know if I need to send it differently.