Hey Alan, Thanks again for your help. I have a couple of questions about some things you commented:
Usually, server 2 and 3 have more stale sessions than server 1. This would tell me that there are accounting packets not processed or missing between server 1 and server 2 and 3.
Or, they're received in a different order, and that causes problems.
Could you explain a little bit more what kind of problems could this cause? Is it receiving packets in a different order for that user/session, or just receiving packets in a different order from that server? I would like to see it there is a way to minimize this behaviour. I read that the server has support for radius over tcp. I don't know anything about it, but woud it be possible to proxy between servers over tcp? Would that solve the reordering issue? The second question is about the following comment:
Now, with some NAS i also see issues where i miss accounting stop packets, and i end up with stale sessions. I am working on solving that, but in the mean time i run a script on all three servers that queries all NAS for active sessions and marks the stale sessions as Terminated.
Why? Just run the script once, and have it write RADIUS packets to a detail file. Those can be replicated to every server.
This was my first idea all along, and i have the code ready. i ran into an issue though, I started using the server with the default calculation for unique accounting session: Acct-Unique-Session-Id := "%{md5:%{User-Name},%{Acct-Session-ID},%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}},%{NAS-Identifier},%{NAS-Port-ID},%{NAS-Port}}" Now, the problem that i have is that some of my NAS send both NAS-port and NAS-Port-ID. Nas-port is not on the accounting database, so i cannot really add it to the attribute, and that makes the accounting session ID different. I would also prefer not to add it, as i dont use it for anything. The options that i see are: -Force all my NAS to not sent NAS-port (I am working on this, but it is not solved yet. I am not sure how to do it yet, or if it is possible) -Change the calculation so it does not include the nas-port attribute. This would be my preferred option, but in order to do this (since i already have sessions opened), i also need to update my database with the new calculation. How is that md5 calculated? Is it a plain md5 of the concatenation of the fields? -I also tried to just put the acct_unique_session_id attribute in the acct stop request, but it looks like the server overwrites it. Also, that is the proper behavior, so i prefer it that way Thank you very much for your help! Regards On Wed, Jul 8, 2015 at 9:35 AM, Alan DeKok <aland@deployingradius.com> wrote:
On Jul 7, 2015, at 7:32 AM, Ricardo Larrañaga <ricardo.larranaga@gmail.com> wrote:
Right now, all my NAS are only pointing to one server (Lets call it server1). So server2 and server3 do not receive authentication requests, they just receive accounting packets from server1.
That's simple.
Now, with some NAS i also see issues where i miss accounting stop packets, and i end up with stale sessions. I am working on solving that, but in the mean time i run a script on all three servers that queries all NAS for active sessions and marks the stale sessions as Terminated.
Why? Just run the script once, and have it write RADIUS packets to a detail file. Those can be replicated to every server.
My script logs the stale sessions, so i can compare them between servers. My problem is that the stale sessions between the servers are different.
Since you're doing RADIUS replication, packets can be lost. If you were doing database replication, this wouldn't happen.
Usually, server 2 and 3 have more stale sessions than server 1. This would tell me that there are accounting packets not processed or missing between server 1 and server 2 and 3.
Or, they're received in a different order, and that causes problems.
First thing i did was looking at the networking side of things. All interfaces between the servers run clean, they are all connected to the same switch.
So that should be fine.
So i am trying to troubleshoot what is happeing with these packets. The questions i have are:
1) My copy acct-to-home-server for each server configuration is as follows: - log all the incoming packets to a detail file. -copy-acct-to-home-server server reads the detail file and updates the proxy-to-realm control attribute so the packet gets proxied to the other server. Acct always returns ok.
That should work.
If i understand this correctly, in this configuration the packets are "Sent and forgotten".
No. See the debug output. The packets are retransmitted until the server receives a reply.
So if a packet fails to be received or processed by server2 or server3, it wont be retried. The fact that all the detail files are empty makes me think that this is the way the server is working right now.
i.e. all packets have received replies.
Any ideas on how could i implement a simple "retry 3 times" policy?
The server does this already.
My issue is that since i proxy the packet, and and always return ok on accounting, i am not sure where to put the "IF packet failed to ACK, return fail". From what i see, it cannot be done in the proxy section, unless i do something like: realm SERVER1 {
}
It's put into the Post-Proxy-Type Fail section.
2) I dont see anything in the logs that would indicate there is a problem with packets being sent to the other to servers. How can i catch these issues from the logs or debug perspective?
If there's an error, it will be logged to radius.log.
the only thing i can think of is to turn debug on for all the packets with packet-source-destination equals "server 2" and server 3" in server one, and go from there, but any suggestions would be appreciated, as that would basically be my whole radius acct traffic.
It's RADIUS... it's imperfect. If you want more reliable replication, use database replication. But that has it's own set of issues.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On Wed, Jul 8, 2015 at 9:35 AM, Alan DeKok <aland@deployingradius.com> wrote:
On Jul 7, 2015, at 7:32 AM, Ricardo Larrañaga <ricardo.larranaga@gmail.com> wrote:
Right now, all my NAS are only pointing to one server (Lets call it server1). So server2 and server3 do not receive authentication requests, they just receive accounting packets from server1.
That's simple.
Now, with some NAS i also see issues where i miss accounting stop packets, and i end up with stale sessions. I am working on solving that, but in the mean time i run a script on all three servers that queries all NAS for active sessions and marks the stale sessions as Terminated.
Why? Just run the script once, and have it write RADIUS packets to a detail file. Those can be replicated to every server.
My script logs the stale sessions, so i can compare them between servers. My problem is that the stale sessions between the servers are different.
Since you're doing RADIUS replication, packets can be lost. If you were doing database replication, this wouldn't happen.
Usually, server 2 and 3 have more stale sessions than server 1. This would tell me that there are accounting packets not processed or missing between server 1 and server 2 and 3.
Or, they're received in a different order, and that causes problems.
First thing i did was looking at the networking side of things. All interfaces between the servers run clean, they are all connected to the same switch.
So that should be fine.
So i am trying to troubleshoot what is happeing with these packets. The questions i have are:
1) My copy acct-to-home-server for each server configuration is as follows: - log all the incoming packets to a detail file. -copy-acct-to-home-server server reads the detail file and updates the proxy-to-realm control attribute so the packet gets proxied to the other server. Acct always returns ok.
That should work.
If i understand this correctly, in this configuration the packets are "Sent and forgotten".
No. See the debug output. The packets are retransmitted until the server receives a reply.
So if a packet fails to be received or processed by server2 or server3, it wont be retried. The fact that all the detail files are empty makes me think that this is the way the server is working right now.
i.e. all packets have received replies.
Any ideas on how could i implement a simple "retry 3 times" policy?
The server does this already.
My issue is that since i proxy the packet, and and always return ok on accounting, i am not sure where to put the "IF packet failed to ACK, return fail". From what i see, it cannot be done in the proxy section, unless i do something like: realm SERVER1 {
}
It's put into the Post-Proxy-Type Fail section.
2) I dont see anything in the logs that would indicate there is a problem with packets being sent to the other to servers. How can i catch these issues from the logs or debug perspective?
If there's an error, it will be logged to radius.log.
the only thing i can think of is to turn debug on for all the packets with packet-source-destination equals "server 2" and server 3" in server one, and go from there, but any suggestions would be appreciated, as that would basically be my whole radius acct traffic.
It's RADIUS... it's imperfect. If you want more reliable replication, use database replication. But that has it's own set of issues.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html