Freeradius not sending traffic-shaping attributes randomly (Works when in debug mode)
Hello everyone, I am facing a strange problem with one radius server (FreeBSD 12.2 + FreeRADIUS 2.0.21). In a random way, Freeradius is not sending some traffic-shaping attributes in the Access-Accept to the NAS (I confirmed it with wireshark). What is puzzling me is that when I run radiusd in debug mode (-X), the problem disappears. I tried using raddebug but it uses a lot of CPU power, so it is not viable to run it in production. What do you guys suggest? -- Att, *Antônio Modesto*
On Dec 15, 2021, at 3:31 PM, Antônio Modesto <modesto@hubsoft.com.br> wrote:
I am facing a strange problem with one radius server (FreeBSD 12.2 + FreeRADIUS 2.0.21).
Which is end of life, dead, and not supported.
In a random way, Freeradius is not sending some traffic-shaping attributes in the Access-Accept to the NAS (I confirmed it with wireshark). What is puzzling me is that when I run radiusd in debug mode (-X), the problem disappears.
Where do these attributes come from? You've been careful to not give any information about your configuration. The server doesn't magically decide to behave differently from day to day. In 99.999% of these situations, the database is overloaded. So FreeRADIUS can't get the data it needs, and then things break. If the server just uses flat-text files (and no database), then it will *never* suddenly start behaving differently. It will *always* give exactly the same output for the same input.
I tried using raddebug but it uses a lot of CPU power, so it is not viable to run it in production. What do you guys suggest?
Upgrade. And investigate database issues. Check the radius.log file. Odds are that there will be LOTS of complaints about "unresponsive child" or "module SQL is blocked". Fix that. Alan DeKok.
On 15/12/2021 17:59, Alan DeKok wrote:
On Dec 15, 2021, at 3:31 PM, Antônio Modesto <modesto@hubsoft.com.br> wrote:
I am facing a strange problem with one radius server (FreeBSD 12.2 + FreeRADIUS 2.0.21). Which is end of life, dead, and not supported.
In a random way, Freeradius is not sending some traffic-shaping attributes in the Access-Accept to the NAS (I confirmed it with wireshark). What is puzzling me is that when I run radiusd in debug mode (-X), the problem disappears. Where do these attributes come from? You've been careful to not give any information about your configuration.
The server doesn't magically decide to behave differently from day to day. In 99.999% of these situations, the database is overloaded. So FreeRADIUS can't get the data it needs, and then things break.
If the server just uses flat-text files (and no database), then it will *never* suddenly start behaving differently. It will *always* give exactly the same output for the same input.
I tried using raddebug but it uses a lot of CPU power, so it is not viable to run it in production. What do you guys suggest? Upgrade.
And investigate database issues. Check the radius.log file. Odds are that there will be LOTS of complaints about "unresponsive child" or "module SQL is blocked". Fix that.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.htm
I am sorry, I am using Freeradius 3.0.21, not 2.0.21. But regarding the attributes, they are being read from the database (PostgreSQL). What you said about database issues does not make sense to me. If freeradius can't read something from the database, it should halt the request, not respond a Access-Accept with truncated attributes. As I said before, the weird part is that running freeradius in debug-mode (single-threaded) the problem didn't happen. I will try to upgrade to the last build and see what happens. Thanks.
l -- Att, *Antônio Modesto*
On Dec 15, 2021, at 4:21 PM, Antônio Modesto <modesto@hubsoft.com.br> wrote:
I am sorry, I am using Freeradius 3.0.21, not 2.0.21.
OK, that's good.
But regarding the attributes, they are being read from the database (PostgreSQL). What you said about database issues does not make sense to me. If freeradius can't read something from the database, it should halt the request, not respond a Access-Accept with truncated attributes.
It depends on how you configure the server. It's perfectly possible to have it return Access-Accept if the DB is down. So... how did you configure the server?
As I said before, the weird part is that running freeradius in debug-mode (single-threaded) the problem didn't happen.
That doesn't mean anything. If the issue happens randomly, then it would be rare for it to happen when you're running in debug mode for a short time.
I will try to upgrade to the last build and see what happens.
It would also help to try to understand what you've configured the server to do. Where are the attributes coming from? What happens if the DB is down? Alan DeKok.
On 15/12/2021 21:29, Alan DeKok wrote:
On Dec 15, 2021, at 4:21 PM, Antônio Modesto<modesto@hubsoft.com.br> wrote:
I am sorry, I am using Freeradius 3.0.21, not 2.0.21. OK, that's good.
But regarding the attributes, they are being read from the database (PostgreSQL). What you said about database issues does not make sense to me. If freeradius can't read something from the database, it should halt the request, not respond a Access-Accept with truncated attributes. It depends on how you configure the server. It's perfectly possible to have it return Access-Accept if the DB is down.
So... how did you configure the server?
As I said before, the weird part is that running freeradius in debug-mode (single-threaded) the problem didn't happen. That doesn't mean anything. If the issue happens randomly, then it would be rare for it to happen when you're running in debug mode for a short time.
I will try to upgrade to the last build and see what happens. It would also help to try to understand what you've configured the server to do.
Where are the attributes coming from? What happens if the DB is down?
Alan DeKok.
The attributes are coming from the database, from the radgroupreply table. My configuration is basically the default one with minor adjustments. I use attr_filter to not send attributes from one vendor to another, but I already checked that and it is not the source of the problem. I think it may be caused by the bug reported by our friend in a previous message. Thanks a lot.
- List info/subscribe/unsubscribe? Seehttp://www.freeradius.org/list/users.html
-- Att, *Antônio Modesto*
On Dec 16, 2021, at 6:39 AM, Antônio Modesto <modesto@hubsoft.com.br> wrote:
The attributes are coming from the database, from the radgroupreply table.
It shouldn't take 4 rounds of messages for you to say this. The more information you give, the easier it is for us to help you. If you don't give any useful information, we're left guessing.
My configuration is basically the default one with minor adjustments. I use attr_filter to not send attributes from one vendor to another, but I already checked that and it is not the source of the problem. I think it may be caused by the bug reported by our friend in a previous message.
Except that when the SQL module can't get the group membership information, the module returns "fail". Which results in an Access-Reject. The ONLY way for it to return Access-Accept is if you edited the configuration, so that the server ignored the sql module "fail". So again, you should understand what you did to your local configuration. The server works exactly as you *want*, in that failed group lookups result in reject. However, the server also works exactly how you *configured* it, which is to over-ride the default, and instead return "accept" when group lookups fail. The lesson here is (a) describe what you're doing in detail, and (b) understand what you've configured the server to do. Alan DeKok.
On Wed, 15 Dec 2021 at 21:22, Antônio Modesto <modesto@hubsoft.com.br> wrote:
On 15/12/2021 17:59, Alan DeKok wrote:
On Dec 15, 2021, at 3:31 PM, Antônio Modesto <modesto@hubsoft.com.br> wrote:
I am facing a strange problem with one radius server (FreeBSD 12.2 + FreeRADIUS 2.0.21). Which is end of life, dead, and not supported.
In a random way, Freeradius is not sending some traffic-shaping attributes in the Access-Accept to the NAS (I confirmed it with wireshark). What is puzzling me is that when I run radiusd in debug mode (-X), the problem disappears. Where do these attributes come from? You've been careful to not give any information about your configuration.
The server doesn't magically decide to behave differently from day to day. In 99.999% of these situations, the database is overloaded. So FreeRADIUS can't get the data it needs, and then things break.
If the server just uses flat-text files (and no database), then it will *never* suddenly start behaving differently. It will *always* give exactly the same output for the same input.
I tried using raddebug but it uses a lot of CPU power, so it is not viable to run it in production. What do you guys suggest? Upgrade.
And investigate database issues. Check the radius.log file. Odds are that there will be LOTS of complaints about "unresponsive child" or "module SQL is blocked". Fix that.
I am sorry, I am using Freeradius 3.0.21, not 2.0.21.
But regarding the attributes, they are being read from the database (PostgreSQL). What you said about database issues does not make sense to me. If freeradius can't read something from the database, it should halt the request, not respond a Access-Accept with truncated attributes. As I said before, the weird part is that running freeradius in debug-mode (single-threaded) the problem didn't happen.
If the missing attributes are derived from group lookups then it is likely fixed by this: https://github.com/FreeRADIUS/freeradius-server/commit/f0b48c07ba986b09c75fb...
On 16/12/2021 08:07, Terry Burton wrote:
On Wed, 15 Dec 2021 at 21:22, Antônio Modesto<modesto@hubsoft.com.br> wrote:
On 15/12/2021 17:59, Alan DeKok wrote:
On Dec 15, 2021, at 3:31 PM, Antônio Modesto<modesto@hubsoft.com.br> wrote:
I am facing a strange problem with one radius server (FreeBSD 12.2 + FreeRADIUS 2.0.21). Which is end of life, dead, and not supported.
In a random way, Freeradius is not sending some traffic-shaping attributes in the Access-Accept to the NAS (I confirmed it with wireshark). What is puzzling me is that when I run radiusd in debug mode (-X), the problem disappears. Where do these attributes come from? You've been careful to not give any information about your configuration.
The server doesn't magically decide to behave differently from day to day. In 99.999% of these situations, the database is overloaded. So FreeRADIUS can't get the data it needs, and then things break.
If the server just uses flat-text files (and no database), then it will *never* suddenly start behaving differently. It will *always* give exactly the same output for the same input.
I tried using raddebug but it uses a lot of CPU power, so it is not viable to run it in production. What do you guys suggest? Upgrade.
And investigate database issues. Check the radius.log file. Odds are that there will be LOTS of complaints about "unresponsive child" or "module SQL is blocked". Fix that.
I am sorry, I am using Freeradius 3.0.21, not 2.0.21.
But regarding the attributes, they are being read from the database (PostgreSQL). What you said about database issues does not make sense to me. If freeradius can't read something from the database, it should halt the request, not respond a Access-Accept with truncated attributes. As I said before, the weird part is that running freeradius in debug-mode (single-threaded) the problem didn't happen. If the missing attributes are derived from group lookups then it is likely fixed by this: https://github.com/FreeRADIUS/freeradius-server/commit/f0b48c07ba986b09c75fb...
That's exactly the case. The attributes are attached to a group, not directly to the user. I will try to upgrade and see what happens.
- List info/subscribe/unsubscribe? Seehttp://www.freeradius.org/list/users.html
-- Att, *Antônio Modesto*
participants (3)
-
Alan DeKok -
Antônio Modesto -
Terry Burton