FreeRADIUS 2.X.X small functionality extensions
Greetings to all, in our company (a big commercial telecom/ISP) we run a project to completely migrate our AAA infrastructure to FreeRADIUS (we use FreeRADIUS in parts of it for many years). One of our requirements in authentication/authorization is DSLAM port checking. The port info comes in NAS-Port-Id in our case and can be extracted in a normalized form with a regular expression. The problem was that FreeRADIUS provided in variables only the first 8 matching groups in %{0}..%{8}, whereas in our setup we needed up until %{12} to get the port information. So I developed a relevant patch which I communicated through GitHub as a pull request (following procedure described in http://wiki.freeradius.org/contributing/GitHub). The patch is against 2.X.X (tested with 2.2.5) and provides two functions: - make available %{0}..%{16} in regex matches - make available a new xlat function tonumber() to translate strings as numbers In our case the port info coming from the regular expression is a string of the form 15243:1/1/02/04 and we want to transform it to 15243:1/1/2/4. I would really like to see the functionality included upstream and hopefully it can benefit others as well. Let me know what you think (and bear with me since it's my first patch to FreeRADIUS). Regards, Kostas -- Kostas Zorbadelos twitter:@kzorbadelos http://gr.linkedin.com/in/kzorba ---------------------------------------------------------------------------- () www.asciiribbon.org - against HTML e-mail & proprietary attachments /\
There are often more ways to reach a goal, I think this e-mail is a nice example of this. On 04-06-14 15:39, Kostas Zorbadelos wrote:
One of our requirements in authentication/authorization is DSLAM port checking. The port info comes in NAS-Port-Id in our case and can be extracted in a normalized form with a regular expression. The problem was that FreeRADIUS provided in variables only the first 8 matching groups in %{0}..%{8}, whereas in our setup we needed up until %{12} to get the port information.
The standard trick to do this (often used with mod_rewrite in the Apache HTTP server) is to do the rewriting in multiple steps. Disadvantage here is that unlang doesn't have a while-statement, so we'd have to know beforehand how many matches we expect.
- make available a new xlat function tonumber() to translate strings as numbers
In our case the port info coming from the regular expression is a string of the form 15243:1/1/02/04 and we want to transform it to 15243:1/1/2/4.
This is about the same as matching (.*?)0*(\d+)(.*) and overwriting the value with "%{1}%{2}%{3}". When more flexibility is required, it's pretty easy to do a call to rlm_perl (rlm_python works probably just as good, but I've never tried that). It's pretty easy to work them up from the examples that are given in the default config. -- Herwin Weststrate
On 4 Jun 2014, at 14:39, Kostas Zorbadelos <kzorba@otenet.gr> wrote:
Greetings to all,
in our company (a big commercial telecom/ISP) we run a project to completely migrate our AAA infrastructure to FreeRADIUS (we use FreeRADIUS in parts of it for many years). One of our requirements in authentication/authorization is DSLAM port checking. The port info comes in NAS-Port-Id in our case and can be extracted in a normalized form with a regular expression. The problem was that FreeRADIUS provided in variables only the first 8 matching groups in %{0}..%{8}, whereas in our setup we needed up until %{12} to get the port information.
OK then 16 does make sense. Just for our edification could you provide an example of such a port string, and provide info on the BNG and L1 termination device (if different).
So I developed a relevant patch which I communicated through GitHub as a pull request (following procedure described in http://wiki.freeradius.org/contributing/GitHub).
FreeRADIUS v2.x.x is in feature freeze, and will not feature thaw. You're welcome to resubmit against v3.0.x.
The patch is against 2.X.X (tested with 2.2.5) and provides two functions:
- make available %{0}..%{16} in regex matches - make available a new xlat function tonumber() to translate strings as numbers
As I said in the GitHub ticket, I'm not sure this is needed. Even in v2.x.x you should be able to do update request { Tmp-Integer-0 := "%{<cap group number>}" } update request { Tmp-String-0 := "%{Tmp-Integer-0}" } and it'll strip any leading zeros. You could also define your own integer type attributes and use those to record the port string components. When the string is stored in the integer attribute the leading zeros should be stripped. I'm very much against duplicative behaviour unless there's a real benefit and I don't see a benefit here. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Hello, let me be more specific to display the need. We operate DSLAMs of various vendors (we have at least 3 now and noone knows if that number will grow in the future). The BRAS/BNG terminating devices send a NAS-Port-Id as in the following examples: NAS-Port-Id -> Physical access ID (parsed form) ------------------------------------------------ (HUAWEI) <#VOIOT_LEFKTRA_D_HUA_10261 atm 0/1/0/40:8.35%Up:975kbps Down:1582#> -> 10261:0/1/0/40 <#KYKLA_EMPORIO_D_HUA_9440 atm 0/12/0/54:8.35%Up:947kbps Down:4846#> -> 9440:0/2/0/54 <#THESS_KALAM_D_HUA_13115_ONU_9502 atm 0/2/0/7:8.35#> -> 13115:0/2/0/7 <#THESS_FINIK_D_HUA_9732_ONU_9543 atm 0/1/0/57:8.35%Up:1023kbps Do#> -> 9732:0/1/0/57 <#LAKON_ELAIA_D_HUA_10157_ONU atm 0/0/0/59:8.35%Up:1023kbps Down:1#> -> 10157:0/0/0/59 <#LASIT_MOCHLOS_D_HUA_14154_KV atm 0/0/0/37:8.35%Up:1023kbps Down:#> -> 14154:0/0/0/37 <#CHALK_PSAKOUDIA_D_HUA_14214_DLC atm 0/1/0/1:8.35%Up:511kbps Down#> -> 14214:0/1/0/1 <#KERKY_POTAM_D_HUA_9857_DLC_7073 atm 0/9/0/9:8.35%Up:1023kbps Dow#> -> 9857:0/9/0/9 (ALCATEL) <#KEFAL_ARGOSTOLI_D_ALU_13748_KV atm 1/1/01/47:8.35#2671024780> -> 13748:1/1/1/47 <#ACHAI_EGLYKADA_D_ALU_14296 eth 1/1/01/42:835#2610621234> -> 14296:1/1/1/42 <#ATTIC_PENTELI_D_ALU_13822_KV eth 1/1/01/12:835#> -> 13822:1/1/1/12 <#THESS_COSMO_D_ALU_14419_ONU_8523 atm 1/1/01/03:8.35#2310476377> -> 14419:1/1/1/3 The parsed form is the string we want to extract and use it later against an LDAP repository containing ports to verify the port status and make decisions based on that. We have reached in a regex containing more than 8 groups. Also, as you can see, some DSLAMs send initial zeroes in front of numbers describing DSLAM port/slot. Given the fact that our LDAP repo will contain the parsed forms I describe could you provide an alternative config to express the desired functionality? I know I can develop a specific module for the port parsing (have it parse and store the value in a specific attribute of my choice) but I found it more elegant to have a simple extension in the already provided functionality of the core server. This will also allow me to have it in config files (changing the expression as needed in the future). I also think others can find it useful. Finally, I would rather not use rlm_perl or rlm_python (which would be my choice). Performance in our environment is critical, we host more than one million subscribers. As I said I would not like to maintain seperate patches. I understand the feature freeze of 2.x.x branch but we will not run 3.x in our environment for now, we focus on stability. In my opinion, I do not find it a "big" change or feature but of course I am not the "owner" or even master of the sources so I cannot provide an authoritative judgement on the matter. Thoughts, comments are welcome. Regards, Kostas
On 4 Jun 2014, at 17:08, Kostas Zorbadelos <kzorba@otenet.gr> wrote:
Hello,
let me be more specific to display the need. We operate DSLAMs of various vendors (we have at least 3 now and noone knows if that number will grow in the future). The BRAS/BNG terminating devices send a NAS-Port-Id as in the following examples:
NAS-Port-Id -> Physical access ID (parsed form) ------------------------------------------------ (HUAWEI) <#VOIOT_LEFKTRA_D_HUA_10261 atm 0/1/0/40:8.35%Up:975kbps Down:1582#> -> 10261:0/1/0/40 <#KYKLA_EMPORIO_D_HUA_9440 atm 0/12/0/54:8.35%Up:947kbps Down:4846#> -> 9440:0/2/0/54 <#THESS_KALAM_D_HUA_13115_ONU_9502 atm 0/2/0/7:8.35#> -> 13115:0/2/0/7 <#THESS_FINIK_D_HUA_9732_ONU_9543 atm 0/1/0/57:8.35%Up:1023kbps Do#> -> 9732:0/1/0/57 <#LAKON_ELAIA_D_HUA_10157_ONU atm 0/0/0/59:8.35%Up:1023kbps Down:1#> -> 10157:0/0/0/59 <#LASIT_MOCHLOS_D_HUA_14154_KV atm 0/0/0/37:8.35%Up:1023kbps Down:#> -> 14154:0/0/0/37 <#CHALK_PSAKOUDIA_D_HUA_14214_DLC atm 0/1/0/1:8.35%Up:511kbps Down#> -> 14214:0/1/0/1 <#KERKY_POTAM_D_HUA_9857_DLC_7073 atm 0/9/0/9:8.35%Up:1023kbps Dow#> -> 9857:0/9/0/9
(ALCATEL) <#KEFAL_ARGOSTOLI_D_ALU_13748_KV atm 1/1/01/47:8.35#2671024780> -> 13748:1/1/1/47 <#ACHAI_EGLYKADA_D_ALU_14296 eth 1/1/01/42:835#2610621234> -> 14296:1/1/1/42 <#ATTIC_PENTELI_D_ALU_13822_KV eth 1/1/01/12:835#> -> 13822:1/1/1/12 <#THESS_COSMO_D_ALU_14419_ONU_8523 atm 1/1/01/03:8.35#2310476377> -> 14419:1/1/1/3
The parsed form is the string we want to extract and use it later against an LDAP repository containing ports to verify the port status and make decisions based on that. We have reached in a regex containing more than 8 groups. Also, as you can see, some DSLAMs send initial zeroes in front of numbers describing DSLAM port/slot.
You can strip those out trivially in a regex... Say you had /1/01/01/12 Your regex would be /0?([0-9]+)/0?([0-9]+)/0?([0-9]+)/0?([0-9]+) https://www.debuggex.com/r/sgPccw67Z_fviF9u The optional 0s are consumed before the capture group data starts. In the examples above there still only appears to be 5 bits of data being pulled out. For v2.x.x the 0 stripping stuff can be done in a bunch of ways without requiring extra code, so that's definitely not going to be added. Alan has final on the number of capture groups and whether they'll be increased for v2.x.x. I suspect they won't be as it's in feature freeze. It's the sort of thing that would be quite nice to define at build time TBH. Instead of increasing it for everyone, as there is a performance penalty.
Given the fact that our LDAP repo will contain the parsed forms I describe could you provide an alternative config to express the desired functionality? I know I can develop a specific module for the port parsing (have it parse and store the value in a specific attribute of my choice) but I found it more elegant to have a simple extension in the already provided functionality of the core server. This will also allow me to have it in config files (changing the expression as needed in the future). I also think others can find it useful. Finally, I would rather not use rlm_perl or rlm_python (which would be my choice). Performance in our environment is critical, we host more than one million subscribers.
That's pretty small for an ISP TBH, but sure, throwing rlm_perl into the mix is probably a bad idea, and the GIL hurts rlm_python's performance in a multithreaded environment.
As I said I would not like to maintain seperate patches. I understand the feature freeze of 2.x.x branch but we will not run 3.x in our environment for now, we focus on stability.
Sure. I'm assuming that's because you don't have the facilities to properly test your RADIUS service, and don't want to introduce changes where you can't fully determine what the impact will be.
In my opinion, I do not find it a "big" change or feature but of course I am not the "owner" or even master of the sources so I cannot provide an authoritative judgement on the matter.
Well the LDAP code doesn't suck, you'd probably get a marked performance improvement just from using the new rlm_ldap module. In general the policy language is more efficient, and there's a greater number of validation checks on startup. It's generally a lot more consistent with v2.x.x. But i'm not here to sell it to you. Upgrade when you want... or don't. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
We are an ISP with around 800k subs and are running 2.1.12 since that's what comes with RHEL 6.5 and use perl for the slightly more complex things you can't easily do with unlang. I was running performance testing against our ldap database last week using a custom JMeter sampler I built to make the radius calls using tinyradius as the client. Managed to get up to 760tps where a transaction was one access request and two accounting starts (so 2280rps). I suspect that the performance hit was coming from jmeter as those numbers smashed anything the bngs could throw at us and the cpu never went above 8% so I am happy enough. So it's really a matter of how important testing the solution is. For us it is very important so now I have a full functional and non functional test script in jmeter... it's ugly but gives me what I wanted. Cheers Peter On 5/06/2014 5:42 AM, "Arran Cudbard-Bell" <a.cudbardb@freeradius.org> wrote:
On 4 Jun 2014, at 17:08, Kostas Zorbadelos <kzorba@otenet.gr> wrote:
Hello,
let me be more specific to display the need. We operate DSLAMs of various vendors (we have at least 3 now and noone knows if that number will grow in the future). The BRAS/BNG terminating devices send a NAS-Port-Id as in the following examples:
NAS-Port-Id -> Physical access ID (parsed form) ------------------------------------------------ (HUAWEI) <#VOIOT_LEFKTRA_D_HUA_10261 atm 0/1/0/40:8.35%Up:975kbps Down:1582#> -> 10261:0/1/0/40 <#KYKLA_EMPORIO_D_HUA_9440 atm 0/12/0/54:8.35%Up:947kbps Down:4846#> -> 9440:0/2/0/54 <#THESS_KALAM_D_HUA_13115_ONU_9502 atm 0/2/0/7:8.35#> -> 13115:0/2/0/7 <#THESS_FINIK_D_HUA_9732_ONU_9543 atm 0/1/0/57:8.35%Up:1023kbps Do#> -> 9732:0/1/0/57 <#LAKON_ELAIA_D_HUA_10157_ONU atm 0/0/0/59:8.35%Up:1023kbps Down:1#> -> 10157:0/0/0/59 <#LASIT_MOCHLOS_D_HUA_14154_KV atm 0/0/0/37:8.35%Up:1023kbps Down:#> -> 14154:0/0/0/37 <#CHALK_PSAKOUDIA_D_HUA_14214_DLC atm 0/1/0/1:8.35%Up:511kbps Down#> -> 14214:0/1/0/1 <#KERKY_POTAM_D_HUA_9857_DLC_7073 atm 0/9/0/9:8.35%Up:1023kbps Dow#> -> 9857:0/9/0/9
(ALCATEL) <#KEFAL_ARGOSTOLI_D_ALU_13748_KV atm 1/1/01/47:8.35#2671024780> -> 13748:1/1/1/47 <#ACHAI_EGLYKADA_D_ALU_14296 eth 1/1/01/42:835#2610621234> -> 14296:1/1/1/42 <#ATTIC_PENTELI_D_ALU_13822_KV eth 1/1/01/12:835#> -> 13822:1/1/1/12 <#THESS_COSMO_D_ALU_14419_ONU_8523 atm 1/1/01/03:8.35#2310476377> -> 14419:1/1/1/3
The parsed form is the string we want to extract and use it later against an LDAP repository containing ports to verify the port status and make decisions based on that. We have reached in a regex containing more than 8 groups. Also, as you can see, some DSLAMs send initial zeroes in front of numbers describing DSLAM port/slot.
You can strip those out trivially in a regex...
Say you had /1/01/01/12
Your regex would be
/0?([0-9]+)/0?([0-9]+)/0?([0-9]+)/0?([0-9]+)
https://www.debuggex.com/r/sgPccw67Z_fviF9u
The optional 0s are consumed before the capture group data starts.
In the examples above there still only appears to be 5 bits of data being pulled out.
For v2.x.x the 0 stripping stuff can be done in a bunch of ways without requiring extra code, so that's definitely not going to be added.
Alan has final on the number of capture groups and whether they'll be increased for v2.x.x. I suspect they won't be as it's in feature freeze.
It's the sort of thing that would be quite nice to define at build time TBH. Instead of increasing it for everyone, as there is a performance penalty.
Given the fact that our LDAP repo will contain the parsed forms I describe could you provide an alternative config to express the desired functionality? I know I can develop a specific module for the port parsing (have it parse and store the value in a specific attribute of my choice) but I found it more elegant to have a simple extension in the already provided functionality of the core server. This will also allow me to have it in config files (changing the expression as needed in the future). I also think others can find it useful. Finally, I would rather not use rlm_perl or rlm_python (which would be my choice). Performance in our environment is critical, we host more than one million subscribers.
That's pretty small for an ISP TBH, but sure, throwing rlm_perl into the mix is probably a bad idea, and the GIL hurts rlm_python's performance in a multithreaded environment.
As I said I would not like to maintain seperate patches. I understand the feature freeze of 2.x.x branch but we will not run 3.x in our environment for now, we focus on stability.
Sure. I'm assuming that's because you don't have the facilities to properly test your RADIUS service, and don't want to introduce changes where you can't fully determine what the impact will be.
In my opinion, I do not find it a "big" change or feature but of course I am not the "owner" or even master of the sources so I cannot provide an authoritative judgement on the matter.
Well the LDAP code doesn't suck, you'd probably get a marked performance improvement just from using the new rlm_ldap module.
In general the policy language is more efficient, and there's a greater number of validation checks on startup. It's generally a lot more consistent with v2.x.x.
But i'm not here to sell it to you. Upgrade when you want... or don't.
Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team
FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On 4 Jun 2014, at 19:11, Peter Lambrechtsen <peter@crypt.co.nz> wrote:
We are an ISP with around 800k subs and are running 2.1.12 since that's what comes with RHEL 6.5 and use perl for the slightly more complex things you can't easily do with unlang.
I was running performance testing against our ldap database last week using a custom JMeter sampler I built to make the radius calls using tinyradius as the client. Managed to get up to 760tps where a transaction was one access request and two accounting starts (so 2280rps). I suspect that the performance hit was coming from jmeter as those numbers smashed anything the bngs could throw at us and the cpu never went above 8% so I am happy enough.
For comparison lightly tuned v3.0.x gets around 22,000-25,000 RPS against the latest version of OpenLDAP with LMDB, so although it may be good enough for your purposes, that represents a huge slow down over what's possible. Even with BDB (properly tuned) you should expect a rate of around 12,000 RPS (though i've not personally tested that). As with all RADIUS deployments the problem isn't handling day to day load, it's when your access layer (be it DSLAMs, APs, or Ethernet switches) goes offline and comes back, and you suddenly have to deal with extremely high continuous load. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Now, this is a nice coincidence, switch in the thread context. I would also like to know performance numbers and seached very briefly for RADIUS stress test tools but the most promising I found, radperf, is unavailable.
We are an ISP with around 800k subs and are running 2.1.12 since that's what comes with RHEL 6.5 and use perl for the slightly more complex things you can't easily do with unlang.
I was running performance testing against our ldap database last week using a custom JMeter sampler I built to make the radius calls using tinyradius as the client. Managed to get up to 760tps where a transaction was one access request and two accounting starts (so 2280rps). I suspect that the performance hit was coming from jmeter as those numbers smashed anything the bngs could throw at us and the cpu never went above 8% so I am happy enough.
Very nice to know, thanks.
For comparison lightly tuned v3.0.x gets around 22,000-25,000 RPS against the latest version of OpenLDAP with LMDB, so although it may be good enough for your purposes, that represents a huge slow down over what's possible.
Even with BDB (properly tuned) you should expect a rate of around 12,000 RPS (though i've not personally tested that).
Since we do not have OpenLDAP, but former SUN's JES, I am interested to know how you perfomed the tests. Custom developed tool, or anything generally available?
As with all RADIUS deployments the problem isn't handling day to day load, it's when your access layer (be it DSLAMs, APs, or Ethernet switches) goes offline and comes back, and you suddenly have to deal with extremely high continuous load.
Exactly. Regards, Kostas
On 5 Jun 2014, at 10:08, Kostas Zorbadelos <kzorba@otenet.gr> wrote:
Now, this is a nice coincidence, switch in the thread context.
I would also like to know performance numbers and seached very briefly for RADIUS stress test tools but the most promising I found, radperf, is unavailable.
I believe Network RADIUS is still offering it with support contracts. But yes, the download link should be removed or fixed, leaving it broken like that is annoying.
We are an ISP with around 800k subs and are running 2.1.12 since that's what comes with RHEL 6.5 and use perl for the slightly more complex things you can't easily do with unlang.
I was running performance testing against our ldap database last week using a custom JMeter sampler I built to make the radius calls using tinyradius as the client. Managed to get up to 760tps where a transaction was one access request and two accounting starts (so 2280rps). I suspect that the performance hit was coming from jmeter as those numbers smashed anything the bngs could throw at us and the cpu never went above 8% so I am happy enough.
Very nice to know, thanks.
For comparison lightly tuned v3.0.x gets around 22,000-25,000 RPS against the latest version of OpenLDAP with LMDB, so although it may be good enough for your purposes, that represents a huge slow down over what's possible.
Even with BDB (properly tuned) you should expect a rate of around 12,000 RPS (though i've not personally tested that).
Since we do not have OpenLDAP, but former SUN's JES, I am interested to know how you perfomed the tests. Custom developed tool, or anything generally available?
FreeRADIUS v3.0.x against LDAP with 100 workers, and radperf running 5M requests with 100 in parallel. I increased the number of rlm_ldap calls per request to find out the max TPS of OpenLDAP. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
On 5/06/2014 9:09 PM, "Kostas Zorbadelos" <kzorba@otenet.gr> wrote:
Now, this is a nice coincidence, switch in the thread context.
I would also like to know performance numbers and seached very briefly for RADIUS stress test tools but the most promising I found, radperf, is unavailable.
We are an ISP with around 800k subs and are running 2.1.12 since that's what comes with RHEL 6.5 and use perl for the slightly more complex things you can't easily do with unlang.
I was running performance testing against our ldap database last week using a custom JMeter sampler I built to make the radius calls using tinyradius as the client. Managed to get up to 760tps where a transaction was one access request and two accounting starts (so 2280rps). I suspect that the performance hit was coming from jmeter as those numbers smashed anything the bngs could throw at us and the cpu never went above 8% so I am happy enough.
Very nice to know, thanks.
Happy to email you the two extra jars you need offline plus some instructions on how to drive it. Not sure if it's something that would be useful on the wiki or not.
For comparison lightly tuned v3.0.x gets around 22,000-25,000 RPS against the latest version of OpenLDAP with LMDB, so although it may be good enough for your purposes, that represents a huge slow down over what's possible.
Even with BDB (properly tuned) you should expect a rate of around 12,000 RPS (though i've not personally tested that).
Since we do not have OpenLDAP, but former SUN's JES, I am interested to know how you perfomed the tests. Custom developed tool, or anything generally available?
As with all RADIUS deployments the problem isn't handling day to day load, it's when your access layer (be it DSLAMs, APs, or Ethernet switches) goes offline and comes back, and you suddenly have to deal with extremely high continuous load.
Exactly.
Regards, Kostas - List info/subscribe/unsubscribe? See
Arran Cudbard-Bell <a.cudbardb@freeradius.org> writes: Hi,
The parsed form is the string we want to extract and use it later against an LDAP repository containing ports to verify the port status and make decisions based on that. We have reached in a regex containing more than 8 groups. Also, as you can see, some DSLAMs send initial zeroes in front of numbers describing DSLAM port/slot.
You can strip those out trivially in a regex...
Say you had /1/01/01/12
Your regex would be
/0?([0-9]+)/0?([0-9]+)/0?([0-9]+)/0?([0-9]+)
Of course you are right and it was really silly I didn't think of that.
For v2.x.x the 0 stripping stuff can be done in a bunch of ways without requiring extra code, so that's definitely not going to be added.
Yes, I am convinced about that.
Alan has final on the number of capture groups and whether they'll be increased for v2.x.x. I suspect they won't be as it's in feature freeze.
Yes indeed, he refused to add it. I our case I managed to overcome the problem by defining a slightly different regex. It is true that the info I 'd like to extract is only 5 variables (fitting in the 8) but more groups can be needed in more compicated expressions. In our case the original regex was /.*_((HUA)|(ALU))_([[:digit:]]+)(.*)? ((atm)|(eth)) ([[:digit:]]+)\/([[:digit:]]+)\/([[:digit:]]+)\/([[:digit:]]+)(:)?.*/ and I reduced the groups with /.*_[HUAALU]{3}_([[:digit:]]+)(.*)? [atmeth]{3} 0?([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)(:)?.*/) Not exactly the same but fits our purposes fine.
It's the sort of thing that would be quite nice to define at build time TBH. Instead of increasing it for everyone, as there is a performance penalty.
I think generally this is the way to go, have it configurable since people can use it. Matching multiple times to reduce the number of groups is at least "not elegant". Maybe you can have it in the v3.x.x branch?
Sure. I'm assuming that's because you don't have the facilities to properly test your RADIUS service, and don't want to introduce changes where you can't fully determine what the impact will be.
We do have such facilities, however: - we have a short timeframe for this project - the project is generally very big (company wide) and involves large changes in various layers - we already operate FreeRADIUS v2.x.x for many years - (rather important) the environment I work is REALLY conservative in changes These are the reasons we will go with 2.x.x for now. We can revisit the subject and upgrade in the future (I also need to catchup with 3.x.x).
Well the LDAP code doesn't suck, you'd probably get a marked performance improvement just from using the new rlm_ldap module.
In general the policy language is more efficient, and there's a greater number of validation checks on startup. It's generally a lot more consistent with v2.x.x.
But i'm not here to sell it to you. Upgrade when you want... or don't.
Thanks, Kostas
Yes indeed, he refused to add it. I our case I managed to overcome the problem by defining a slightly different regex. It is true that the info I 'd like to extract is only 5 variables (fitting in the 8) but more groups can be needed in more compicated expressions. In our case the original regex was
/.*_((HUA)|(ALU))_([[:digit:]]+)(.*)? ((atm)|(eth)) ([[:digit:]]+)\/([[:digit:]]+)\/([[:digit:]]+)\/([[:digit:]]+)(:)?.*/
and I reduced the groups with
/.*__([[:digit:]]+)(.*)? [atmeth]{3} 0?([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)(:)?.*/)
Not exactly the same but fits our purposes fine.
[HUAALU]{3}
That'll work, but it's pretty bad.
((HUA)|(ALU))
Inputting (ALU|HUA) on debuggex, you get: Meaning HUA and ALU count as atoms for the alternation. In fact (HUA|ALU) is exactly the same thing as ((HUA)|(ALU)) but uses two fewer capture groups. I can't remember if it's the same in extended regular expressions but for PCRE at least (?:<expression>) makes expression an atom, but doesn't add the match to the capture groups. 'foo bar baz' =~ /([[:alnum:]]+) (?:[[:alnum:]]+) ([[:alnum:]]+)/ Would result in %{0} -> 'foo bar baz' %{1} -> 'foo' %{2} -> 'baz' So you don't waste the capture groups if you're not actually going to use the data. Try it with extended regular expressions and see if it works. If it doesn't the only way to fix it would be upgrade to v3.0.x and build with PCRE unfortunately.
It's the sort of thing that would be quite nice to define at build time TBH. Instead of increasing it for everyone, as there is a performance penalty.
I think generally this is the way to go, have it configurable since people can use it. Matching multiple times to reduce the number of groups is at least "not elegant". Maybe you can have it in the v3.x.x branch?
Yes, i'll have a look.
Sure. I'm assuming that's because you don't have the facilities to properly test your RADIUS service, and don't want to introduce changes where you can't fully determine what the impact will be.
We do have such facilities, however:
- we have a short timeframe for this project - the project is generally very big (company wide) and involves large changes in various layers - we already operate FreeRADIUS v2.x.x for many years - (rather important) the environment I work is REALLY conservative in changes
Yes, I know what ISPs can be like :)
These are the reasons we will go with 2.x.x for now. We can revisit the subject and upgrade in the future (I also need to catchup with 3.x.x).
Well the LDAP code doesn't suck, you'd probably get a marked performance improvement just from using the new rlm_ldap module.
In general the policy language is more efficient, and there's a greater number of validation checks on startup. It's generally a lot more consistent with v2.x.x.
*than v2.x.x Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Arran Cudbard-Bell <a.cudbardb@freeradius.org> writes:
Yes indeed, he refused to add it. I our case I managed to overcome the problem by defining a slightly different regex. It is true that the info I 'd like to extract is only 5 variables (fitting in the 8) but more groups can be needed in more compicated expressions. In our case the original regex was
/.*_((HUA)|(ALU))_([[:digit:]]+)(.*)? ((atm)|(eth)) ([[:digit:]]+)\/([[:digit:]]+)\/([[:digit:]]+)\/([[:digit:]]+) (:)?.*/
and I reduced the groups with
/.*__([[:digit:]]+)(.*)? [atmeth]{3} 0?([[:digit:]]+)\/0? ([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)(:)?.*/)
Not exactly the same but fits our purposes fine.
[HUAALU]{3}
That'll work, but it's pretty bad.
Yes, I know.
((HUA)|(ALU))
Inputting (ALU|HUA) on debuggex, you get:
*
Meaning HUA and ALU count as atoms for the alternation. In fact (HUA|ALU) is exactly the same thing as ((HUA)|(ALU)) but uses two fewer capture groups.
This is very nice.
I can't remember if it's the same in extended regular expressions but for PCRE at least
(?:<expression>) makes expression an atom, but doesn't add the match to the capture groups.
'foo bar baz' =~ /([[:alnum:]]+) (?:[[:alnum:]]+) ([[:alnum:]]+)/
Would result in
%{0} -> 'foo bar baz' %{1} -> 'foo' %{2} -> 'baz'
So you don't waste the capture groups if you're not actually going to use the data.
Very useful, but unfortunately only available in pcre it seems.
Try it with extended regular expressions and see if it works. If it doesn't the only way to fix it would be upgrade to v3.0.x and build with PCRE unfortunately.
OK.
I think generally this is the way to go, have it configurable since people can use it. Matching multiple times to reduce the number of groups is at least "not elegant". Maybe you can have it in the v3.x.x branch?
Yes, i'll have a look.
Thanks, although in my case the problem is overcome. Latest version of regex for my case: /.*_(HUA|ALU)_([[:digit:]]+)(.*)? (atm|eth) 0?([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)\/0?([[:digit:]]+)(:)?.*/ which gives me the result in exactly 8 groups.
Yes, I know what ISPs can be like :)
Then you are in the group of people who understand ;-) Thank you and keep up the good work. Regards, Kostas
participants (4)
-
Arran Cudbard-Bell -
Herwin Weststrate -
Kostas Zorbadelos -
Peter Lambrechtsen