regex matching can be convinced to be TRUE if you're insistive enough?
Hi, while working on server optimisations for one of our customers, I stumbled upon something funny (2.1.10). At one point, a huge regex needs to be executed. When the input string has a length n < ??, and the regex doesn't match, the result is FALSE - no surprise here. If I make the input string long enough to m > ?? > n, and the regex doesn't match, the result is TRUE. That is not what I'd usually expect; and it has some uneasy implications. Depending on what an incoming user manages to send, it could convince to make the server take a wrong code path; maybe sneaking in when he shouldn't. A simple test case to prove this: in authorize, add update control { Bla = "*1* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*2* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*3* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*4* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*5* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*6* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh" } if ( "%{control:Bla[*]}" =~ /zzzz/ ) { update control { Bla2 := "Hit!" } } This doesn't match, and debug output tells you so: Tue May 24 16:09:56 2011 : Info: ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) Tue May 24 16:09:56 2011 : Info: expand: %{control:Bla[*]} -> *1* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *2* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *3* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *4* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *5* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *6* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh Tue May 24 16:09:56 2011 : Info: ? Evaluating ("%{control:Bla[*]}" =~ /zzzz/) -> FALSE Tue May 24 16:09:56 2011 : Info: ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) -> FALSE Now, make the Bla blob larger: update control { RESTENA-Service-Type = "IMAP", Bla = "*1* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*2* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*3* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*4* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*5* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*6* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*7* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*8* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*9* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh", Bla += "*A* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh" } And you will get a TRUE: Tue May 24 16:11:42 2011 : Info: ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) Tue May 24 16:11:42 2011 : Info: expand: %{control:Bla[*]} -> *1* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *2* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *3* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *4* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *5* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *6* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *7* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *8* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh * Tue May 24 16:11:42 2011 : Info: ? Evaluating ("%{control:Bla[*]}" =~ /zzzz/) -> TRUE Tue May 24 16:11:42 2011 : Info: ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) -> TRUE Note also that the debug output stops its string content output prematurely, but that's not as bad as the string regex mis-matching. My guess is that the breakage occurs when the string is longer than what's printed out in debug mode, but I didn't really try. Greetings, Stefan Winter -- Stefan WINTER Ingenieur de Recherche Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg Tel: +352 424409 1 Fax: +352 422473
Stefan Winter wrote:
while working on server optimisations for one of our customers, I stumbled upon something funny (2.1.10).
At one point, a huge regex needs to be executed. When the input string has a length n < ??, and the regex doesn't match, the result is FALSE - no surprise here.
If I make the input string long enough to m > ?? > n, and the regex doesn't match, the result is TRUE.
There are some issues with regex matching in 2.1.10. 2.1.11 contains fixes. Alan DeKok.
Hi,
There are some issues with regex matching in 2.1.10. 2.1.11 contains fixes.
That is, v2.1.x already contains these fixes? If so, I'll be happy to test if this particular issue is fixed... Stefan
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
-- Stefan WINTER Ingenieur de Recherche Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg Tel: +352 424409 1 Fax: +352 422473
Stefan Winter wrote:
There are some issues with regex matching in 2.1.10. 2.1.11 contains fixes.
That is, v2.1.x already contains these fixes? If so, I'll be happy to test if this particular issue is fixed...
Yes. I think it should be fixed in v2.1.x. Alan DeKok.
Hi,
Yes. I think it should be fixed in v2.1.x.
Doesn't seem to work; I get the exact same behviour: +- entering group authorize {...} ++[control] returns notfound ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) expand: %{control:Bla[*]} -> *1* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *2* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *3* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *4* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *5* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *6* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *7* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *8* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh * ? Evaluating ("%{control:Bla[*]}" =~ /zzzz/) -> TRUE I'm trying to find the tipping point right now; it is *not* at the debug output length delimiter, but somewhere later. Stefan
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
-- Stefan WINTER Ingenieur de Recherche Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg Tel: +352 424409 1 Fax: +352 422473
HI again, the bug isn't triggered exclusively by string length. It also must have to do with number of attributes being globbed together via [*]. To illustrate: Bla = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*100*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*200*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*300*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*400*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*500*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*600*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*700*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*800*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*900*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1000*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1100*", Bla += "A" This triggered a false match of the regex. Bla = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*100*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*200*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*300*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*400*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*500*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*600*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*700*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*800*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*900*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1000*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1100*A" While this didn't. Nor did "AB" in the end of the string, after all the newline is also a character after globbing. Strange. Stefan Am 26.05.2011 09:29, schrieb Stefan Winter:
Hi,
Yes. I think it should be fixed in v2.1.x. Doesn't seem to work; I get the exact same behviour:
+- entering group authorize {...} ++[control] returns notfound ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) expand: %{control:Bla[*]} -> *1* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *2* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *3* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *4* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *5* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *6* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *7* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh *8* lkkgflaksdjfhklsajfdghaösfdlghaskljfdghalksdfjghaklsdfjghkdalsfjghaklsfdghaklsdfjghlaksdfghaklsdfjghsdaklfjghkladsfjgh * ? Evaluating ("%{control:Bla[*]}" =~ /zzzz/) -> TRUE
I'm trying to find the tipping point right now; it is *not* at the debug output length delimiter, but somewhere later.
Stefan
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
-- Stefan WINTER Ingenieur de Recherche Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg Tel: +352 424409 1 Fax: +352 422473
Stefan Winter wrote:
the bug isn't triggered exclusively by string length. It also must have to do with number of attributes being globbed together via [*]. To illustrate:
Fixed && pushed to git. The issue was that [*] wasn't terminating the string with a final '\0' Alan DeKok.
Hi, right, the false positive matches for the regex are now gone, thanks! However, looks like the regex comparison stops after 1023 Bytes, if I actually put a matching pattern beyond that mark, the result keeps being FALSE. Any chance to allow for a longer string? The concrete use case I have is that group membership attributes are pulled via ldap.attrmap, and if a user is in many groups, checking the group membership with [*] and a regex can easily go beyond 1023 Bytes. Greetings, Stefan Winter Am 26.05.2011 10:55, schrieb Alan DeKok:
Stefan Winter wrote:
the bug isn't triggered exclusively by string length. It also must have to do with number of attributes being globbed together via [*]. To illustrate: Fixed && pushed to git.
The issue was that [*] wasn't terminating the string with a final '\0'
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
-- Stefan WINTER Ingenieur de Recherche Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg Tel: +352 424409 1 Fax: +352 422473
Stefan Winter wrote:
right, the false positive matches for the regex are now gone, thanks!
That's good to hear.
However, looks like the regex comparison stops after 1023 Bytes, if I actually put a matching pattern beyond that mark, the result keeps being FALSE.
Any chance to allow for a longer string?
$ git pull It's a simple fix.
The concrete use case I have is that group membership attributes are pulled via ldap.attrmap, and if a user is in many groups, checking the group membership with [*] and a regex can easily go beyond 1023 Bytes.
A better fix would be to be able to do loop over attributes, doing checks for each one. But that's getting into the area of a real programming language... Alan DeKok.
Hm, that buffer expansion doesn't seem to be sufficient: if I place the matching text near the beginning, it matches. If it's near the end, it doesn't. There's a couple of buffer limits on 1024 in xlat.c ; maybe they need to be expanded, too? This one: Bla = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*100*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*200*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*300*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*400*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890zzzz*500*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*600*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*700*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*800*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*900*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1000*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1100*", Bla = "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1200*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1300*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1400*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1500*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1600*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1700*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1800*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1900*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*2000*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*zzzz*" } if ( "%{control:Bla[*]}" =~ /zzzz/ ) { update control { Bla2 := "Hit!" } } works (match in line "500"): ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) expand: %{control:Bla[*]} -> 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*100* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*200* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*300* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*400* 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890zzzz*500* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*600* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*700* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*800* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*900* 012345678901234567890123456789012345678901234567890123456789012345678901234567890123 ? Evaluating ("%{control:Bla[*]}" =~ /zzzz/) -> TRUE ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) -> TRUE But this one doesn't match, even though it matches on the last line: Bla = "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*100*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*200*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*300*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*400*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*500*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*600*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*700*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*800*", Bla += "01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*900*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1000*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1100*", Bla = "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1200*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1300*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1400*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1500*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1600*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1700*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1800*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*1900*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*2000*", Bla += "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123*zzzz*" ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) expand: %{control:Bla[*]} -> 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*100* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*200* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*300* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*400* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*500* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*600* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*700* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*800* 01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234*900* 012345678901234567890123456789012345678901234567890123456789012345678901234567890123 ? Evaluating ("%{control:Bla[*]}" =~ /zzzz/) -> FALSE ++? if ("%{control:Bla[*]}" =~ /zzzz/ ) -> FALSE Am 26.05.2011 14:17, schrieb Alan DeKok:
Stefan Winter wrote:
right, the false positive matches for the regex are now gone, thanks! That's good to hear.
However, looks like the regex comparison stops after 1023 Bytes, if I actually put a matching pattern beyond that mark, the result keeps being FALSE.
Any chance to allow for a longer string? $ git pull
It's a simple fix.
The concrete use case I have is that group membership attributes are pulled via ldap.attrmap, and if a user is in many groups, checking the group membership with [*] and a regex can easily go beyond 1023 Bytes. A better fix would be to be able to do loop over attributes, doing checks for each one. But that's getting into the area of a real programming language...
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
-- Stefan WINTER Ingenieur de Recherche Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg Tel: +352 424409 1 Fax: +352 422473
Stefan Winter wrote:
that buffer expansion doesn't seem to be sufficient: if I place the matching text near the beginning, it matches. If it's near the end, it doesn't. There's a couple of buffer limits on 1024 in xlat.c ; maybe they need to be expanded, too?
Nah. Using the "master' branch, do "git pull", and see "man unlang". Access-Request User-Name = "bob" User-Password = "bob" Filter-Id = "hello" Filter-Id += "bar" authorize { ... foreach Filter-Id { update reply { Reply-Message += "%{Foreach-Variable-0}" } } ... } Access-Accept Reply-Message = "hello" Reply-Message = "bar" That's a much better way to do it. :) That's not going into 2.1.x, though. Alan DeKok.
On Thu, May 26, 2011 at 02:17:35PM +0200, Alan DeKok wrote:
A better fix would be to be able to do loop over attributes, doing checks for each one. But that's getting into the area of a real programming language...
But it would be really useful. Here is a real-life example. I have a database with Cisco-AVPair attributes of the form Cisco-AVPair = "ip:vrf-id=XXX" Cisco-AVPair = "ip:ip-unnumbered=YYY" Cisco-AVPair = "ip:addr-pool=ZZZ" But when replying to certain older devices I need to rewrite them to the obsolete form: Cisco-AVPair = "lcp:interface-config=ip vrf forwarding XXX\nip unnumbered YYY\npeer default ip address pool ZZZ" I do this in unlang today, but it's ugly. update reply { Tmp-String-9 !* "" } if ("%{reply:Cisco-AVPair[*]}" =~ /(^|\n)(ip:vrf-id=([^\n]*))/) { update reply { Tmp-String-9 += "ip vrf forwarding %{3}" Cisco-AVPair -= "%{2}" } } if ("%{reply:Cisco-AVPair[*]}" =~ /(^|\n)(ip:ip-unnumbered=([^\n]*))/) { update reply { Tmp-String-9 += "ip unnumbered %{3}" Cisco-AVPair -= "%{2}" } } if ("%{reply:Cisco-AVPair[*]}" =~ /(^|\n)(ip:addr-pool=([^\n]*))/) { update reply { Tmp-String-9 += "peer default ip address pool %{3}" Cisco-AVPair -= "%{2}" } } if ("%{reply:Tmp-String-9}") { update reply { Cisco-AVPair += "lcp:interface-config=%{reply:Tmp-String-9[*]}" Tmp-String-9 !* "" } } The payoff is avoiding having to link in perl/ruby (whose thread-safety worries me), or use a custom C module for rewriting. Now in the above case, if Foo =~ /pattern/ iterated over the values of Foo and stopped on the first match, that would be good enough for me. But actually that's not very general. Users might require: Foo =~ /pattern/ # first match only Foo =~ /pattern/ # process every match Foo =~ /pattern/ # only true if all values match Foo !~ /pattern/ # first one which doesn't match Foo !~ /pattern/ # process all which don't match Foo !~ /pattern/ # only true if no values match A simple approach might be: if (any Foo =~ /pattern/) { ... } if (all Foo =~ /pattern/) { ... } (but it's not clear how that interacts with string expansions, especially if an expression refers to multiple attributes) Else you have an explicit loop: e.g. foreach Foo { if (Foo =~ /pattern/) { ... last Foo # early termination (if required) } } Within the foreach Foo construct, Foo would expand to the n'th instance instead of the 0'th instance. This would work with string expansions too, but you'd still need to be able to specify the list to iterate on, e.g. foreach reply:Foo { ... } Aside: I would also like to see direct support for accessing control and reply lists without string expansions. Then you could write if (reply:Service-Type == Framed-User) (I don't think that's possible today). It would also make the unwieldy 'update' block obsolete: if (reply:Service-Type == Framed-User) { reply:Framed-Protocol = PPP } radius 3.x perhaps? :-) Regards, Brian. P.S. One other thing for multi-valued attributes: Foo[*] always uses \n as separator and adds \n to the end. It would be useful to be able to join with another character, e.g. Foo[*,] - and be able to suppress the trailing one.
Brian Candler wrote:
Else you have an explicit loop: e.g.
foreach Foo {
$ git pull origin master:master ... build $ man unlang
last Foo # early termination (if required)
Uh.. yeah. That will be supported eventually. Right now, not so much.
Aside: I would also like to see direct support for accessing control and reply lists without string expansions. Then you could write
if (reply:Service-Type == Framed-User)
(I don't think that's possible today). It would also make the unwieldy 'update' block obsolete:
if (reply:Service-Type == Framed-User) { reply:Framed-Protocol = PPP }
That's useful, but potentially awkward in the current code. I'll take a look as a low priority.
P.S. One other thing for multi-valued attributes: Foo[*] always uses \n as separator and adds \n to the end. It would be useful to be able to join with another character, e.g. Foo[*,] - and be able to suppress the trailing one.
Sure... send a patch for 3.0. Alan DeKok.
On Fri, May 27, 2011 at 01:10:59PM +0200, Alan DeKok wrote:
$ git pull origin master:master
... build
$ man unlang
Very neat, thank you. This sort of iteration for vendor attributes like Cisco-AVPair is extremely helpful. The key operations will be things like: 1. replace the (current iteration) attribute 2. delete the (current iteration) attribute You can do these already, more or less, using foreach reply:Foo { update reply { Foo -= "%{Foreach-Variable-0}" Foo += "replacement value" } } That is, it will be fine unless you have multiple attributes with identical values. Cheers, Brian.
Brian Candler wrote:
Very neat, thank you. This sort of iteration for vendor attributes like Cisco-AVPair is extremely helpful.
The key operations will be things like:
1. replace the (current iteration) attribute
Uh... right.
2. delete the (current iteration) attribute
<boom>
You can do these already, more or less, using
foreach reply:Foo { update reply { Foo -= "%{Foreach-Variable-0}" Foo += "replacement value" } }
That is, it will be fine unless you have multiple attributes with identical values.
Yes. Remember, "unlang" is not a language. I am very wary of making it a language. Alan DeKok.
participants (3)
-
Alan DeKok -
Brian Candler -
Stefan Winter