Quoting =~ in unlang on string expansion

[Alan DeKok]

On Sep 13, 2021, at 9:32 AM, Walter Schober <walter.schober at neotel.at> wrote:
> String expansion is behaving different in case just adding a string or being a result in a regex compare.

 Yes.

> (1)           if (&Called-Station-ID =~ /%{reply:FeatureA}/) {
> (1)           EXPAND %{reply:FeatureA}
> (1)              --> \^\(3\[0-79]\|38\[0-25-9]\|4\[014-9]\|42\[123]\|29\[89]\|43\[1-6]\|43800\|43720)
> (1) ERROR: \^\(3\[0-79]\|38\[0-25-9]\|4\[014-9]\|42\[123]\|29\[89]\|43\[1-6]\|43800\|43720)
> (1) ERROR:                                                                              ^ Pattern compilation failed: unmatched parentheses
> (1)           ERROR: Failed retrieving values required to evaluate condition
> 
> Sure this does not compile because it quoted "(" but not ")". 

  OK.  That should be fixed.  But... v3 doesn't track "trusted" versus "untrusted" data.  So it treats all data as untrusted.

  This means that you can't really create a regular expression from an attribute.  Because the attribute contents will be quoted.  Which is probably not what you want.

  For v4, we're looking on fixing this.  But it will be a bit before v4 is ready.

> In this use case some chars in the reply string are quoted: ^(|[ are quoted while ]) are not for some reasons
> But adding the reply value to some debug AVPair this is done without quoting:

  Yes, because it's not being used in a regular expression.

> I was looking through source code why this is happening, but I am lost. Any hint to source code is appreciated.

  src/main/evaluate.c, function cond_do_regex().  For the list of escaped characters, see regex_escape() in that file

> I'm using version from Debian Buster Backports: 3.0.21+dfsg-1~bpo10+1
> I tried a lot already: "/%{}/", /"%{}"/, %{unescape:...}, ... even if (&Called-Station-ID =~ /&reply:FeatureA/) (which results in FALSE always)
> 
> Why are some character quoted here in this expansion?

  Because we don't want untrusted data from the network being used in a trusted regex.  It's a security issue.

> Where are the "safe charaters" defined for unlang compare operator?

  See regex_escape()

  Alan DeKok.