[master] questions about recent changes in xlat

[Alan DeKok]

On Oct 25, 2020, at 2:59 PM, Chaigneau, Nicolas via Freeradius-Devel <freeradius-devel at lists.freeradius.org> wrote:
> My use case is the following:
> 
> I receive a string buffer from an external source, which can contain any text (encoding doesn't matter).
> Xlat expressions are allowed in this buffer.
> This entails that if the text contains some random "%" characters which would trigger expansion, they must be escaped.
> 
> For example: "this must be xlated: %D, these however must not be modified: \%{  \%D  \x61 \t".

  That's an issue.  You want *some* things to be escaped, but (sort of randomly) other things to *not* be escaped.

  One of the things we've been doing in v4 is to regularize all of the escaping.  We're gradually getting rid of "ad hoc" things like the above, and just moving to:

* strings are 'single quoted' - minimal escaping, no xlat expansions
* strings are "double quoted" - more escaping, and xlat expansions.

  What you're asking for is something else.  The escaping rules will be custom...

  So the best way is for you to just use xlat_tokenize_ephemeral(), which lets you define your own escaping rules.  Because the rules you propose for this case aren't generalizable to other cases.  You can then call xlat_eval_compiled() to get the results.

  i.e. just re-implement xlat_eval(), but with your rules.

  We've been gradually moving all of the "bare xlat expansion" calls to using tmpls, which contain the escaping rules.  This change avoids the issue you're running into, by making the rules simple:  "It's a double quoted string".

  There are still a number of calls to xlat_eval() in the server.  e.g. creating the detail filename.  These calls will have exactly the same issue you're seeing.  So they're broken to some extent.  But since people don't generally put "%" into filenames, it's less of an issue.

  We're working on fixing all that.  But there's only so much time in a day.  :(

  Alan DeKok.