[master] questions about recent changes in xlat

[Arran Cudbard-Bell]

> On 26 Oct 2020, at 12:11, Chaigneau, Nicolas via Freeradius-Devel <freeradius-devel at lists.freeradius.org> wrote:
> 
> Thanks for your detailled explanation.
> I was a bit confused when you and Arran told me escaping "just worked", but of course you couldn't guess what I was trying to do (custom things using some FreeRADIUS functions that are not meant for that).

Yes.

> So I'll just use my own version of xlat_eval as you suggested.
> By the way, the new parsing facilities you've added recently makes this very easy. I like that :)

Thanks :) It’s usually 3:1 reduction in the overall amount of code when we convert things to sbuffs... and yes making your own custom escaping schemes much easier.  

One of the things that’s important to understand is how the rules get applied.  The parse rules passed into the xlat tokenizer only change how the string outside the expansions gets unescaped.  with “foo %{bar: args} baz”, parsing of args is not effected by the rules passed in.  You can think of the string being parsed more like “foo “%{bar: args}” baz”.

This means we can have “%{sql:select * from bar where foo = “bar”}” without escaping the double quotes.  This works because in the grammar applied to the arguments of the sql function “ isn’t a terminal sequence... which is both awesome because you don’t need to escape double quotes, and a bit weird.
 
-Arran
> 
> Keep up the good work!
> 
> 
> Regards,
> Nicolas.
> 
> 
>>  That's an issue.  You want *some* things to be escaped, but (sort of randomly) other things to *not* be escaped.
>> 
>>  One of the things we've been doing in v4 is to regularize all of the escaping.  We're gradually getting rid of "ad hoc" things like the above, and just moving to:
>> 
>> * strings are 'single quoted' - minimal escaping, no xlat expansions
>> * strings are "double quoted" - more escaping, and xlat expansions.
>> 
>>  What you're asking for is something else.  The escaping rules will be custom...
>> 
>>  So the best way is for you to just use xlat_tokenize_ephemeral(), which lets you define your own escaping rules.  Because the rules you propose for this case aren't generalizable to other cases.  You can then call xlat_eval_compiled() to get the results.
>> 
>>  i.e. just re-implement xlat_eval(), but with your rules.
>> 
>>  We've been gradually moving all of the "bare xlat expansion" calls to using tmpls, which contain the escaping rules.  This change avoids the issue you're running into, by making the rules simple:  "It's a double quoted string".
>> 
>>  There are still a number of calls to xlat_eval() in the server.  e.g. creating the detail filename.  These calls will have exactly the same issue you're seeing.  So they're broken to some extent.  But since people don't generally put "%" into filenames, it's less of an issue.
>> 
>>  We're working on fixing all that.  But there's only so much time in a day.  :(
> This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
> 
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html