Nulls embedded in Cisco-AVPairs
Hi folks, I'm trying to do a regular expression match on a Cisco-AVPair value that is sent by the NAD in an Accounting-Request message. He’s a sample of the configuration using the suggested “foreach” method. I’ve added it to the accounting{} section of sites-available/default. foreach &Cisco-AVPair { if (Foreach-Variable-0 =~ /raspberrypi/) { update { &Url-Name := "getaclname" &Url-DataType := "MAC_ADDR" &Url-Data := "%{Foreach-Variable-0}" } rest } However, some of the Cisco-AVPairs we’re wanting to parse contain zeros in the value, such as Cisco-AVPair = "lldp-tlv=\000\005\000\013raspberrypi" Note that the print function above is translating the bytes to an octal format, so \000 is really 0x00 in memory. The zeros in the attribute interfere with the regular expression processing.The line proceeded with a hash prints the entire value representing non-printable octets with octal representations (as above), but the regular expression processing stops at the first zero (e.g., following 'lldp-tlv=’ in the example string), obviously interpreting the zero as the string termination null. This is shown in this log. Note that following EXPAND X that the reported string has been truncated. foreach &Cisco-AVPair { # Foreach-Variable-0 = "lldp-tlv=\000\005\000\013raspberrypi" { EXPAND %{Foreach-Variable-0} %{Foreach-Variable-0} Parsed xlat tree: virtual --> Foreach-Variable-0 EXPAND X Foreach-Variable-0 --> lldp-tlv= --> lldp-tlv= ... } Looking in the code, I see that both the full and trunctated log entries are the output of fr_pair_value_snprint(). The call that display the full value specifies ‘“‘ as a quote value, which results in it representing the non-printable characters as octal values. But the call that creates a string for regular expression processing (the EXPAND X string) specifies no quote value, which results in a string truncated at the first zero byte. Note that not only is the value shorted in the log above, but the regular expression processing uses this string as well. This effectively doesn’t allow regular expression processing matching strings following the zero in the value. This truncation seems to be because some regular expression libraries do not safely process nulls. These are messages I received when changing the quote variable to ‘“' for debugging purposes: ERROR : (57) regex failed: Found null in subject at offset 9. String unsafe for evaluation ERROR : (57) Condition evaluation failed because the value of an operand could not be determined I’ve tried many different string representations in the code snippet above and nothing changes this behavior. But I also can’t find a way to transform Foreach-Variable-0 such that it can be given to another module for processing … always the fr_pair_value_snprint() call with a zero quote value in is used, which truncates the string. This is the call in in xlat_foreach(). There is a FreeRADIUS compile option to use regex code where nulls are safe within the value (i.e., regnexec), which would seem to avoid the message above. But that does not seem sufficient as the current code in xlat_foreach() would still provide a truncated string to the regex library. Suggestions for a workaround? I’m using FreeRADIUS 3.1.0. Thanks, Brian -- Brian Weis Security, CSG, Cisco Systems Telephone: +1 408 526 4796 Email: bew@cisco.com
This truncation seems to be because some regular expression libraries do not safely process nulls. These are messages I received when changing the quote variable to ‘“' for debugging purposes:
ERROR : (57) regex failed: Found null in subject at offset 9. String unsafe for evaluation ERROR : (57) Condition evaluation failed because the value of an operand could not be determined
It works :D
I’ve tried many different string representations in the code snippet above and nothing changes this behavior. But I also can’t find a way to transform Foreach-Variable-0 such that it can be given to another module for processing … always the fr_pair_value_snprint() call with a zero quote value in is used, which truncates the string. This is the call in in xlat_foreach().
There is a FreeRADIUS compile option to use regex code where nulls are safe within the value (i.e., regnexec), which would seem to avoid the message above. But that does not seem sufficient as the current code in xlat_foreach() would still provide a truncated string to the regex library.
If you compile with libpcre the error message will go away too.
Suggestions for a workaround?
Unfortunately the xlat processing code is one of the few places which is still not binary safe. It's very close to the top of the todo list for v4.0.x You could pass the request into rlm_perl or rlm_python and do Cisco-AVPair parsing there until the xlat code is fixed.
I’m using FreeRADIUS 3.1.0.
Hm, that development branch is dead at this point. If you need something bleeding edge you should switch to v4.0.x, but that's not at all in a stable state currently either. -Arran
On Jun 10, 2017, at 3:09 PM, Brian Weis (bew) <bew@cisco.com> wrote:
However, some of the Cisco-AVPairs we’re wanting to parse contain zeros in the value, such as
Cisco-AVPair = "lldp-tlv=\000\005\000\013raspberrypi"
While embedded zeros are allowed in string attributes, they do cause problems. The best solution here would be to decode the lldp-tlv into LLDP attributes. But it looks like (from a cursory google search) that "lldp-tlv=..." can contain pretty much anything, and not just binary LLDP TLVs.
Looking in the code, I see that both the full and trunctated log entries are the output of fr_pair_value_snprint(). The call that display the full value specifies ‘“‘ as a quote value, which results in it representing the non-printable characters as octal values. But the call that creates a string for regular expression processing (the EXPAND X string) specifies no quote value, which results in a string truncated at the first zero byte. Note that not only is the value shorted in the log above, but the regular expression processing uses this string as well. This effectively doesn’t allow regular expression processing matching strings following the zero in the value.
Yeah. The string expansion works on C strings. The whole API is missing a "length" parameter. Fixing that is non-trivial.
This truncation seems to be because some regular expression libraries do not safely process nulls. These are messages I received when changing the quote variable to ‘“' for debugging purposes:
ERROR : (57) regex failed: Found null in subject at offset 9. String unsafe for evaluation
At least the server figured out what's wrong and told you, instead of allowing a bad regex library to crash the system. Yes, that's why the checks are there. :(
ERROR : (57) Condition evaluation failed because the value of an operand could not be determined
I’ve tried many different string representations in the code snippet above and nothing changes this behavior. But I also can’t find a way to transform Foreach-Variable-0 such that it can be given to another module for processing … always the fr_pair_value_snprint() call with a zero quote value in is used, which truncates the string. This is the call in in xlat_foreach().
I think there should be a way to fix that...the main thing would be to treat the Foreach-Variable-... attribute as the correct type, instead of printing it.
There is a FreeRADIUS compile option to use regex code where nulls are safe within the value (i.e., regnexec), which would seem to avoid the message above. But that does not seem sufficient as the current code in xlat_foreach() would still provide a truncated string to the regex library.
Yeah...
Suggestions for a workaround?
I’m using FreeRADIUS 3.1.0.
Use the v4.0.x branch. It is (or should be) stable on a day to day basis. The test framework ensures that every individual piece of the server works. But the internal API is being massively re-architected. In order to fix this, and many other little issues. The short way in 3.1.0 to fix this is to just write a C module to look for this attribute, and do the right thing. We're looking at adding native Lua support into v4, which should make all of this much, much, easier. Alan DeKok.
Hi Aaron and Alan, Many thanks for your help. The hint to use a perl module was very useful, and gets me going for now. I’ll look into your other suggestions later (including upgrading to the v4.0.x branch). Thanks, Brian
On Jun 11, 2017, at 1:11 PM, Alan DeKok <aland@deployingradius.com> wrote:
On Jun 10, 2017, at 3:09 PM, Brian Weis (bew) <bew@cisco.com> wrote:
However, some of the Cisco-AVPairs we’re wanting to parse contain zeros in the value, such as
Cisco-AVPair = "lldp-tlv=\000\005\000\013raspberrypi"
While embedded zeros are allowed in string attributes, they do cause problems.
The best solution here would be to decode the lldp-tlv into LLDP attributes. But it looks like (from a cursory google search) that "lldp-tlv=..." can contain pretty much anything, and not just binary LLDP TLVs.
Looking in the code, I see that both the full and trunctated log entries are the output of fr_pair_value_snprint(). The call that display the full value specifies ‘“‘ as a quote value, which results in it representing the non-printable characters as octal values. But the call that creates a string for regular expression processing (the EXPAND X string) specifies no quote value, which results in a string truncated at the first zero byte. Note that not only is the value shorted in the log above, but the regular expression processing uses this string as well. This effectively doesn’t allow regular expression processing matching strings following the zero in the value.
Yeah. The string expansion works on C strings. The whole API is missing a "length" parameter. Fixing that is non-trivial.
This truncation seems to be because some regular expression libraries do not safely process nulls. These are messages I received when changing the quote variable to ‘“' for debugging purposes:
ERROR : (57) regex failed: Found null in subject at offset 9. String unsafe for evaluation
At least the server figured out what's wrong and told you, instead of allowing a bad regex library to crash the system.
Yes, that's why the checks are there. :(
ERROR : (57) Condition evaluation failed because the value of an operand could not be determined
I’ve tried many different string representations in the code snippet above and nothing changes this behavior. But I also can’t find a way to transform Foreach-Variable-0 such that it can be given to another module for processing … always the fr_pair_value_snprint() call with a zero quote value in is used, which truncates the string. This is the call in in xlat_foreach().
I think there should be a way to fix that...the main thing would be to treat the Foreach-Variable-... attribute as the correct type, instead of printing it.
There is a FreeRADIUS compile option to use regex code where nulls are safe within the value (i.e., regnexec), which would seem to avoid the message above. But that does not seem sufficient as the current code in xlat_foreach() would still provide a truncated string to the regex library.
Yeah...
Suggestions for a workaround?
I’m using FreeRADIUS 3.1.0.
Use the v4.0.x branch. It is (or should be) stable on a day to day basis. The test framework ensures that every individual piece of the server works.
But the internal API is being massively re-architected. In order to fix this, and many other little issues.
The short way in 3.1.0 to fix this is to just write a C module to look for this attribute, and do the right thing.
We're looking at adding native Lua support into v4, which should make all of this much, much, easier.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- Brian Weis Security, CSG, Cisco Systems Telephone: +1 408 526 4796 Email: bew@cisco.com
participants (3)
-
Alan DeKok -
Arran Cudbard-Bell -
Brian Weis (bew)