3.0.4: binary LDAP attributes
Hi everyone, Our (Red Hat) QA was testing the effect of this entry in 3.0.4 ChangeLog: * Modify pairparsevalue to deal with embedded NULLs better, and use the binary versions of attribute values in rlm_ldap. They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for radiusReplyMessage:: cmVwbHkgd2l0aCBhAGI= The response output by radtest was Reply-Message = 'reply with a' The network capture also showed that RADIUS reply packets contained truncated values. Is this intended, or was there a fix for this? In related discoveries, it seems that backslashes get removed from LDAP attribute values in RADIUS replies, so radiusReplyMessage: reply with a\0b becomes Reply-Message = 'reply with a0b' in radtest output. Is this intended? Thank you. Nick
On Dec 9, 2014, at 6:51 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
Arran and I have spent the last two weeks fixing those issues. The server *never* dealt well with embedded zeros in “string” data. Octets, yes. Strings, no.
In related discoveries, it seems that backslashes get removed from LDAP attribute values in RADIUS replies, so
That’s part of what we were fixing.
Is this intended?
No. Please try the v3.0.x git branch. We’ve fixed these issues, and added test cases for them all. The normal user won’t see any change. People who have embedded zeros in strings will see that they now work. Alan DeKok.
Hi Alan, On 12/09/2014 03:02 PM, Alan DeKok wrote:
On Dec 9, 2014, at 6:51 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
Arran and I have spent the last two weeks fixing those issues. The server *never* dealt well with embedded zeros in “string” data. Octets, yes. Strings, no.
We already have an integration test for strings with embedded zeros. We would like to add a test for zeros in "binary" attributes. I'm not sure exactly what you mean by octets here. Is it attributes with "octets" type in dictionaries? If so, are LDAP attributes supposed to contain hex strings for them, and it is basically "00" bytes which were the problem? Or could there be a direct binary representation for "octets"? Is the "abinary" type affected? Could you perhaps suggest attribute names/types and LDAP attribute values to test for? Sorry, if this is very basic knowledge I could have extracted myself. Thank you. Nick
On Jan 7, 2015, at 8:07 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
We already have an integration test for strings with embedded zeros. We would like to add a test for zeros in "binary" attributes.
That’s good.
I'm not sure exactly what you mean by octets here. Is it attributes with "octets" type in dictionaries?
Yes.
If so, are LDAP attributes supposed to contain hex strings for them, and it is basically "00" bytes which were the problem?
The changes in 3.0.6 were so FreeRADIUS could read *binary* data from LDAP. Not hex strings.
Is the "abinary" type affected?
No. The “abinary” type should *always* be stored as a printable string. There are no provisions in the server for reading or writing it as a binary blob.
Could you perhaps suggest attribute names/types and LDAP attribute values to test for?
There’s an LDAP schema which maps RADIUS attributes to LDAP. Try that, and just use RADIUS attributes which are type “octets”. Alan DeKok.
On 01/07/2015 04:19 PM, Alan DeKok wrote:
On Jan 7, 2015, at 8:07 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
We already have an integration test for strings with embedded zeros. We would like to add a test for zeros in "binary" attributes.
That’s good.
I'm not sure exactly what you mean by octets here. Is it attributes with "octets" type in dictionaries?
Yes.
If so, are LDAP attributes supposed to contain hex strings for them, and it is basically "00" bytes which were the problem?
The changes in 3.0.6 were so FreeRADIUS could read *binary* data from LDAP. Not hex strings.
Is the "abinary" type affected?
No. The “abinary” type should *always* be stored as a printable string. There are no provisions in the server for reading or writing it as a binary blob.
Could you perhaps suggest attribute names/types and LDAP attribute values to test for?
There’s an LDAP schema which maps RADIUS attributes to LDAP. Try that, and just use RADIUS attributes which are type “octets”.
Thanks Alan! Nick
On 7 Jan 2015, at 20:07, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
Hi Alan,
On 12/09/2014 03:02 PM, Alan DeKok wrote:
On Dec 9, 2014, at 6:51 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
Arran and I have spent the last two weeks fixing those issues. The server *never* dealt well with embedded zeros in “string” data. Octets, yes. Strings, no.
We already have an integration test for strings with embedded zeros. We would like to add a test for zeros in "binary" attributes.
I'm not sure exactly what you mean by octets here. Is it attributes with "octets" type in dictionaries? If so, are LDAP attributes supposed to contain hex strings for them, and it is basically "00" bytes which were the problem? Or could there be a direct binary representation for "octets"?
IIRC there's still issues with embedded zeroes in string attributes, because they were going via pairparse value. I don't know if Alan fixed this, if he didn't, i'll try and get it sorted for 3.0.8. To test, insert binary data (with embedded zeroes) into any string attribute in LDAP then map the string attribute to an octets type attribute in the server. You should see that the entire attribute value is copied, embedded zeroes and all. Previously the copy would have stopped at the first embedded zero.
Is the "abinary" type affected?
abinary is ascend binary filters. It's a way of packing filtering rules into a binary blob. abinary is expected to be in its presentation format (text) when entering the server from any route other than the RADIUS decoder, so no.
Could you perhaps suggest attribute names/types and LDAP attribute values to test for?
See above. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS development team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
On 01/15/2015 04:45 AM, Arran Cudbard-Bell wrote:
On 7 Jan 2015, at 20:07, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
Hi Alan,
On 12/09/2014 03:02 PM, Alan DeKok wrote:
On Dec 9, 2014, at 6:51 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
Arran and I have spent the last two weeks fixing those issues. The server *never* dealt well with embedded zeros in “string” data. Octets, yes. Strings, no.
We already have an integration test for strings with embedded zeros. We would like to add a test for zeros in "binary" attributes.
I'm not sure exactly what you mean by octets here. Is it attributes with "octets" type in dictionaries? If so, are LDAP attributes supposed to contain hex strings for them, and it is basically "00" bytes which were the problem? Or could there be a direct binary representation for "octets"?
IIRC there's still issues with embedded zeroes in string attributes, because they were going via pairparse value. I don't know if Alan fixed this, if he didn't, i'll try and get it sorted for 3.0.8.
To test, insert binary data (with embedded zeroes) into any string attribute in LDAP then map the string attribute to an octets type attribute in the server.
You should see that the entire attribute value is copied, embedded zeroes and all.
Previously the copy would have stopped at the first embedded zero.
Is the "abinary" type affected?
abinary is ascend binary filters. It's a way of packing filtering rules into a binary blob. abinary is expected to be in its presentation format (text) when entering the server from any route other than the RADIUS decoder, so no.
Could you perhaps suggest attribute names/types and LDAP attribute values to test for?
See above.
Thanks, Arran! We've postponed the binary value testing for now, but I think we'll get around to it. I'll forward this to our QE guy. Nick
On 12/09/2014 01:51 PM, Nikolai Kondrashov wrote:
Our (Red Hat) QA was testing the effect of this entry in 3.0.4 ChangeLog:
* Modify pairparsevalue to deal with embedded NULLs better, and use the binary versions of attribute values in rlm_ldap.
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
radiusReplyMessage:: cmVwbHkgd2l0aCBhAGI=
The response output by radtest was
Reply-Message = 'reply with a'
The network capture also showed that RADIUS reply packets contained truncated values.
We still see the above behavior in v3.0.13. Please excuse me, if you explained it before, but is this intended, or is this a bug? Thank you. Nick
On 23/03/17 09:23, Nikolai Kondrashov wrote:
On 12/09/2014 01:51 PM, Nikolai Kondrashov wrote:
Our (Red Hat) QA was testing the effect of this entry in 3.0.4 ChangeLog:
* Modify pairparsevalue to deal with embedded NULLs better, and use the binary versions of attribute values in rlm_ldap.
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
radiusReplyMessage:: cmVwbHkgd2l0aCBhAGI=
The response output by radtest was
Reply-Message = 'reply with a'
The network capture also showed that RADIUS reply packets contained truncated values.
We still see the above behavior in v3.0.13.
This looks relevant: https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/src/modules/rlm_... https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/src/lib/pair.c#L... https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/src/lib/value.c#... These appear to all be 8-bit clean, using talloc_bstrndup and the quote handling seems OK. At a glance it looks fine to me, I wonder if the problem is deeper in the server core. What happens if you butcher the dictionaries and change Reply-Message to be "octets", just to test?
On Mar 23, 2017, at 5:23 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
On 12/09/2014 01:51 PM, Nikolai Kondrashov wrote:
Our (Red Hat) QA was testing the effect of this entry in 3.0.4 ChangeLog:
* Modify pairparsevalue to deal with embedded NULLs better, and use the binary versions of attribute values in rlm_ldap.
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
radiusReplyMessage:: cmVwbHkgd2l0aCBhAGI=
The response output by radtest was
Reply-Message = 'reply with a'
The network capture also showed that RADIUS reply packets contained truncated values.
We still see the above behavior in v3.0.13.
Please excuse me, if you explained it before, but is this intended, or is this a bug?
The underlying issue is how strings are dealt with. The various LDAP APIs and rlm_ldap take char* and length pointers, so that works. The underlying issue is in src/main/map.c, map_afrom_attr_str(). It takes a char* pointer which contains operator and value. e.g. ":= bob". That gets parsed into separate fields, but the length of the underlying string is not used. I'm not sure why that's done for rlm_ldap, as the operators are already in the "map" config of raddbs/mods-enabled/ldap. So the short answer is that the values in LDAP are *printable* strings, not *raw* strings. This likely goes back to the origin of the server. i.e. if you want embedded NULs, double quotes, etc... you've got to escape them. And on top of that, Reply-Message is usually a UTF-8 string. So putting embedded zeros into a printable string will confuse a lot more products than just FreeRADIUS. i.e. don't do it. Alan DeKok.
On 03/23/2017 02:59 PM, Alan DeKok wrote:
On Mar 23, 2017, at 5:23 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
On 12/09/2014 01:51 PM, Nikolai Kondrashov wrote:
Our (Red Hat) QA was testing the effect of this entry in 3.0.4 ChangeLog:
* Modify pairparsevalue to deal with embedded NULLs better, and use the binary versions of attribute values in rlm_ldap.
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
radiusReplyMessage:: cmVwbHkgd2l0aCBhAGI=
The response output by radtest was
Reply-Message = 'reply with a'
The network capture also showed that RADIUS reply packets contained truncated values.
We still see the above behavior in v3.0.13.
Please excuse me, if you explained it before, but is this intended, or is this a bug?
The underlying issue is how strings are dealt with. The various LDAP APIs and rlm_ldap take char* and length pointers, so that works.
The underlying issue is in src/main/map.c, map_afrom_attr_str(). It takes a char* pointer which contains operator and value. e.g. ":= bob". That gets parsed into separate fields, but the length of the underlying string is not used.
I'm not sure why that's done for rlm_ldap, as the operators are already in the "map" config of raddbs/mods-enabled/ldap.
So the short answer is that the values in LDAP are *printable* strings, not *raw* strings. This likely goes back to the origin of the server.
i.e. if you want embedded NULs, double quotes, etc... you've got to escape them.
And on top of that, Reply-Message is usually a UTF-8 string. So putting embedded zeros into a printable string will confuse a lot more products than just FreeRADIUS.
i.e. don't do it.
Well, zero is a valid UTF-8 character, but I think I understand what you're saying about attribute values being "printable" strings. I assume that means they are expected to contain escapes instead. Thanks, everyone! Nick
On Mar 24, 2017, at 5:13 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
On 03/23/2017 02:59 PM, Alan DeKok wrote:
On Mar 23, 2017, at 5:23 AM, Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com> wrote:
On 12/09/2014 01:51 PM, Nikolai Kondrashov wrote:
Our (Red Hat) QA was testing the effect of this entry in 3.0.4 ChangeLog:
* Modify pairparsevalue to deal with embedded NULLs better, and use the binary versions of attribute values in rlm_ldap.
They have noticed that binary LDAP values get truncated on embedded zero characters (\0) in RADIUS replies, in radiusReplyMessage in particular. I.e. for
radiusReplyMessage:: cmVwbHkgd2l0aCBhAGI=
The response output by radtest was
Reply-Message = 'reply with a'
The network capture also showed that RADIUS reply packets contained truncated values.
We still see the above behavior in v3.0.13.
Please excuse me, if you explained it before, but is this intended, or is this a bug?
The underlying issue is how strings are dealt with. The various LDAP APIs and rlm_ldap take char* and length pointers, so that works.
The underlying issue is in src/main/map.c, map_afrom_attr_str(). It takes a char* pointer which contains operator and value. e.g. ":= bob". That gets parsed into separate fields, but the length of the underlying string is not used.
I'm not sure why that's done for rlm_ldap, as the operators are already in the "map" config of raddbs/mods-enabled/ldap.
So the short answer is that the values in LDAP are *printable* strings, not *raw* strings. This likely goes back to the origin of the server.
i.e. if you want embedded NULs, double quotes, etc... you've got to escape them.
And on top of that, Reply-Message is usually a UTF-8 string. So putting embedded zeros into a printable string will confuse a lot more products than just FreeRADIUS.
i.e. don't do it.
Well, zero is a valid UTF-8 character, but I think I understand what you're saying about attribute values being "printable" strings. I assume that means they are expected to contain escapes instead.
It's an ongoing battle. In v4 more things should be binary safe. We've even overloaded the fmt specifiers for our logging functions so that they can print binary strings, and auto-escape them. Using talloc has helped with this a lot, as talloc chunk headers contain the length of the buffer. -Arran
Some minor changes, as I was reading the code wrong.... rlm_ldap calls fr_pair_value_from_str(), which does take a length from LDAP. But the code it calls still treats the input string as a printed string, and tries to parse it. Which means double quotes, zeros, etc. still need to be escaped. Alan DeKok.
participants (4)
-
Alan DeKok -
Arran Cudbard-Bell -
Nikolai Kondrashov -
Phil Mayers