LDAP group query optimisation
I am testing out freeradius with FreeIPA (= 389 directory server). This is freeradius-3.0.11 from Ubuntu 16.04, talking to FreeIPA under CentOS 7. The 389 directory server in FreeIPA has a "memberOf" plugin installed (by default), which exposes all the groups as part of the user record. For example: # bcandler, users, accounts, ipa.example.com dn: uid=bcandler,cn=users,cn=accounts,dc=ipa,dc=example,dc=com objectclass: ipaobject objectclass: person objectclass: top objectclass: ipasshuser objectclass: inetorgperson objectclass: organizationalperson objectclass: krbticketpolicyaux objectclass: krbprincipalaux objectclass: inetuser objectclass: posixaccount objectclass: ipaSshGroupOfPubKeys objectclass: mepOriginEntry objectclass: ipantuserattrs cn: Brian Candler memberOf: cn=ipausers,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com memberOf: cn=server_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com memberOf: cn=network_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com memberOf: cn=vpn,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com memberOf: cn=staff,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com And I have freeradius's ldap module configured for group membership like this: group { membership_attribute = 'memberOf' name_attributes = 'cn' } The problem is, whenever I touch the LDAP-Group attribute it triggers off a whole load of LDAP queries, one for each group, to translate the group DN to the cn. However, since all I'm asking for the cn, and the cn is the RDN of the group, the cn could be extracted directly from the DN. Here's an example of what I see: (1) if (&Called-Station-Id =~ /:Staff$/ && &LDAP-Group[*] == "staff") { (1) Searching for user in group "staff" rlm_ldap (ldap): Reserved connection (3) (1) Using user DN from request "uid=bcandler,cn=users,cn=accounts,dc=ipa,dc=example,dc=com" (1) Checking user object's memberOf attributes (1) Performing unfiltered search in "uid=bcandler,cn=users,cn=accounts,dc=ipa,dc=example,dc=com", scope "base" (1) Waiting for search result... (1) Processing memberOf value "cn=ipausers,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" as a DN (1) Resolving group DN "cn=ipausers,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" to group name (1) Performing unfiltered search in "cn=ipausers,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com", scope "base" (1) Waiting for search result... (1) Group DN "cn=ipausers,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" resolves to name "ipausers" (1) Processing memberOf value "cn=server_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" as a DN (1) Resolving group DN "cn=server_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" to group name (1) Performing unfiltered search in "cn=server_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com", scope "base" (1) Waiting for search result... (1) Group DN "cn=server_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" resolves to name "server_guru" (1) Processing memberOf value "cn=network_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" as a DN (1) Resolving group DN "cn=network_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" to group name (1) Performing unfiltered search in "cn=network_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com", scope "base" (1) Waiting for search result... (1) Group DN "cn=network_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" resolves to name "network_guru" (1) Processing memberOf value "cn=vpn,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" as a DN (1) Resolving group DN "cn=vpn,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" to group name (1) Performing unfiltered search in "cn=vpn,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com", scope "base" (1) Waiting for search result... (1) Group DN "cn=vpn,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" resolves to name "vpn" (1) Processing memberOf value "cn=staff,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" as a DN (1) Resolving group DN "cn=staff,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" to group name (1) Performing unfiltered search in "cn=staff,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com", scope "base" (1) Waiting for search result... (1) Group DN "cn=staff,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com" resolves to name "staff" (1) User found in group "staff". Comparison between membership: name (resolved from DN "cn=staff,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com"), check: name I guess this is intentional: one group object *could* have multiple cn attributes, so maybe it's querying the group to be sure. That is, memberOf: cn=ipausers,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com in theory could translate to LDAP-Group += ipausers LDAP-Group += another name for ipausers However in my case I don't need this. Is there a way I can configure the LDAP module not to do this? Alternatively I could make an explicit group membership query (i.e. return all group entries that this user is a member of); but that still involves two queries, and I would then not be making use of the memberOf feature. Thanks, Brian.
On Oct 13, 2016, at 9:38 AM, Brian Candler <b.candler@pobox.com> wrote:
I am testing out freeradius with FreeIPA (= 389 directory server). This is freeradius-3.0.11 from Ubuntu 16.04, talking to FreeIPA under CentOS 7.
The 389 directory server in FreeIPA has a "memberOf" plugin installed (by default), which exposes all the groups as part of the user record. For example:
...
The problem is, whenever I touch the LDAP-Group attribute it triggers off a whole load of LDAP queries, one for each group, to translate the group DN to the cn.
Try the v3.0.x branch. It's largely v3, with a number of changes. One major one is LDAP group caching. The LDAP module grabs all of the groups once, and then caches them along with the request. Subsequent LDAP group comparisons are done internally, and don't touch LDAP. Alan DeKok.
On Oct 13, 2016, at 9:42 AM, Alan DeKok <aland@deployingradius.com> wrote:
On Oct 13, 2016, at 9:38 AM, Brian Candler <b.candler@pobox.com> wrote:
I am testing out freeradius with FreeIPA (= 389 directory server). This is freeradius-3.0.11 from Ubuntu 16.04, talking to FreeIPA under CentOS 7.
The 389 directory server in FreeIPA has a "memberOf" plugin installed (by default), which exposes all the groups as part of the user record. For example:
...
The problem is, whenever I touch the LDAP-Group attribute it triggers off a whole load of LDAP queries, one for each group, to translate the group DN to the cn.
Specify the group as a DN and it won't do the translation. If you want to examine the DN values yourself using a foreach loop, toggle on cacheable_dn, all the membership DNs will then be available as LDAP-Group instances. I can see some advantages do extracting groups from the RDN, so feel free to add an issue ticket. -Arran
No. I think he did mean 3.0.x - ie the latest version of 3.0.x via GIT ... that'll be 3.0.13 when released, its currently 3.0.12 with some extra bugs/issues fixed (read release notes and/or git submission logs ) alan
On 14/10/2016 18:45, Alan Buxey wrote:
No. I think he did mean 3.0.x - ie the latest version of 3.0.x via GIT ... that'll be 3.0.13 when released, its currently 3.0.12 with some extra bugs/issues fixed (read release notes and/or git submission logs )
I did look at "git diff release_3_0_11 head" before posting. I saw nothing in doc/Changelog relating to LDAP between 3.0.11 and 3.0.13(pre), except * Don't complain on /dev/urandom in ldap And I saw some small tweaks to src/modules/rlm_ldap/rlm_ldap.c, but didn't note anything which would change the fundamental logic. B.
Hi,
On 13/10/2016 14:42, Alan DeKok wrote:
Try the v3.0.x branch. It's largely v3
I don't get that. Did you mean "Try the v4.0.x branch" ?
maybe he did mean 4.0.x - which is interesting as previously told to stay clear of it on production systems especially as its about to go all config change crazy. however, if there is now some change to that policy and 4.0.x is to be very much 3.x with tweaks - with the new crazy stuff being 4.1.x or such then please let us know Alan :/ alan
On Oct 16, 2016, at 9:15 AM, A.L.M.Buxey@lboro.ac.uk wrote:
Hi,
On 13/10/2016 14:42, Alan DeKok wrote:
Try the v3.0.x branch. It's largely v3
I don't get that. Did you mean "Try the v4.0.x branch" ?
maybe he did mean 4.0.x - which is interesting as previously told to stay clear of it on production systems especially as its about to go all config change crazy.
I meant v3.1.x, sorry.
however, if there is now some change to that policy and 4.0.x is to be very much 3.x with tweaks - with the new crazy stuff being 4.1.x or such then please let us know Alan :/
I'm going to be dropping much of 4.0 on the floor in the next week. For a while, it will likely be only a UDP server. No TLS or TCP. Then once the re-architecture is fixed, TCP and TLS become much easier. Alan DeKok.
Specify the group as a DN and it won't do the translation.
Thanks, that works. I guess there is some implicit logic which checks to see if the string tastes like a DN or not. Ah yes: rlm_ldap_is_dn().
If you want to examine the DN values yourself using a foreach loop, toggle on cacheable_dn, all the membership DNs will then be available as LDAP-Group instances.
OK, that works too. It materializes LDAP-Group and LDAP-Group[*], which can then also be used in string expansions. I didn't investigate this flag before, because it said it was for use with rlm_cache, which I'm not using. Some documentation on the magical behaviour of the LDAP-Group attribute would be nice to have :-)
I can see some advantages do extracting groups from the RDN, so feel free to add an issue ticket.
Done: https://github.com/FreeRADIUS/freeradius-server/issues/1788 Cheers, Brian.
On Oct 13, 2016, at 10:48 AM, Brian Candler <b.candler@pobox.com> wrote:
Specify the group as a DN and it won't do the translation.
Thanks, that works.
I guess there is some implicit logic which checks to see if the string tastes like a DN or not. Ah yes: rlm_ldap_is_dn().
:)
If you want to examine the DN values yourself using a foreach loop, toggle on cacheable_dn, all the membership DNs will then be available as LDAP-Group instances.
OK, that works too. It materializes LDAP-Group and LDAP-Group[*], which can then also be used in string expansions.
Well it converts the groups to VP instances, LDAP-Group[*] is just the all instance selector.
I didn't investigate this flag before, because it said it was for use with rlm_cache, which I'm not using.
Some documentation on the magical behaviour of the LDAP-Group attribute would be nice to have :-)
RTFC :p Actually RTFMing reveals: # If cacheable_name or cacheable_dn are enabled, # all group information for the user will be # retrieved from the directory and written to LDAP-Group # attributes appropriate for the instance of rlm_ldap. I added a note about it not just being for rlm_cache.
I can see some advantages do extracting groups from the RDN, so feel free to add an issue ticket.
Done: https://github.com/FreeRADIUS/freeradius-server/issues/1788
Thanks. -Arran
On 13/10/2016 16:07, Arran Cudbard-Bell wrote:
Actually RTFMing reveals:
# If cacheable_name or cacheable_dn are enabled, # all group information for the user will be # retrieved from the directory and written to LDAP-Group # attributes appropriate for the instance of rlm_ldap.
I added a note about it not just being for rlm_cache.
Great. That text did mean anything without the knowledge that control:LDAP-Group is not a "real" attribute, unless you turn on those cacheable_xxxx attributes. Instead, it is a magical pseudo attribute which triggers behind-the-scenes queries when you match on it. Now grepping for paircompare_register in the code, it looks like there are some other attributes which might have similar magical powers.
A couple of other points about LDAP-Group. (1) I tried adding the following for debugging inside a policy module (invoked after the LDAP lookup has taken place): update control { Tmp-String-0 := "%{request:LDAP-Group}" Tmp-String-1 := "%{control:LDAP-Group}" Tmp-String-2 := "%{reply:LDAP-Group}" Tmp-String-3 := "%{request:LDAP-Group[*]}" Tmp-String-4 := "%{control:LDAP-Group[*]}" Tmp-String-5 := "%{reply:LDAP-Group[*]}" } All six variations expand to empty: rlm_ldap (ldap): Bind successful (0) [ldap] = updated (0) policy group_authorization { (0) update control { (0) EXPAND %{request:LDAP-Group} (0) --> (0) Tmp-String-0 := (0) EXPAND %{control:LDAP-Group} (0) --> (0) Tmp-String-1 := (0) EXPAND %{reply:LDAP-Group} (0) --> (0) Tmp-String-2 := (0) EXPAND %{request:LDAP-Group[*]} (0) --> (0) Tmp-String-3 := (0) EXPAND %{control:LDAP-Group[*]} (0) --> (0) Tmp-String-4 := (0) EXPAND %{reply:LDAP-Group[*]} (0) --> (0) Tmp-String-5 := (0) } # update control = noop Changing %{...} to %{&...} didn't make any difference either. Is this expected behaviour? If the LDAP-Group pseudo-attribute is not valid in string expansions, perhaps it should at least generate a warning? (2) I found another fragility issue with LDAP-Group: if you are comparing this attribute but miss the leading &, it silently fails, without even so much as a warning. Here is a snippet from a working policy: if (&Huntgroup-Name == "wifi") { if (&Called-Station-Id =~ /:Admin$/ && ( &LDAP-Group[*] == "server_guru" || &LDAP-Group[*] == "network_guru")) { return } Debug is happy: (0) if (&Huntgroup-Name == "wifi") { (0) if (&Huntgroup-Name == "wifi") -> TRUE (0) if (&Huntgroup-Name == "wifi") { (0) if (&Called-Station-Id =~ /:Admin$/ && ( &LDAP-Group[*] == "server_guru" || &LDAP-Group[*] == "network_guru")) { (0) Searching for user in group "server_guru" rlm_ldap (ldap): Reserved connection (1) (0) Using user DN from request "uid=bcandler,cn=users,cn=accounts,dc=ipa,dc=example,dc=com" (0) Checking user object's memberOf attributes ... snip (0) User found in group "server_guru". Comparison between membership: name (resolved from DN "cn=server_guru,cn=groups,cn=accounts,dc=ipa,dc=example,dc=com"), check: name rlm_ldap (ldap): Released connection (1) (0) if (&Called-Station-Id =~ /:example Admin$/ && ( &LDAP-Group[*] == "server_guru" || &LDAP-Group[*] == "network_guru")) -> TRUE However in the first iteration of writing this policy, I had accidentally missed out one of the & signs. To reproduce the issue, I have changed it to this: if (&Huntgroup-Name == "wifi") { if (&Called-Station-Id =~ /:Admin$/ && ( LDAP-Group[*] == "server_guru" || LDAP-Group[*] == "network_guru")) { return } The only difference is dropping & before the two instances of LDAP-Group[*] The debug output now looks like this: (0) if (&Huntgroup-Name == "wifi") { (0) if (&Called-Station-Id =~ /:example Admin$/ && ( LDAP-Group[*] == "server_guru" || LDAP-Group[*] == "network_guru")) { (0) if (&Called-Station-Id =~ /:example Admin$/ && ( LDAP-Group[*] == "server_guru" || LDAP-Group[*] == "network_guru")) -> FALSE That's it. The whole expression fails, and there's no other indication that I did something wrong. This, I think, could lead to some difficult-to-find bugs in policy. And the unlang documentation does say that the & is optional for attributes on the left-hand side of a comparison operator: " Where the left-hand side is an attribute, the "&" can be omitted." Regards, Brian.
participants (5)
-
A.L.M.Buxey＠lboro.ac.uk -
Alan Buxey
-
Alan DeKok -
Arran Cudbard-Bell -
Brian Candler