Hi, For our LDAP queries, we have specified the forest DNS name as the LDAP server, so that we achieve via DNS a random distribution of queries against our AD servers. Previously we had hammered the first server on the list. This has kept our AD guys happy, but we have noticed that at busy times our FR servers are doing over 100 DNS queries per second, for the same thing. I can not spot anything in the FreeRADIUS config files, but is there any way to reduce the number of DNS lookups? We are running 3.0.12 on RHEL7. Outside of FreeRADIUS, we could either make sure each server has a different /etc/hosts file, which is not ideal, or implementing local DNS caching seems to be the more sensible way to go. Are there any other alternatives other people are using? Yours Dave Hartburn
On 02/21/2017 07:10 AM, David Hartburn wrote:
Hi,
For our LDAP queries, we have specified the forest DNS name as the LDAP server, so that we achieve via DNS a random distribution of queries against our AD servers. Previously we had hammered the first server on the list.
This has kept our AD guys happy, but we have noticed that at busy times our FR servers are doing over 100 DNS queries per second, for the same thing.
I can not spot anything in the FreeRADIUS config files, but is there any way to reduce the number of DNS lookups? We are running 3.0.12 on RHEL7.
Outside of FreeRADIUS, we could either make sure each server has a different /etc/hosts file, which is not ideal, or implementing local DNS caching seems to be the more sensible way to go. Are there any other alternatives other people are using?
Yours
Dave Hartburn - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
i have wondered if using SRV records would be feasible. the SSSD project has a nifty config option that allows you to specify "_srv_" as a value in a comma separated string, to indicate the use of the _ldap._tcp.domain.tld and _kerberos._udp.domain.tld SRV records. the SRV record can be set with weight and priority to steer load, and is a round-robin style load balancing mechanism. in addition, you can also specify specific ldap URIs or kerberos servers in the config option for fallback to specific servers.
On Feb 21, 2017, at 7:10 AM, David Hartburn <D.J.Hartburn@kent.ac.uk> wrote:
For our LDAP queries, we have specified the forest DNS name as the LDAP server, so that we achieve via DNS a random distribution of queries against our AD servers. Previously we had hammered the first server on the list.
This has kept our AD guys happy, but we have noticed that at busy times our FR servers are doing over 100 DNS queries per second, for the same thing.
I can not spot anything in the FreeRADIUS config files, but is there any way to reduce the number of DNS lookups? We are running 3.0.12 on RHEL7.
DNS doesn't work that way. FreeRADIUS doesn't work that way. It's simple. If you put a DNS name into the configuration file, it's resolved to an IP address when the server starts, and cached forever. If your LDAP server returns a redirect (as AD does... a LOT), that redirect is a DNS name. That name *must* then be resolved to an IP at run time. There is simply no other way to do this. The DNS resolver on the RADIUS system is responsible for doing DNS lookups, and for caching the results. If it doesn't cache the results, you'll see endless DNS queries.
Outside of FreeRADIUS, we could either make sure each server has a different /etc/hosts file, which is not ideal, or implementing local DNS caching seems to be the more sensible way to go. Are there any other alternatives other people are using?
Configure a local DNS caching resolver on each RADIUS system. The DNS lookups will be done once, and then cached. But the underlying problem is likely that your AD system is returning redirects. A LOT of them. There may be ways for FreeRADIUS to do fewer LDAP lookups, but without knowing more about your use-case, it's hard to tell. Alan DeKok.
On Tue, Feb 21, 2017 at 07:48:25AM -0500, Alan DeKok wrote:
On Feb 21, 2017, at 7:10 AM, David Hartburn <D.J.Hartburn@kent.ac.uk> wrote:
For our LDAP queries, we have specified the forest DNS name as the LDAP server, so that we achieve via DNS a random distribution of queries against our AD servers. Previously we had hammered the first server on the list.
But the underlying problem is likely that your AD system is returning redirects. A LOT of them.
If the data is in the Global Catalogue, point FreeRADIUS at port 3269 on the DCs instead of 636. It should stop the referrals, and therefore also speed up LDAP searches. Otherwise, just chuck BIND on the box and point at that for DNS. Matthew -- Matthew Newton, Ph.D. <mcn4@leicester.ac.uk> Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk>
Hi,
If the data is in the Global Catalogue, point FreeRADIUS at port 3269 on the DCs instead of 636. It should stop the referrals, and therefore also speed up LDAP searches.
yes, that can be useful - should be in the LDAP FAQ
Otherwise, just chuck BIND on the box and point at that for DNS.
unbound :) alan
Matthew Newton wrote:
On Tue, Feb 21, 2017 at 07:48:25AM -0500, Alan DeKok wrote:
On Feb 21, 2017, at 7:10 AM, David Hartburn <D.J.Hartburn@kent.ac.uk> wrote:
For our LDAP queries, we have specified the forest DNS name as the LDAP server, so that we achieve via DNS a random distribution of queries against our AD servers. Previously we had hammered the first server on the list.
But the underlying problem is likely that your AD system is returning redirects. A LOT of them.
If the data is in the Global Catalogue, point FreeRADIUS at port 3269 on the DCs instead of 636. It should stop the referrals, and therefore also speed up LDAP searches.
Since chasing LDAPv3 referrals is a broken concept I always recommend to set chase_referrals = no IMHO it should be the default in the FreeRADIUS sample config. Ciao, Michael.
Alan DeKok wrote:
It's simple. If you put a DNS name into the configuration file, it's resolved to an IP address when the server starts, and cached forever.
I think we are talking of the the server= entries in an ldap config section which are passed to libldap directly and the DNS entries are looked up at runtime from there. For load balancing LDAP here we simply have multiple server= lines all specifying the same DNS name, which has roundrobin entries and this seems to work just fine, especially given the connection pooling. Most of our DNS traffic is windbind finding kerberos SRV records... but then we don't LDAP directly to AD. 100 DNS lookups per second being a problem would seem to indicate the DNS is not well situated close to the authentication server, though.
On Tue, 2017-02-21 at 12:10 +0000, David Hartburn wrote:
I can not spot anything in the FreeRADIUS config files, but is there any way to reduce the number of DNS lookups? We are running 3.0.12 on RHEL7.
Install a caching resolver. nscd works, but has some caveats when working with SSSD (look in the RHEL 7 admin guide for details) or systemd-resolved. -- Regards, Adam Bishop gpg: 0x6609D460 jisc.ac.uk Jisc is a registered charity (number 1149740) and a company limited by guarantee which is registered in England under Company No. 5747339, VAT No. GB 197 0632 86. Jisc’s registered office is: One Castlepark, Tower Hill, Bristol, BS2 0JA. T 0203 697 5800. Jisc Services Limited is a wholly owned Jisc subsidiary and a company limited by guarantee which is registered in England under company number 2881024, VAT number GB 197 0632 86. The registered office is: One Castle Park, Tower Hill, Bristol BS2 0JA. T 0203 697 5800.
Adam Bishop wrote:
On Tue, 2017-02-21 at 12:10 +0000, David Hartburn wrote:
I can not spot anything in the FreeRADIUS config files, but is there any way to reduce the number of DNS lookups? We are running 3.0.12 on RHEL7.
Install a caching resolver. nscd works, but has some caveats when working with SSSD (look in the RHEL 7 admin guide for details) or systemd-resolved.
Yes, one would have to tweak nscd.conf to disable caching for the maps served by sssd. See also this: https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.o... Ciao, Michael.
David Hartburn wrote:
For our LDAP queries, we have specified the forest DNS name as the LDAP server, so that we achieve via DNS a random distribution of queries against our AD servers. Previously we had hammered the first server on the list.
You're doing DNS round-robin now? Note that this does not necessarily means good load-balancing.
This has kept our AD guys happy, but we have noticed that at busy times our FR servers are doing over 100 DNS queries per second, for the same thing.
What is "for the same thing"? What are the DNS TTLs for the queried records? You also did not say anything about the authc mechs used. If Kerberos or similar is used there could be more DNS lookups. As others said I'd also recommend to use a local caching DNS resolver in case you cannot improve anything else. Ciao, Michael.
participants (8)
-
A.L.M.Buxey＠lboro.ac.uk -
Adam Bishop -
Alan DeKok -
Brendan Kearney -
Brian Julin -
David Hartburn -
Matthew Newton -
Michael Ströder