Ldap searches don't seem to honour connect_timeout
Hi all, FR 3.1.0 git#64aa7f9 I've been doing some testing around HA and servers looking at multiple ldap sources. For the most part everything is fine and dandy, but when the ldap search can't be made (e.g. one of the round robin dns entries goes off line), the connection timeout doesn't seem to behave. I've tried setting the connect_timeout ldap option, and also the NETWORK_TIMEOUT option in /etc/ldap/ldap.conf, but to no avail; connection timeouts take at least a minute or more. Using ldapsearch does seem to honour the latter perfectly. It's difficult to show the behaviour in a debug readout, since Are there any known issues with FR or the ldap library version that would cause this? Am I missing something!? Config: rlm_ldap (ldap_sath): Couldn't find configuration for accounting, will return NOOP for calls from this section rlm_ldap (ldap_sath): Couldn't find configuration for post-auth, will return NOOP for calls from this section rlm_ldap (ldap_sath): Initialising connection pool pool { start = 0 min = 4 max = 10 spare = 3 uses = 0 lifetime = 0 cleanup_interval = 30 idle_timeout = 20 connect_timeout = 3.000000 retry_delay = 1 spread = no FR output: Tue May 10 17:46:39 2016 : Info: rlm_ldap (ldap_sath): Opening additional connection (2), 1 of 10 pending slots used Tue May 10 17:46:39 2016 : Debug: rlm_ldap (ldap_sath): Connecting to ldaps://sath.nhs.uk:636 Tue May 10 17:46:39 2016 : Debug: rlm_ldap (ldap_sath): New libldap handle 0x2a2ea10 Tue May 10 17:48:46 2016 : Debug: rlm_ldap (ldap_sath): Waiting for bind result... Tue May 10 17:48:46 2016 : Debug: rlm_ldap (ldap_sath): Bind successful Tue May 10 17:48:46 2016 : Debug: rlm_ldap (ldap_sath): Reserved connection (2) Tue May 10 17:48:46 2016 : Debug: (3) Performing search in "dc=SATH,dc=nhs,dc=uk" with filter "sAMAccountName=989096b80618", scope "sub" Tue May 10 17:48:46 2016 : Debug: (3) Waiting for search result... Tue May 10 17:48:46 2016 : Debug: rlm_ldap (ldap_sath): Released connection (2) Tue May 10 17:48:46 2016 : Info: rlm_ldap (ldap_sath): Need 3 more connections to reach 3 spares Tue May 10 17:48:46 2016 : Info: rlm_ldap (ldap_sath): Opening additional connection (3), 1 of 9 pending slots used Tue May 10 17:48:46 2016 : Debug: rlm_ldap (ldap_sath): Connecting to ldaps://sath.nhs.uk:636 Tue May 10 17:48:46 2016 : Debug: rlm_ldap (ldap_sath): New libldap handle 0x2aa2d00 Ldapsearch (it's set to 6 seconds here, but 3 in FR, I was seeing which took precendence. Not setting the ldap.conf, i.e. commenting out gives same result btw). time ldapsearch -x -H ldaps://10.128.176.1 -D "ldapquery" -b "OU=Phones,OU=MAC Addresses,OU=Trust owned,OU=Wired 802.1x MAC,OU=SATHNetwork,DC=SATH,DC=nhs,DC=uk" -s sub "(cn=*)" cn mail sn dn -w <hidden> ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) real 0m6.009s user 0m0.000s sys 0m0.000s Thanks Andy
I've seen some ldap behaviour in similar vein but am currently side tracked by other issues - what I've seen is ldap working but then after some time ldap searches no longer working though the connection settings are to keep connections open and not close. Firstly though, your ldap is compiled with openssl and not NSS, right? alan
Hi Alan, Configure reports the below – is doesn’t say which it’s using afaics:. === configuring in src/modules/rlm_ldap (/home/andy/freeradius-server/src/modules/rlm_ldap) checking for ldap_init in -lldap... yes checking for ldap.h... yes checking for ldap_sasl_interactive_bind... yes checking for ldap_unbind_ext_s... yes checking for ldap_start_tls_s... yes checking for ldap_initialize... yes checking for ldap_set_rebind_proc... yes checking for ldap_create_sort_control... yes checking for ldap_create_sort_keylist... yes checking for ldap_free_sort_keylist... yes checking for ldap_create_session_tracking_control... yes checking for ldap_url_parse... yes checking for ldap_is_ldap_url... yes checking for ldap_url_desc2str... yes checking whether ldap_create_session_tracking_control is declared... no checking whether ldap_set_rebind_proc takes 3 arguments... 3 .. however I don’t have any nss specific named dev libraries installed, but do have libldap2-dev:amd64 which apt-cache says is openldap : Description-en: OpenLDAP development libraries This package allows development of LDAP applications using the OpenLDAP libraries. It includes headers, libraries and links to allow static and dynamic linking. .. so I guess it’s openldap. Thanks Andy From: Alan Buxey [mailto:A.L.M.Buxey@lboro.ac.uk] Sent: 10 May 2016 18:50 To: FreeRadius users mailing list; Franks Andy (IT Technical Architecture Manager); 'freeradius-users@lists.freeradius.org' Subject: Re: Ldap searches don't seem to honour connect_timeout I've seen some ldap behaviour in similar vein but am currently side tracked by other issues - what I've seen is ldap working but then after some time ldap searches no longer working though the connection settings are to keep connections open and not close. Firstly though, your ldap is compiled with openssl and not NSS, right? alan
On May 10, 2016, at 2:00 PM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Hi Alan, Configure reports the below – is doesn’t say which it’s using afaics:.
=== configuring in src/modules/rlm_ldap (/home/andy/freeradius-server/src/modules/rlm_ldap) checking for ldap_init in -lldap... yes checking for ldap.h... yes checking for ldap_sasl_interactive_bind... yes checking for ldap_unbind_ext_s... yes checking for ldap_start_tls_s... yes checking for ldap_initialize... yes checking for ldap_set_rebind_proc... yes checking for ldap_create_sort_control... yes checking for ldap_create_sort_keylist... yes checking for ldap_free_sort_keylist... yes checking for ldap_create_session_tracking_control... yes checking for ldap_url_parse... yes checking for ldap_is_ldap_url... yes checking for ldap_url_desc2str... yes checking whether ldap_create_session_tracking_control is declared... no checking whether ldap_set_rebind_proc takes 3 arguments... 3
.. however I don’t have any nss specific named dev libraries installed, but do have libldap2-dev:amd64 which apt-cache says is openldap :
Description-en: OpenLDAP development libraries This package allows development of LDAP applications using the OpenLDAP libraries. It includes headers, libraries and links to allow static and dynamic linking.
.. so I guess it’s openldap.
If you're running redhat then it will have been built against nss. -Arran
On May 10, 2016, at 2:04 PM, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
If you're running redhat then it will have been built against nss.
I'll see if I can put a "configure" check in for this. Since FR + LDAP + NSS doesn't work, we shouldn't allow the server to be built using it. Alan DeKok.
Hi,
.. however I don’t have any nss specific named dev libraries installed, but do have libldap2-dev:amd64 which apt-cache says is openldap :
Description-en: OpenLDAP development libraries This package allows development of LDAP applications using the OpenLDAP libraries. It includes headers, libraries and links to allow static and dynamic linking.
.. so I guess it’s openldap.
If you're running redhat then it will have been built against nss.
redhat? apt-cache and libldap2-dev package name suggests debian/ubuntu ;-) you can check with ldd against the /usr/lib/openldap* files.... or whereever they are alan
Yes ubuntu, I'll check the ldd links when i am back at my pc. Thanks all. ________________________________________ From: Freeradius-Users [freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] on behalf of A.L.M.Buxey@lboro.ac.uk [A.L.M.Buxey@lboro.ac.uk] Sent: 10 May 2016 19:38 To: Arran Cudbard-Bell Cc: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout Hi,
.. however I don’t have any nss specific named dev libraries installed, but do have libldap2-dev:amd64 which apt-cache says is openldap :
Description-en: OpenLDAP development libraries This package allows development of LDAP applications using the OpenLDAP libraries. It includes headers, libraries and links to allow static and dynamic linking.
.. so I guess it’s openldap.
If you're running redhat then it will have been built against nss.
redhat? apt-cache and libldap2-dev package name suggests debian/ubuntu ;-) you can check with ldd against the /usr/lib/openldap* files.... or whereever they are alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Ok, ldd against rlm_ldap.so gives rlm_ldap.so: linux-vdso.so.1 => (0x00007fff639fe000) libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f7e4884e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7e48485000) liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f7e48276000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f7e4805c000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f7e47e41000) libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f7e47c03000) libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f7e47947000) .. Libldap_r-2.4.so.2 belongs to the openldap package according to apt-file. Is that correct? @rsh-radius-lan0:/usr/local/lib# -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of Franks Andy (IT Technical Architecture Manager) Sent: 10 May 2016 21:30 To: FreeRadius users mailing list; Arran Cudbard-Bell Subject: RE: Ldap searches don't seem to honour connect_timeout Yes ubuntu, I'll check the ldd links when i am back at my pc. Thanks all. ________________________________________ From: Freeradius-Users [freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] on behalf of A.L.M.Buxey@lboro.ac.uk [A.L.M.Buxey@lboro.ac.uk] Sent: 10 May 2016 19:38 To: Arran Cudbard-Bell Cc: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout Hi,
.. however I don’t have any nss specific named dev libraries installed, but do have libldap2-dev:amd64 which apt-cache says is openldap :
Description-en: OpenLDAP development libraries This package allows development of LDAP applications using the OpenLDAP libraries. It includes headers, libraries and links to allow static and dynamic linking.
.. so I guess it’s openldap.
If you're running redhat then it will have been built against nss.
redhat? apt-cache and libldap2-dev package name suggests debian/ubuntu ;-) you can check with ldd against the /usr/lib/openldap* files.... or whereever they are alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On May 10, 2016, at 4:44 PM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Ok, ldd against rlm_ldap.so gives
rlm_ldap.so: ... libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f7e47947000) ..
Ugh. I wouldn't be surprised if that was it. Both GnuTLS and NSS provide compatibility layers for OpenSSL. But.... they're *compatibility* layers, not 100% emulators. The solution is ensure that all libraries and applications use the same SSL library. Since FreeRADIUS *can't* be ported to GnuTLS / NSS, then LDAP, etc. has to be build with OpenSSL. OpenSSL just provides more functionality than the other libraries. We would lose a lot of features if we tried to use them. Alan DeKok.
Thanks Alan, I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think? Thanks again andy ________________________________________ From: Freeradius-Users [freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] on behalf of Alan DeKok [aland@deployingradius.com] Sent: 10 May 2016 21:48 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout On May 10, 2016, at 4:44 PM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Ok, ldd against rlm_ldap.so gives
rlm_ldap.so: ... libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f7e47947000) ..
Ugh. I wouldn't be surprised if that was it. Both GnuTLS and NSS provide compatibility layers for OpenSSL. But.... they're *compatibility* layers, not 100% emulators. The solution is ensure that all libraries and applications use the same SSL library. Since FreeRADIUS *can't* be ported to GnuTLS / NSS, then LDAP, etc. has to be build with OpenSSL. OpenSSL just provides more functionality than the other libraries. We would lose a lot of features if we tried to use them. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi,
I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think?
remove the openldap2-dev package grab the latest openldap source from their page (or mirror) , ./configure, make, make install (it'll all go into /usr/local/ ) then redo the freeradius configure stuff..... should pick up the local openldap dev stuff. ensure tht the local openldap library is known (output of ldconfig -v shows it....may need to add the /usr/local/lib as first path in /etc/ld.so.conf make ; make install - when you do the ldd stuff against rlm_ldap.so it should show openssl linkage instead alan
Ah, thanks Alan. I thought it may be so embedded that it would be more than just recompiling openldap. I'll give it a whirl, and some tests. Cheers, that's great. Andy -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of A.L.M.Buxey@lboro.ac.uk Sent: 10 May 2016 23:52 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout Hi,
I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think?
remove the openldap2-dev package grab the latest openldap source from their page (or mirror) , ./configure, make, make install (it'll all go into /usr/local/ ) then redo the freeradius configure stuff..... should pick up the local openldap dev stuff. ensure tht the local openldap library is known (output of ldconfig -v shows it....may need to add the /usr/local/lib as first path in /etc/ld.so.conf make ; make install - when you do the ldd stuff against rlm_ldap.so it should show openssl linkage instead alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi, Does this look better? Just thought I'd confirm before I went ahead and installed it all.. No mentioned of gnutls and it's got libssl instead, which belongs to libssl1.0.0, the description being Description-en: SSL shared libraries libssl and libcrypto shared libraries needed by programs like apache-ssl, telnet-ssl and openssh. It is part of the OpenSSL implementation of SSL. Ldd output: /home/andy/freeradius-server/build/lib/local/.libs# ldd rlm_ldap.so linux-vdso.so.1 => (0x00007fff1dfec000) libldap-2.4.so.2 => /usr/local/lib/libldap-2.4.so.2 (0x00007fb516981000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb5165b9000) liblber-2.4.so.2 => /usr/local/lib/liblber-2.4.so.2 (0x00007fb5163a9000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fb51618f000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fb515f74000) libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fb515d15000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fb515939000) /lib64/ld-linux-x86-64.so.2 (0x00007fb516df7000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb515735000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fb51551b000) Thanks again all. -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of A.L.M.Buxey@lboro.ac.uk Sent: 10 May 2016 23:52 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout Hi,
I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think?
remove the openldap2-dev package grab the latest openldap source from their page (or mirror) , ./configure, make, make install (it'll all go into /usr/local/ ) then redo the freeradius configure stuff..... should pick up the local openldap dev stuff. ensure tht the local openldap library is known (output of ldconfig -v shows it....may need to add the /usr/local/lib as first path in /etc/ld.so.conf make ; make install - when you do the ldd stuff against rlm_ldap.so it should show openssl linkage instead alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hmm, so it looks like it might be using the correct libraries now, as the ldap debug string now works, for example here where I removed the self-signed CA for the self-signed cert for ldaps: TLS: can't connect: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (unable to get local issuer certificate). rlm_ldap (ldap1): Bind with CN=LDAPQuery_RADIUS,OU=NonLockableUsers,OU=SpecialUsers,OU=SATHUsers,DC=SATH,DC=nhs,DC=uk to ldaps://sath.nhs.uk:636 failed: Can't contact LDAP server rlm_ldap (ldap1): Opening connection failed (0) It didn't used to spit out any debugging with the GNU tls library as far as I can remember. Getting past that though, it still stalls when the host is dead : rlm_ldap (ldap1): Initialising connection pool pool { .. connect_timeout = 2.000000 .. } It still takes a couple of minutes: Wed May 11 14:23:29 2016 : Info: rlm_ldap (ldap1): Opening additional connection (0), 1 of 10 pending slots used Wed May 11 14:23:29 2016 : Debug: rlm_ldap (ldap1): Connecting to ldaps://sath-ad1wk8.sath.nhs.uk:636 Wed May 11 14:23:29 2016 : Debug: rlm_ldap (ldap1): New libldap handle 0x209a550 Wed May 11 14:25:36 2016 : Error: rlm_ldap (ldap1): Bind with CN=LDAPQuery_RADIUS,OU=NonLockableUsers,OU=SpecialUsers,OU=SATHUsers,DC=SATH,DC=nhs,DC=uk to ldaps://sath.nhs.uk:636 failed: Can't contact LDAP server Wed May 11 14:25:36 2016 : Debug: rlm_ldap: Closing libldap handle 0x209a550 Wed May 11 14:25:36 2016 : Error: rlm_ldap (ldap1): Opening connection failed (0) Wed May 11 14:25:36 2016 : Debug: (0) modsingle[authorize]: returned from ldap1 (ldap) for request 0 Not sure what else to try! Thanks Andy -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of Franks Andy (IT Technical Architecture Manager) Sent: 11 May 2016 12:46 To: 'FreeRadius users mailing list' Subject: RE: Ldap searches don't seem to honour connect_timeout Hi, Does this look better? Just thought I'd confirm before I went ahead and installed it all.. No mentioned of gnutls and it's got libssl instead, which belongs to libssl1.0.0, the description being Description-en: SSL shared libraries libssl and libcrypto shared libraries needed by programs like apache-ssl, telnet-ssl and openssh. It is part of the OpenSSL implementation of SSL. Ldd output: /home/andy/freeradius-server/build/lib/local/.libs# ldd rlm_ldap.so linux-vdso.so.1 => (0x00007fff1dfec000) libldap-2.4.so.2 => /usr/local/lib/libldap-2.4.so.2 (0x00007fb516981000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb5165b9000) liblber-2.4.so.2 => /usr/local/lib/liblber-2.4.so.2 (0x00007fb5163a9000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fb51618f000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fb515f74000) libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fb515d15000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fb515939000) /lib64/ld-linux-x86-64.so.2 (0x00007fb516df7000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb515735000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fb51551b000) Thanks again all. -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of A.L.M.Buxey@lboro.ac.uk Sent: 10 May 2016 23:52 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout Hi,
I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think?
remove the openldap2-dev package grab the latest openldap source from their page (or mirror) , ./configure, make, make install (it'll all go into /usr/local/ ) then redo the freeradius configure stuff..... should pick up the local openldap dev stuff. ensure tht the local openldap library is known (output of ldconfig -v shows it....may need to add the /usr/local/lib as first path in /etc/ld.so.conf make ; make install - when you do the ldd stuff against rlm_ldap.so it should show openssl linkage instead alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Just for reference, it seems ubuntu have taken a decision to use GnuTLS for tls encryption over openSSL due to licensing worries - see the top paragraph of this: https://help.ubuntu.com/community/GnuTLS so.. anyone seeking to compile Freeradius from scratch and using ldap will be using some bits compiled with GnuTLS intead of openSSL. For me this seems to be some lack of support for certain freeradius ldap options in the configuration, and lack of debugging options working. Hopefully that's helpful. Thanks Andy -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of Franks Andy (IT Technical Architecture Manager) Sent: 11 May 2016 14:29 To: 'FreeRadius users mailing list' Subject: RE: Ldap searches don't seem to honour connect_timeout Hmm, so it looks like it might be using the correct libraries now, as the ldap debug string now works, for example here where I removed the self-signed CA for the self-signed cert for ldaps: TLS: can't connect: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (unable to get local issuer certificate). rlm_ldap (ldap1): Bind with CN=LDAPQuery_RADIUS,OU=NonLockableUsers,OU=SpecialUsers,OU=SATHUsers,DC=SATH,DC=nhs,DC=uk to ldaps://sath.nhs.uk:636 failed: Can't contact LDAP server rlm_ldap (ldap1): Opening connection failed (0) It didn't used to spit out any debugging with the GNU tls library as far as I can remember. Getting past that though, it still stalls when the host is dead : rlm_ldap (ldap1): Initialising connection pool pool { .. connect_timeout = 2.000000 .. } It still takes a couple of minutes: Wed May 11 14:23:29 2016 : Info: rlm_ldap (ldap1): Opening additional connection (0), 1 of 10 pending slots used Wed May 11 14:23:29 2016 : Debug: rlm_ldap (ldap1): Connecting to ldaps://sath-ad1wk8.sath.nhs.uk:636 Wed May 11 14:23:29 2016 : Debug: rlm_ldap (ldap1): New libldap handle 0x209a550 Wed May 11 14:25:36 2016 : Error: rlm_ldap (ldap1): Bind with CN=LDAPQuery_RADIUS,OU=NonLockableUsers,OU=SpecialUsers,OU=SATHUsers,DC=SATH,DC=nhs,DC=uk to ldaps://sath.nhs.uk:636 failed: Can't contact LDAP server Wed May 11 14:25:36 2016 : Debug: rlm_ldap: Closing libldap handle 0x209a550 Wed May 11 14:25:36 2016 : Error: rlm_ldap (ldap1): Opening connection failed (0) Wed May 11 14:25:36 2016 : Debug: (0) modsingle[authorize]: returned from ldap1 (ldap) for request 0 Not sure what else to try! Thanks Andy -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of Franks Andy (IT Technical Architecture Manager) Sent: 11 May 2016 12:46 To: 'FreeRadius users mailing list' Subject: RE: Ldap searches don't seem to honour connect_timeout Hi, Does this look better? Just thought I'd confirm before I went ahead and installed it all.. No mentioned of gnutls and it's got libssl instead, which belongs to libssl1.0.0, the description being Description-en: SSL shared libraries libssl and libcrypto shared libraries needed by programs like apache-ssl, telnet-ssl and openssh. It is part of the OpenSSL implementation of SSL. Ldd output: /home/andy/freeradius-server/build/lib/local/.libs# ldd rlm_ldap.so linux-vdso.so.1 => (0x00007fff1dfec000) libldap-2.4.so.2 => /usr/local/lib/libldap-2.4.so.2 (0x00007fb516981000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb5165b9000) liblber-2.4.so.2 => /usr/local/lib/liblber-2.4.so.2 (0x00007fb5163a9000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fb51618f000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fb515f74000) libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fb515d15000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fb515939000) /lib64/ld-linux-x86-64.so.2 (0x00007fb516df7000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb515735000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fb51551b000) Thanks again all. -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of A.L.M.Buxey@lboro.ac.uk Sent: 10 May 2016 23:52 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout Hi,
I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think?
remove the openldap2-dev package grab the latest openldap source from their page (or mirror) , ./configure, make, make install (it'll all go into /usr/local/ ) then redo the freeradius configure stuff..... should pick up the local openldap dev stuff. ensure tht the local openldap library is known (output of ldconfig -v shows it....may need to add the /usr/local/lib as first path in /etc/ld.so.conf make ; make install - when you do the ldd stuff against rlm_ldap.so it should show openssl linkage instead alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On May 11, 2016, at 11:31 AM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Just for reference, it seems ubuntu have taken a decision to use GnuTLS for tls encryption over openSSL due to licensing worries - see the top paragraph of this: https://help.ubuntu.com/community/GnuTLS
Ugh. The problem is that they're just not compatible.
so.. anyone seeking to compile Freeradius from scratch and using ldap will be using some bits compiled with GnuTLS intead of openSSL. For me this seems to be some lack of support for certain freeradius ldap options in the configuration, and lack of debugging options working.
Ubuntu is welcome to submit patches which allow FreeRADIUS to build with GnuTLS. Until then, FreeRADIUS will be broken on their platform, because they broke it. Alan DeKok.
On Wed, May 11, 2016 at 11:43:22AM -0400, Alan DeKok wrote:
On May 11, 2016, at 11:31 AM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Just for reference, it seems ubuntu have taken a decision to use GnuTLS for tls encryption over openSSL due to licensing worries - see the top paragraph of this: https://help.ubuntu.com/community/GnuTLS
Ugh. The problem is that they're just not compatible.
I believe it's been Debian policy for years to use GnuTLS over OpenSSL if the application has support for it. The OpenSSL licence issue is so annoying.
so.. anyone seeking to compile Freeradius from scratch and using ldap will be using some bits compiled with GnuTLS intead of openSSL. For me this seems to be some lack of support for certain freeradius ldap options in the configuration, and lack of debugging options working.
Ubuntu is welcome to submit patches which allow FreeRADIUS to build with GnuTLS. Until then, FreeRADIUS will be broken on their platform, because they broke it.
Hmm, this got me digging... it didn't help that we hit an ldap problem today (which is a weird one, caused by a group getting too big - it may be related to this, I'm still digging). It turns out Debian is also linked against gnutls. However, having run with ldap code for ages, apart from today, things have generally been solid. Not saying libldap linked against gnutls is perfect, but given our experience and the queries on this list, it seems to be more stable than when linked against nss. If I get time I'll compile libldap against openssl and see if that fixes the issue we've seen today. In which case it'll be one case against gnutls as well. Cheers, Matthew -- Matthew Newton, Ph.D. <mcn4@le.ac.uk> Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk>
On May 11, 2016, at 11:31 AM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Just for reference, it seems ubuntu have taken a decision to use GnuTLS for tls encryption over openSSL due to licensing worries - see the top paragraph of this: https://help.ubuntu.com/community/GnuTLS
so.. anyone seeking to compile Freeradius from scratch and using ldap will be using some bits compiled with GnuTLS intead of openSSL. For me this seems to be some lack of support for certain freeradius ldap options in the configuration, and lack of debugging options working.
Network timeout should make the bind timeout. Unless it's not a network timeout. -Arran
Thanks Alan, That's great; I'll check it out. Andy -----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of Alan DeKok Sent: 11 May 2016 19:58 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout On May 11, 2016, at 2:02 PM, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
Network timeout should make the bind timeout. Unless it's not a network timeout.
I've pushed some more fixes. v3.1.x head should now honour time out on initial connect. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi, I've tried with version git#8303894, but still a couple of minutes before timeout. It's weird, it always seems to be 127 - 128 seconds, regardless of whether connect_timeout=2.0 or 20.0, maybe of no consequence. Fri May 13 12:01:26 2016 : Debug : (1) ldap1 - 0 of 0 connections in use. You may need to increase "spare" Fri May 13 12:01:26 2016 : Debug : (1) ldap1 - Opening additional connection (0), 1 of 10 pending slots used Fri May 13 12:01:26 2016 : Debug : rlm_ldap (ldap1) - Connecting to ldaps://sath-ad1wk8.sath.nhs.uk:636 Fri May 13 12:01:26 2016 : Debug : rlm_ldap (ldap1) - New libldap handle 0x2ad90f0 Fri May 13 12:03:34 2016 : Debug : rlm_ldap (ldap1) - Closing libldap handle 0x2ad90f0 Fri May 13 12:03:34 2016 : ERROR : (1) ldap1 - Opening connection failed (0) Fri May 13 12:03:34 2016 : Debug : (1) modsingle[authorize]: returned from ldap1 (ldap) for request 1 Fri May 13 12:03:34 2016 : Debug : (1) ldap1.authorize (fail) Fri May 13 12:03:34 2016 : Debug : (1) if (updated) { The ldd output seems to be using the compiled version of openldap as previously noted: linux-vdso.so.1 => (0x00007fff341fe000) libldap-2.4.so.2 => /usr/local/lib/libldap-2.4.so.2 (0x00007f0712920000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0712558000) liblber-2.4.so.2 => /usr/local/lib/liblber-2.4.so.2 (0x00007f0712348000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f071212e000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f0711f13000) libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f0711cb4000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f07118d8000) /lib64/ld-linux-x86-64.so.2 (0x00007f0712d99000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f07116d4000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f07114ba000) It's a fairly straightforward test I'm trying, just in case that has any bearing on things - I've just nobbled the dns lookup via the hosts file to point to an ip address which is "dead" (just for testing!, I know it's dirty). Thanks Andy
Thanks Alan, That's great; I'll check it out. Andy
-----Original Message----- From: Freeradius-Users [mailto:freeradius-users-bounces+andy.franks=sath.nhs.uk@lists.freeradius.org] On Behalf Of Alan DeKok Sent: 11 May 2016 19:58 To: FreeRadius users mailing list Subject: Re: Ldap searches don't seem to honour connect_timeout On May 11, 2016, at 2:02 PM, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
Network timeout should make the bind timeout. Unless it's not a network timeout.
I've pushed some more fixes. v3.1.x head should now honour time out on initial connect. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi,
I've tried with version git#8303894, but still a couple of minutes before timeout. It's weird, it always seems to be 127 - 128 seconds, regardless of whether connect_timeout=2.0 or 20.0, maybe of no consequence.
during server startup? set , in your pool start = 0 (rather than start = ${thread[pool].start_servers} ) alan
Franks Andy (IT Technical Architecture Manager) wrote:
Just for reference, it seems ubuntu have taken a decision to use GnuTLS for tls encryption over openSSL due to licensing worries - see the top paragraph of this: https://help.ubuntu.com/community/GnuTLS
This has been decided in the Debian project many years ago. It was a very unwise decision because the OpenLDAP developers do not really care of the GnuTLS binding. Ciao, Michael.
A.L.M.Buxey@lboro.ac.uk wrote:
Hi,
I followed part of that, can't profess to be an expert in library interaction and such, sorry. I am not sure really what to do next. Most of the libraries I got from standard installs off the ubuntu repos, are we talking fairly significant compilation of other stuff do you think?
remove the openldap2-dev package
grab the latest openldap source from their page (or mirror) , ./configure, make, make install (it'll all go into /usr/local/ )
then redo the freeradius configure stuff..... should pick up the local openldap dev stuff.
ensure tht the local openldap library is known (output of ldconfig -v shows it....may need to add the /usr/local/lib as first path in /etc/ld.so.conf
make ; make install - when you do the ldd stuff against rlm_ldap.so it should show openssl linkage instead
Be very careful which such an advice. Depending on the system's nsswitch.cnf using custom LDAP libs can lead to a library mix crashing the software. You could do it right. But you have to really know what you're doing regarding all the dependency stack of libldap -> cyrus-sasl -> berkeley-DB... Ciao, Michael.
Hi, I have RHEL7, freeradius-3.0.11-1.el7.x86_64.rpm, freeradius-ldap-3.0.11-1.el7.x86_64.rpm (built myself, using the very helpful documentation here: http://wiki.freeradius.org/guide/Red-Hat-FAQ) Currently I am using a 389ds ldap on another host without ssl, but I'm planning to change to ssl. ldd rlm_ldap.so linux-vdso.so.1 => (0x00007ffd5f51e000) libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f734d771000) libc.so.6 => /lib64/libc.so.6 (0x00007f734d3b0000) liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f734d1a0000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f734cf86000) libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f734cd69000) libssl3.so => /lib64/libssl3.so (0x00007f734cb26000) libsmime3.so => /lib64/libsmime3.so (0x00007f734c8ff000) libnss3.so => /lib64/libnss3.so (0x00007f734c5d9000) libnssutil3.so => /lib64/libnssutil3.so (0x00007f734c3ac000) libplds4.so => /lib64/libplds4.so (0x00007f734c1a8000) libplc4.so => /lib64/libplc4.so (0x00007f734bfa3000) libnspr4.so => /lib64/libnspr4.so (0x00007f734bd64000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f734bb48000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f734b944000) /lib64/ld-linux-x86-64.so.2 (0x00007f734dbe3000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f734b70c000) libz.so.1 => /lib64/libz.so.1 (0x00007f734b4f6000) librt.so.1 => /lib64/librt.so.1 (0x00007f734b2ed000) libfreebl3.so => /lib64/libfreebl3.so (0x00007f734b0ea000) Looks like I have to rebuild my freeradius-ldap too to use openssl right? The RedHat documentation on the freeradius site doesn't say anything about how to switch to openssl. Are there any pointers how to do this? Yours: Laszlo On 05/10/2016 10:48 PM, Alan DeKok wrote:
On May 10, 2016, at 4:44 PM, Franks Andy (IT Technical Architecture Manager) <Andy.Franks@sath.nhs.uk> wrote:
Ok, ldd against rlm_ldap.so gives
rlm_ldap.so: ... libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f7e47947000) ..
Ugh. I wouldn't be surprised if that was it.
Both GnuTLS and NSS provide compatibility layers for OpenSSL. But.... they're *compatibility* layers, not 100% emulators.
The solution is ensure that all libraries and applications use the same SSL library. Since FreeRADIUS *can't* be ported to GnuTLS / NSS, then LDAP, etc. has to be build with OpenSSL.
OpenSSL just provides more functionality than the other libraries. We would lose a lot of features if we tried to use them.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Ok, ldd against rlm_ldap.so gives
rlm_ldap.so: ... libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f7e47947000) ..
Ugh. I wouldn't be surprised if that was it.
Looks like I have to rebuild my freeradius-ldap too to use openssl right? The RedHat documentation on the freeradius site doesn't say anything about how to switch to openssl. Are there any pointers how to do this?
There is no accounting for taste, but there are numerous posts on numerous lists advising against GnuTLS, even OpenLDAP guys do not recommend RedHat packages at all. It is always reminded here on this list that we should read, and read and read as much as we can. Far from 'endorsing' any technology, try to switch to OpenSSL. There are plenty of tutorials online about how to do that. Good luck.
Hi,
Looks like I have to rebuild my freeradius-ldap too to use openssl right? The RedHat documentation on the freeradius site doesn't say anything about how to switch to openssl. Are there any pointers how to do this?
already told you how remove any local openldap devel package that was installed from redhat repos - as that will use NSS download latest openldap2 code (obviously I assume you have the openssl-devel package installed) ./configure make make install (this will slap (no pun intended!) everything into /usr/local/* rather than over the RPM openldap (which you cannot remove due to many many package dependencies) then ensure /usr/local/lib is in /etc/ld.so.conf file and run 'ldconfig -v' now, rebuild freeradius3 as you've already done. the ./configure part will pick up the openldap includes (and only your local one as you've removed the redhat includes....) et voila. freeradius now with openldap2 using OpenSSL (verify check with ldd against the rlm_ldap.so file) TLS/STARTSSL openldap connections will now open/close/repon/reestablish etc alan
On 05/11/2016 10:47 AM, A.L.M.Buxey@lboro.ac.uk wrote:
Hi,
Looks like I have to rebuild my freeradius-ldap too to use openssl right? The RedHat documentation on the freeradius site doesn't say anything about how to switch to openssl. Are there any pointers how to do this?
already told you how
OK, thank you for your response, I am likely to end up doing that. However, as far as I know RHEL supports both nss and openssl (e.g. for the apache httpd), so there should be a way to build the freeradius rpm properly using openssl libs. Perhaps someone who knows more about rpm building than me can do that and add it to the freeradius wiki. Yours: Laszlo
remove any local openldap devel package that was installed from redhat repos - as that will use NSS
download latest openldap2 code (obviously I assume you have the openssl-devel package installed)
./configure make make install
(this will slap (no pun intended!) everything into /usr/local/* rather than over the RPM openldap (which you cannot remove due to many many package dependencies)
then ensure /usr/local/lib is in /etc/ld.so.conf file and run 'ldconfig -v'
now, rebuild freeradius3 as you've already done. the ./configure part will pick up the openldap includes (and only your local one as you've removed the redhat includes....)
et voila. freeradius now with openldap2 using OpenSSL (verify check with ldd against the rlm_ldap.so file)
TLS/STARTSSL openldap connections will now open/close/repon/reestablish etc
alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Freeradius does build using openssl ... the issue here is that another package - openldap , which freeradius links against, isnt. So follow previous instructions posted twice to this list this very week alan
participants (9)
-
A.L.M.Buxey@lboro.ac.uk -
Alan Buxey -
Alan DeKok -
Arran Cudbard-Bell -
Franks Andy (IT Technical Architecture Manager) -
Matthew Newton -
Michael Ströder -
Sergio NNX -
Tornoci Laszlo