Running ntlm_auth as a connection pool
Hi All, I am seeing some performance challenges with ntlm_auth (Currently running 3.0.15). My setup is pretty non-standard so I will apologise for this beforehand. But my hands are tied. Our mschap module actually calls a shell script which than calls ntlm_auth (we actually need to perform an ldap lookup before obtaining the samaccount name to pass to ntlm_auth), as expected this throws up some challenges under load. What I would like to know if its possible to do anything to improve performance under this circumstances. I know the direct winbind mode would have been great but the ldap lookup is critical for things to work our end. Mr Google returned some suggestions that their was an option to run ntlm_auth as a connection pool , is it still there / was it dropped in favour of direct winbind connectivity. Any suggestions welcome. Many Thanks Arnab
Easy. Use the native winbind module in latest 3.0.x series. That runs as a connection pool and is orders of magnitudes faster. Secondly, bash script, calling LD and p and doing things? Terrible for performance. Use the native ldap functionality in FR to get that value out of ldap (create a new LDAP module if needed to separate functions). Use unlang to collect the value and assign it to temporary internal attribute value. Then use that value in your winbind call. All native, all threaded, all fast! alan On 31 Aug 2017 11:11 am, "Arnab Roy" <arnabroy@mail.com> wrote:
Hi All,
I am seeing some performance challenges with ntlm_auth (Currently running 3.0.15). My setup is pretty non-standard so I will apologise for this beforehand. But my hands are tied.
Our mschap module actually calls a shell script which than calls ntlm_auth (we actually need to perform an ldap lookup before obtaining the samaccount name to pass to ntlm_auth), as expected this throws up some challenges under load.
What I would like to know if its possible to do anything to improve performance under this circumstances. I know the direct winbind mode would have been great but the ldap lookup is critical for things to work our end.
Mr Google returned some suggestions that their was an option to run ntlm_auth as a connection pool , is it still there / was it dropped in favour of direct winbind connectivity.
Any suggestions welcome.
Many Thanks Arnab - List info/subscribe/unsubscribe? See http://www.freeradius.org/ list/users.html
Hi All,
I am seeing some performance challenges with ntlm_auth (Currently running 3.0.15). My setup is pretty non-standard so I will apologise for this beforehand. But my hands are tied.
Our mschap module actually calls a shell script which than calls ntlm_auth (we actually need to perform an ldap lookup before obtaining the samaccount name to pass to ntlm_auth), as expected this throws up some challenges under load.
What I would like to know if its possible to do anything to improve performance under this circumstances. I know the direct winbind mode would have been great but the ldap lookup is critical for things to work our end.
Mr Google returned some suggestions that their was an option to run ntlm_auth as a connection pool , is it still there / was it dropped in favour of direct winbind connectivity.
Any suggestions welcome.
Many Thanks Arnab - List info/subscribe/unsubscribe? See [1]http://www.freeradius.org/ list/users.html
Hi Alan, Thanks for the suggestion I am with you on using native ldap, I guest I can just do an on the fly ldap xlat to get the correct username info. However, I forgot to mention another challenge! Our winbind socket is sitting somewhere else as we run multiple instances of winbind :( ntlm_auth allows me to specify where it is. Cant see an option when direct winbind even it means having to do some local patching that would do. Just need to know how I can pass this to Freeradius ? Thanks again for your help. Arnab Sent: Thursday, August 31, 2017 at 11:18 AM From: "Alan Buxey" <alan.buxey@gmail.com> To: "FreeRadius users mailing list" <freeradius-users@lists.freeradius.org> Subject: Re: Running ntlm_auth as a connection pool Easy. Use the native winbind module in latest 3.0.x series. That runs as a connection pool and is orders of magnitudes faster. Secondly, bash script, calling LD and p and doing things? Terrible for performance. Use the native ldap functionality in FR to get that value out of ldap (create a new LDAP module if needed to separate functions). Use unlang to collect the value and assign it to temporary internal attribute value. Then use that value in your winbind call. All native, all threaded, all fast! alan On 31 Aug 2017 11:11 am, "Arnab Roy" <arnabroy@mail.com> wrote: - List info/subscribe/unsubscribe? See [2]http://www.freeradius.org/list/users.html References 1. http://www.freeradius.org/ 2. http://www.freeradius.org/list/users.html
Quickest way is to just run multiple freeradius servers in their own docker containers and just proxy to the one required :) You might want to profile your authentication, add time checks to your bash etc, maybe just using native ldap will help a lot (connection already up, connection pools etc ) alan
On 31 August 2017 11:26:32 BST, Arnab Roy <arnabroy@mail.com> wrote:
Thanks for the suggestion I am with you on using native ldap, I guest I can just do an on the fly ldap xlat to get the correct username info. However, I forgot to mention another challenge! Our winbind socket is sitting somewhere else as we run multiple instances of winbind
IIRC the socket location is hard coded into libwbclient at Samba compile time. FreeRADIUS can't change it. -- Matthew
Hi Matthew, Yes I know this as I had to make some adjustments to NTLM_AUTH to achieve the same. There is some old code on github for running ntlm auth as a pool . Can it still be used ? Many Thanks Arnab
Thanks for the suggestion I am with you on using native ldap, I guest I can just do an on the fly ldap xlat to get the correct username info. However, I forgot to mention another challenge! Our winbind socket is sitting somewhere else as we run multiple instances of winbind IIRC the socket location is hard coded into libwbclient at Samba compile time. FreeRADIUS can't change it. -- Matthew
Hi Mathew/Alan, Just a small additional question, so as it stands unless I completely re-do this whole piece , I am highly unlikely to get out of this hole. So as a temporary suggestion I have got plenty of CPU , DISK IO and Memory resources. Fr is barely using anything , any parameters I can change so FR uses the spare hardware resources to process the concurrent mschap requests ? Many Thanks Arnab Sent: Thursday, August 31, 2017 at 12:29 PM From: "Matthew Newton" <matthew@newtoncomputing.co.uk> To: "FreeRadius users mailing list" <freeradius-users@lists.freeradius.org> Subject: Re: Running ntlm_auth as a connection pool On 31 August 2017 11:26:32 BST, Arnab Roy <arnabroy@mail.com> wrote: - List info/subscribe/unsubscribe? See [1]http://www.freeradius.org/list/users.html References 1. http://www.freeradius.org/list/users.html
On Aug 31, 2017, at 9:56 AM, Arnab Roy <arnabroy@mail.com> wrote:
Just a small additional question, so as it stands unless I completely re-do this whole piece , I am highly unlikely to get out of this hole. So as a temporary suggestion I have got plenty of CPU , DISK IO and Memory resources. Fr is barely using anything , any parameters I can change so FR uses the spare hardware resources to process the concurrent mschap requests ?
Do LDAP lookups in FreeRADIUS. That will help a bit. The real limitation is Samba / Winbind. If their libraries allowed for ntlm_auth / rlm_winbind to set the winbind path, it would be simple. Since that path is hard-coded into their libraries, it's much more difficult. For v4, we're working on making it asynchronous. So you should be able to run many, many, instances of ntlm_auth without having the server wait for each one. Alan DeKok.
Hi Alan, That sounds really good re-v4. to be honest everything looks really good on v4. You are spot with the winbind comment, I did some tracing this morning with the ldap lookup turned off it's taking a fair amount of time for ntlm auth to connect to winbind and getting the auth hash. I have no idea why smb.conf allows you to specify different paths for things and all works. But the same support is missing on client utilities. I had to hack nss-switch/wb_common.c in samba to get the path loaded from a custom parameter. Thanks again. Arnab Sent: Thursday, August 31, 2017 at 3:10 PM From: "Alan DeKok" <aland@deployingradius.com> To: "FreeRadius users mailing list" <freeradius-users@lists.freeradius.org> Subject: Re: Running ntlm_auth as a connection pool On Aug 31, 2017, at 9:56 AM, Arnab Roy <arnabroy@mail.com> wrote:
Just a small additional question, so as it stands unless I completely re-do this whole piece , I am highly unlikely to get out of this
So as a temporary suggestion I have got plenty of CPU , DISK IO and Memory resources. Fr is barely using anything , any parameters I can change so FR uses the spare hardware resources to process the concurrent mschap requests ? Do LDAP lookups in FreeRADIUS. That will help a bit. The real limitation is Samba / Winbind. If their libraries allowed for ntlm_auth / rlm_winbind to set the winbind path, it would be simple. Since that path is hard-coded into their libraries, it's much more difficult. For v4, we're working on making it asynchronous. So you should be able to run many, many, instances of ntlm_auth without having the server wait for each one. Alan DeKok.
hole. - List info/subscribe/unsubscribe? See [1]http://www.freeradius.org/list/users.html References 1. http://www.freeradius.org/list/users.html
Hi All, So I took things apart today. I switched to using direct winbind on a test machine ...what a difference in performance you can't even compare the two method's performance. To check where such high cost of performance comes from I replaced ntlm auth with a simple shell script which just echoed back the NT key and another one directly calling the NTLM_AUTH and the results where same. So it seems like the cost of calling an external binary from inside the server is extremely high. So I went back to the friendly folks at samba and they kindly pointed me to the section of the code responsible for obtaining the winbindd path. I managed to recompile ntlm_auth and it works but it's pointless. The file in question in samba is under nsswitch/wb_common.c My question is does freeradius use the same code as samba for the client side communication. Can it be tweaked and compiled to behave similarly that would solve my problems for now. Any help is welcome. Many Thanks Arnab
On Sep 3, 2017, at 5:22 PM, Arnab Roy <arnabroy@mail.com> wrote:
So I took things apart today. I switched to using direct winbind on a test machine ...what a difference in performance you can't even compare the two method's performance.
That's to be expected.
To check where such high cost of performance comes from I replaced ntlm auth with a simple shell script which just echoed back the NT key and another one directly calling the NTLM_AUTH and the results where same. So it seems like the cost of calling an external binary from inside the server is extremely high.
It's always expensive to create sub-processes. Especially if the server is trying to do 1000's of authentications per second.
So I went back to the friendly folks at samba and they kindly pointed me to the section of the code responsible for obtaining the winbindd path. I managed to recompile ntlm_auth and it works but it's pointless.
Why?
The file in question in samba is under nsswitch/wb_common.c My question is does freeradius use the same code as samba for the client side communication.
FreeRADIUS uses the Samba libraries to talk to winbindd. If you recompile the Samba libraries, the server will use them.
Can it be tweaked and compiled to behave similarly that would solve my problems for now.
Just have it use the recompiled Samba libraries. Alan DeKok.
So I took things apart today. I switched to using direct winbind on a test machine ...what a difference in performance you can't even compare the two method's performance. That's to be expected. To check where such high cost of performance comes from I replaced ntlm auth with a simple shell script which just echoed back the NT key and another one directly calling the NTLM_AUTH and the results where same. So it seems like the cost of calling an external binary from inside the server is extremely high. It's always expensive to create sub-processes. Especially if the server is trying to do 1000's of authentications per second. So I went back to the friendly folks at samba and they kindly pointed me to the section of the code responsible for obtaining the winbindd path. I managed to recompile ntlm_auth and it works but it's pointless. Why? The file in question in samba is under nsswitch/wb_common.c My question is does freeradius use the same code as samba for the client side communication. FreeRADIUS uses the Samba libraries to talk to winbindd. If you recompile the Samba libraries, the server will use them. Can it be tweaked and compiled to behave similarly that would solve my problems for now. Just have it use the recompiled Samba libraries. Alan DeKok.
Hi Alan, Thanks for that, so the way I have got it work with NTLM_AUTH is by executing it like so : export WINBINDD_SOCKET=/var/cache/samba/domain && /usr/local/samba/bin/ntlm_auth --configfile=/etc/samba/domainsmb.conf ........ How can I pass a similar variable to the mschap module in FR so that it can pass it to samba, I am assuming this is not possible without any changes to FR code? Many Thanks Arnab Sent: Sunday, September 03, 2017 at 11:22 PM From: "Alan DeKok" <aland@deployingradius.com> To: "FreeRadius users mailing list" <freeradius-users@lists.freeradius.org> Subject: Re: Running ntlm_auth as a connection pool On Sep 3, 2017, at 5:22 PM, Arnab Roy <arnabroy@mail.com> wrote: - List info/subscribe/unsubscribe? See [1]http://www.freeradius.org/list/users.html References 1. http://www.freeradius.org/list/users.html
On Sep 4, 2017, at 11:38 AM, Arnab Roy <arnabroy@mail.com> wrote:
How can I pass a similar variable to the mschap module in FR so that it can pass it to samba, I am assuming this is not possible without any changes to FR code?
You can't. As I said, it requires changes to the Samba code. Alan DeKok.
Hi Alan, I thought so as well. I actually changed the samba code for this to work with ntlm_auth. Is the asynchronous execution module currently usable in v4 in a test environment? I remember you mentioning about a potential release in fall? Is their a list somewhere of what currently works and what doesn't? Many Thanks Arnab
On Sep 4, 2017, at 1:47 PM, Arnab Roy <arnabroy@mail.com> wrote:
I thought so as well. I actually changed the samba code for this to work with ntlm_auth.
If you want it to work with FreeRADIUS, you'll have to update the winbind API, too. You can't rely on setting an environment variable.
Is the asynchronous execution module currently usable in v4 in a test environment? I remember you mentioning about a potential release in fall? Is their a list somewhere of what currently works and what doesn't?
See the v4.0.x branch on github, and read raddb/README.md Alan DeKok.
participants (4)
-
Alan Buxey -
Alan DeKok -
Arnab Roy -
Matthew Newton