Hello, I am curious to know what people use, if anything, to collect metrics about the various database calls that FreeRADIUS does. E.g. How fast is that LDAP server, or that SQL database? Sure you can measure some of that from the database side itself, and radsniff can give you some idea of the latency for your requests, but what do you do if you make multiple calls to a series of servers, some of which you may not have access to (cough.. AD server )? I have in the past written a wrapper for ntlm_auth that was logging the times (and sent them to a statsd server) but that process is now going away -- which is a good thing -- and that case does not cover metrics for FR's own modules. Ideally I'd like to be able to point to a slowdown in one submodule (redis, ldap, sql, winbind etc.) to explain why the total latency has increased. Short of adding metrics to each module I want them from (something I have played with using a statsd client library) is there a recommended way of getting that information? Especially if I want historical data to be able to compare against for anomaly detection or capacity planning? Any suggestions are welcome and appreciated. -- Louis Munro lmunro@inverse.ca <mailto:lmunro@inverse.ca> :: www.inverse.ca <http://www.inverse.ca/> +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 Inverse inc. :: Leaders behind SOGo (www.sogo.nu <http://www.sogo.nu/>) and PacketFence (www.packetfence.org <http://www.packetfence.org/>)
On Sep 15, 2016, at 1:45 PM, Louis Munro <lmunro@inverse.ca> wrote:
I am curious to know what people use, if anything, to collect metrics about the various database calls that FreeRADIUS does. E.g. How fast is that LDAP server, or that SQL database?
Not much. The problem is that until recently, collecting lots of timing metrics was expensive. i.e. it noticeably slowed systems down. Now.. it's 2016. You can always buy more CPU power for small amounts of money.
Sure you can measure some of that from the database side itself, and radsniff can give you some idea of the latency for your requests, but what do you do if you make multiple calls to a series of servers, some of which you may not have access to (cough.. AD server )?
For 4.0, we'll be adding tons of metrics to the server. They'll likely be an optional part of the build, but they will definitely be available.
Short of adding metrics to each module I want them from (something I have played with using a statsd client library) is there a recommended way of getting that information? Especially if I want historical data to be able to compare against for anomaly detection or capacity planning?
We'll be adding metrics to each individual module, along with a way to export them. Alan DeKok.
On Sep 16, 2016, at 12:34 PM, Alan DeKok <aland@deployingradius.com> wrote:
We'll be adding metrics to each individual module, along with a way to export them.
That is great news. I guess it may be the end of questions to which the answer always is “fix your database”. But then again, people would have to actually look at the metrics for that to be true... Best regards, -- Louis Munro lmunro@inverse.ca <mailto:lmunro@inverse.ca> :: www.inverse.ca <http://www.inverse.ca/> +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 Inverse inc. :: Leaders behind SOGo (www.sogo.nu <http://www.sogo.nu/>) and PacketFence (www.packetfence.org <http://www.packetfence.org/>)
Hi,
On Sep 16, 2016, at 12:34 PM, Alan DeKok <aland@deployingradius.com> wrote:
We'll be adding metrics to each individual module, along with a way to export them.
That is great news. I guess it may be the end of questions to which the answer always is “fix your database”.
But then again, people would have to actually look at the metrics for that to be true...
you can get timing info witha few more x's added to radiusd -X - I've seen some scripts around that will take those values and tell you how long each module took.... obviously, a single radiusd -X isnt really going to be stressing DB etc...but if you ran hot radiusd and alongside, ran another instance connecting to same systems etc but listening on different port with full debug with times you could geta good indicitive guide of what each module is doing and trend view of where issue might be. you can also do the same using eg tcpdump and network performance analysis on the response times from remote connections.... built in metrics with reporting methods is definately the cleaner/direct and less hacky way alan
On Sep 16, 2016, at 1:26 PM, A.L.M.Buxey@lboro.ac.uk wrote:
built in metrics with reporting methods is definately the cleaner/direct and less hacky way
For sure. Especially if there are ways to get that data into some kind of time-series database, à la graphite. I will keep an eye on the FR evolution. I’d be willing to contribute in my limited abilities if it comes to that. -- Louis Munro lmunro@inverse.ca <mailto:lmunro@inverse.ca> :: www.inverse.ca <http://www.inverse.ca/> +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 Inverse inc. :: Leaders behind SOGo (www.sogo.nu <http://www.sogo.nu/>) and PacketFence (www.packetfence.org <http://www.packetfence.org/>)
participants (3)
-
A.L.M.Buxey＠lboro.ac.uk -
Alan DeKok -
Louis Munro