Would a patch adding dtrace/systemtap-compatible source probes be looked upon favourably? I'm fiddling with the debuginfo-based stap stuff to try and get a histogram of our ntlm_auth timings, but it strikes me that the source-based stuff could be very useful indeed; in particular appropriate DTRACE_PROBEx statements could let us time individual requests from receive to reply, with minimal performance overhead. See: https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps
Phil Mayers wrote:
Would a patch adding dtrace/systemtap-compatible source probes be looked upon favourably?
Yes.
I'm fiddling with the debuginfo-based stap stuff to try and get a histogram of our ntlm_auth timings, but it strikes me that the source-based stuff could be very useful indeed; in particular appropriate DTRACE_PROBEx statements could let us time individual requests from receive to reply, with minimal performance overhead.
That would be great. I previously had patches to do this without dtrace, but the patches were a bit invasive, and they had performance impact. Alan DeKok.
Phil Mayers wrote:
https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps
Looking at the page some more, here's my $0.02. - ignore the configure checks for sys/std.h, unless you're a masochist. Arran or I will get that going. For testing, you can just manually add #define entries to src/include/autoconf.h - I'd put the changes into src/include/log.c: /* * Catch brokenness */ #ifdef WITH_DTRACE_PROBES #ifndef HAVE_SYS_STD_H #undef WITH_DTRACE_PROBES #endif ... // typical pattern #ifdef HAVE_SYS_SDT_H #include <sys/sdt.h> #endif // create empty defines if they don't exist #ifndef WITH_DTRACE_PROBES #define DTRACE_PROBE(_x, _y) #define ... // anything else here to avoid #ifdef's in the code #endif For the actual probes, use: DTRACE_PROBE(progname, ...) Because "progname" is a global which is the base name of the binary. e.g. /usr/sbin/radiusd -> radiusd. That lets it automatically change the probe names if we want to run the server as "radrelay", for example. And then sprinkle the probes everywhere. I'd suggest src/main/modcall.c for the various modules. And src/main/threads.c for the queue / dequeue work. And src/main/process.c for the various state machine handling. I can then use dtrace on my Mac to debug performance issues. Nice! Alan DeKok.
With the time values for modules then exposed to the admin via radmin alan -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On 15/10/13 13:33, Alan Buxey wrote:
With the time values for modules then exposed to the admin via radmin
That's not how systemtap/dtrace work; the process itself doesn't "see" the probe results, only the tracing app (and only when tracing is on). Instrumenting in-process is rather harder IMO as the server has to keep track of the data, process it and changing the way it presents the data (e.g. EWMA versus histogram versus...) requires source-code changes.
participants (3)
-
Alan Buxey -
Alan DeKok -
Phil Mayers