(apologies if I've broken the threading - I shouldn't have subscribed using digest mode, now corrected) On Tue, Dec 23, 2014 at 11:05:23AM -0500, Alan DeKok wrote:
The reason is that attributes can have embedded \n in them, either as binary or a as a string "\n". Using those attributes in the "linelog" module would then cause spurious new lines, and break your assumptions about what the log files mean.
Very good point. Because the set of attributes we happen to use in the log entry don't include end user inputted data and the source of those packets we are safe from unexpected expansion for now, but our future intended use and as a general use case this is certainly what's known as a bad idea. Time to rethink.
Not with 3.0.6. We can take a look at correcting it for the next release. We’ve made other fixes in 3.0 which made it easier to separate the "literal" strings passed in from the config files, and the “end user” strings taken from a RADIUS packet. We can have the linelog module convert \n -> CR for literal strings you enter, but not do that expansion for strings taken from RADIUS packets.
Certainly that would help, but I should probably the end result we are trying to achieve in case linelog was completely the wrong approach. It sound like we are doing much the same as Matthew Newton, in addition to performing some normal authentication/authorization/accounting we also send information from those to another radius server as accounting packet - in some cases this is for additional auditing, sometimes as it enables other functionality - using the linelog module we could write out a file in the standard detail format but with just the attributes we wanted (and the type as accounting even if it had originally been an access request), and use a detail listener to pick that up and send them on. Is there some other way to do this that I've missed? I suspect the reason in our case that accounting packets are always used is just because historically it was more convenient to do so. Given we own the systems that the packets get sent onto it's possible that we may be able to change those systems to handle access request or response packets as well as accounting, and modify the source systems to use a detail writer/read and Proxy-to-realm setup rather linelog Paul ------------------------------ Message: 2 Date: Tue, 23 Dec 2014 11:05:23 -0500 From: Alan DeKok <aland@deployingradius.com> To: FreeRadius users mailing list <freeradius-users@lists.freeradius.org> Subject: Re: Expansion of \t and \n in message with linelong module and correct_escapes = true with v3.0.6 Message-ID: <E5F57FCB-CC38-4386-B922-3F36D37A3F88@deployingradius.com> Content-Type: text/plain; charset=windows-1252 On Dec 23, 2014, at 10:38 AM, paul.moser@bt.com wrote:
I have a 3.0.3 FR instance to which some additional logging has been added using the linelog module. The log messages that are written are multi-line and tab indented with \n and \t used in the format option in the linelog modules configuration file.
That?s not really the intended use of the ?linelog? module. It logs lines of text.. not multiple lines.
I have also tried migrating another, older (v2.x) FR instance to v3.0.6. I started with a vanilla 3.0.6 configuration and successfully slowly modified that to give the same behaviour as the v2.x version. However when I finally attempted to add the additional logging that had been added to the 3.0.3 instance the log statements appearing in the log file are not multi-line or tab indented, neither the newline \n nor the tab \t have been expanded, \n and \t appear in the log file where I'd expected newlines and tabs to appear.
I note in the changelog that for v3.0.5 correct_escapes was introduced and in a vanilla 3.0.6 configuration this is set to true
It?s correct, and it creates many other simplifications of the source tree.
Taking my 3.0.3 configuration as is and running it against a 3.0.6 server it behaves just as it does with a 3.0.3 server - expanding \t and \n . However if I modify the configuration to set correct_escapes to true then literal \t and \n appear in the log file rather than tabs and newlines.
Yes. That?s really the correct behaviour. The reason is that attributes can have embedded \n in them, either as binary or a as a string ?\n?. Using those attributes in the ?linelog? module would then cause spurious new lines, and break your assumptions about what the log files mean.
With 3.0.6, with correct_escapes=true, is it possible to use backslash escape sequences, or some equivalent to format linelog log messages?
Not with 3.0.6. We can take a look at correcting it for the next release. We?ve made other fixes in 3.0 which made it easier to separate the ?literal? strings passed in from the config files, and the ?end user? strings taken from a RADIUS packet. We can have the linelog module convert \n -> CR for literal strings you enter, but not do that expansion for strings taken from RADIUS packets.
It maybe that the expansion of \n and \t in 3.0.3 was just undocumented behaviour that we stumbled upon and utilised rather than expected behaviour, the fact that the module is called line (singular) log may indicated we should not expect it to write multi-line log statements.
Both.
I realise that I could just set correct_escapes=false in my 3.0.6 instances, I have briefly tried this however much of the default configuration now appears to assume the correct escapes and I'd have to unwind all those changes. I suspect if I did this I'd just be storing up trouble for myself some point in the future.
I?d recommend keeping the new behaviour. It?s *much* more intuitive. Alan DeKok. ------------------------------ Message: 3 Date: Tue, 23 Dec 2014 23:27:27 +0000 From: Matthew Newton <mcn4@leicester.ac.uk> To: FreeRadius users mailing list <freeradius-users@lists.freeradius.org> Subject: Re: Expansion of \t and \n in message with linelong module and correct_escapes = true with v3.0.6 Message-ID: <20141223232727.GA19006@rootmail.cc.le.ac.uk> Content-Type: text/plain; charset=utf-8 On Tue, Dec 23, 2014 at 11:05:23AM -0500, Alan DeKok wrote:
That?s not really the intended use of the ?linelog? module. It logs lines of text.. not multiple lines.
Of course, "intended" and "what's useful" are two different things. We're also (ab-)using linelog to write out detail log format. These are then picked up by the detail reader and fed (as accounting packets) across to a central RADIUS server for common logging. It means we can set exactly which attributes to forward. Because of the way the logging and detail reader works it is also reliable if the central server goes away for some reason. I guess the alternative would be to have an "include" option for the detail module, countering the "suppress" option. But that's only for our use case.
Not with 3.0.6. We can take a look at correcting it for the next release. We?ve made other fixes in 3.0 which made it easier to separate the ?literal? strings passed in from the config files, and the ?end user? strings taken from a RADIUS packet. We can have the linelog module convert \n -> CR for literal strings you enter, but not do that expansion for strings taken from RADIUS packets.
That would seem the more sensible option - expand \n, \t etc in the string first, then expand %{..}. But I get that string expansion is pretty tricky in the ordering it needs to be done to get a sane output for all cases. Cheers Matthew -- Matthew Newton, Ph.D. <mcn4@le.ac.uk> Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk> ------------------------------ Message: 4 Date: Tue, 23 Dec 2014 23:48:11 +0000 From: Alan Buxey <A.L.M.Buxey@lboro.ac.uk> To: FreeRadius users mailing list <freeradius-users@lists.freeradius.org>, Matthew Newton <mcn4@leicester.ac.uk> Subject: Re: Expansion of \t and \n in message with linelong module and correct_escapes = true with v3.0.6 Message-ID: <ABEC402F-CA0B-418A-A14F-C7F97692988B@lboro.ac.uk> Content-Type: text/plain; charset="utf-8" buffered_sql with extra attr_filter rule? alan -- Sent from my Android device with K-9 Mail. Please excuse my brevity.