Use of buffered-sql for logging auth data to db
FR 3.0.12 Is the use of the buffered-sql virtual host just for processing accounting data ? What about radpostauth data ? Just set up the buffered-sql config and I can see that a file called detail.work has been created .... with Accounting packets in it. Is this file supposed to change size as records get processed? currently just using a file called detail and not detail-<timestamp .....> A
On Mon, Oct 17, 2016 at 10:38:18AM +0100, Alex Sharaz wrote:
FR 3.0.12
Is the use of the buffered-sql virtual host just for processing accounting data ?
It's just a detail reader. So it reads packets from a file and processes them as if they had just arrived over the network.
What about radpostauth data ?
Don't see why not. As long as you are only logging and not expecting to send anything back to a NAS.
Just set up the buffered-sql config and I can see that a file called detail.work has been created .... with Accounting packets in it. Is this file supposed to change size as records get processed?
The detail writer (just rlm_detail) writes to a log file as it usually does - called "detail". The detail reader (in sites-enabled/buffered-sql) reads that log file and feeds the packets into (usually) sql. It renames the file to "detail.work" before reading so that the detail writer doesn't add to the end of the file while it's being processed. So you'll occasionally see both files.
currently just using a file called detail and not detail-<timestamp .....>
Yes. Will need to in this setup. If you want a timestamped file as well then just use another instance of rlm_detail. For general craziness... we currently use linelog to write out a file that looks like an Acct detail file, from an Auth packet. Then a detail reader to reprocess that file and relay it over to another FR server, where it gets logged (rather than using sql as in the buffered-sql config). Probably wouldn't do that now, but shows what you can do with it. It's been rock solid for ~5 years. Matthew -- Matthew Newton, Ph.D. <mcn4@leicester.ac.uk> Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk>
It renames the file to "detail.work" before reading so that the detail writer doesn't add to the end of the file while it's being processed. So you'll occasionally see both files.
Ermmm always seeing both files. detail.work timestamp changing, file never disappears or changes size. Guess that might be the load-factor setting value? A On 17 October 2016 at 10:52, Matthew Newton <mcn4@leicester.ac.uk> wrote:
On Mon, Oct 17, 2016 at 10:38:18AM +0100, Alex Sharaz wrote:
FR 3.0.12
Is the use of the buffered-sql virtual host just for processing accounting data ?
It's just a detail reader. So it reads packets from a file and processes them as if they had just arrived over the network.
What about radpostauth data ?
Don't see why not. As long as you are only logging and not expecting to send anything back to a NAS.
Just set up the buffered-sql config and I can see that a file called detail.work has been created .... with Accounting packets in it. Is this file supposed to change size as records get processed?
The detail writer (just rlm_detail) writes to a log file as it usually does - called "detail".
The detail reader (in sites-enabled/buffered-sql) reads that log file and feeds the packets into (usually) sql.
It renames the file to "detail.work" before reading so that the detail writer doesn't add to the end of the file while it's being processed. So you'll occasionally see both files.
currently just using a file called detail and not detail-<timestamp .....>
Yes. Will need to in this setup. If you want a timestamped file as well then just use another instance of rlm_detail.
For general craziness... we currently use linelog to write out a file that looks like an Acct detail file, from an Auth packet. Then a detail reader to reprocess that file and relay it over to another FR server, where it gets logged (rather than using sql as in the buffered-sql config). Probably wouldn't do that now, but shows what you can do with it. It's been rock solid for ~5 years.
Matthew
-- Matthew Newton, Ph.D. <mcn4@leicester.ac.uk>
Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom
For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk> - List info/subscribe/unsubscribe? See http://www.freeradius.org/ list/users.html
On Mon, Oct 17, 2016 at 11:10:39AM +0100, Alex Sharaz wrote:
It renames the file to "detail.work" before reading so that the detail writer doesn't add to the end of the file while it's being processed. So you'll occasionally see both files.
Ermmm always seeing both files. detail.work timestamp changing, file never disappears or changes size.
Maybe behaviour changed from v2 to v3, here (these servers still on v2 :-( ) the .work file appears and vanishes every second or two. But sounds like it's doing the right thing. Is it actually broken, or are the records making their way into the database? If not, try adding e.g. a new "detail" instance in the buffered-sql virtual server to see what it's reading, or use a debug_request and run in debug mode to make sure it's picking up the data. Matthew -- Matthew Newton, Ph.D. <mcn4@leicester.ac.uk> Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk>
o.k. think things have caught up with themselves, detail file now doing what I'd expect A On 17 October 2016 at 11:16, Matthew Newton <mcn4@leicester.ac.uk> wrote:
On Mon, Oct 17, 2016 at 11:10:39AM +0100, Alex Sharaz wrote:
It renames the file to "detail.work" before reading so that the detail writer doesn't add to the end of the file while it's being processed. So you'll occasionally see both files.
Ermmm always seeing both files. detail.work timestamp changing, file never disappears or changes size.
Maybe behaviour changed from v2 to v3, here (these servers still on v2 :-( ) the .work file appears and vanishes every second or two.
But sounds like it's doing the right thing. Is it actually broken, or are the records making their way into the database?
If not, try adding e.g. a new "detail" instance in the buffered-sql virtual server to see what it's reading, or use a debug_request and run in debug mode to make sure it's picking up the data.
Matthew
-- Matthew Newton, Ph.D. <mcn4@leicester.ac.uk>
Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom
For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk> - List info/subscribe/unsubscribe? See http://www.freeradius.org/ list/users.html
Question - can you use the FR 3.0.x control interface to get some form of status of any db pools being used, or even any errors generated when writing to a DB ? Failing that, guess I could grep radius.log looking for specific text strings relating to Error: rlm_sql (sql): Last connection attempt failed, waiting 30 seconds before retrying A On 17 October 2016 at 11:33, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
o.k. think things have caught up with themselves, detail file now doing what I'd expect A
On 17 October 2016 at 11:16, Matthew Newton <mcn4@leicester.ac.uk> wrote:
On Mon, Oct 17, 2016 at 11:10:39AM +0100, Alex Sharaz wrote:
It renames the file to "detail.work" before reading so that the detail writer doesn't add to the end of the file while it's being processed. So you'll occasionally see both files.
Ermmm always seeing both files. detail.work timestamp changing, file never disappears or changes size.
Maybe behaviour changed from v2 to v3, here (these servers still on v2 :-( ) the .work file appears and vanishes every second or two.
But sounds like it's doing the right thing. Is it actually broken, or are the records making their way into the database?
If not, try adding e.g. a new "detail" instance in the buffered-sql virtual server to see what it's reading, or use a debug_request and run in debug mode to make sure it's picking up the data.
Matthew
-- Matthew Newton, Ph.D. <mcn4@leicester.ac.uk>
Systems Specialist, Infrastructure Services, I.T. Services, University of Leicester, Leicester LE1 7RH, United Kingdom
For IT help contact helpdesk extn. 2253, <ithelp@le.ac.uk> - List info/subscribe/unsubscribe? See http://www.freeradius.org/list /users.html
On Oct 17, 2016, at 11:46 AM, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
Question - can you use the FR 3.0.x control interface to get some form of status of any db pools being used, or even any errors generated when writing to a DB ?
radmin> help Will print out all of the commands that are available.
Failing that, guess I could grep radius.log looking for specific text strings relating to Error: rlm_sql (sql): Last connection attempt failed, waiting 30 seconds before retrying
Yes. We're looking at fixing this all for 4.0... but that may require some re-design. Alan DeKok.
participants (3)
-
Alan DeKok -
Alex Sharaz -
Matthew Newton