Hi, Three questions: * In the default SQL accounting schemas %S is used over the Event-Timestamp attribute included in the accounting packet. I guess this is because of the potential drift between NAS, and it makes correlation easier. Is this the real reason or is it just an omission ? * RFC 2869 Specifies the format of Event-Timestamp to be number of seconds since the Unix Epoch. Yet FR prints it as Event-Timestamp = "Apr 24 2008 20:06:52 BST". Is this FR's interpretation of the integer timestamp as a date string or is the NAS sending the timestamp as a string? * In accounting detail packets, a timestamp attribute is included. But I can't figure out how to access it as an attribute when the detail entries are read back into the server. Any ideas how to ? It would be better to use this in accounting queries than %S and there will be a delay between the packet arriving and the packet being inserted into the SQL db. Thanks, Arran -- Arran Cudbard-Bell (A.Cudbard-Bell@sussex.ac.uk) Authentication, Authorisation and Accounting Officer Infrastructure Services | ENG1 E1-1-08 University Of Sussex, Brighton EXT:01273 873900 | INT: 3900
Arran Cudbard-Bell wrote:
* In the default SQL accounting schemas %S is used over the Event-Timestamp attribute included in the accounting packet. I guess this is because of the potential drift between NAS, and it makes correlation easier. Is this the real reason or is it just an omission ?
Many NASes have broken clocks. Many, many, have broken clocks.
* RFC 2869 Specifies the format of Event-Timestamp to be number of seconds since the Unix Epoch. Yet FR prints it as Event-Timestamp = "Apr 24 2008 20:06:52 BST". Is this FR's interpretation of the integer timestamp as a date string or is the NAS sending the timestamp as a string?
It's being *printed* as a string. The contents of it in the packets are always 32-bit integers. It may be worth adding some logic to the server to double-check for "bad" Event-Timestamps...
* In accounting detail packets, a timestamp attribute is included. But I can't figure out how to access it as an attribute when the detail entries are read back into the server. Any ideas how to ?
Hmm... you can't. It may be useful to add it as something like Packet-Original-Timestamp, to distinguish it from Event-Timestamp. That's not hard to do.
It would be better to use this in accounting queries than %S and there will be a delay between the packet arriving and the packet being inserted into the SQL db.
Yes. But the server adds Acct-Delay-Time to the accounting packet, with exactly that time difference. But I see what you mean... Alan DeKok.
Alan DeKok wrote:
Arran Cudbard-Bell wrote:
* In the default SQL accounting schemas %S is used over the Event-Timestamp attribute included in the accounting packet. I guess this is because of the potential drift between NAS, and it makes correlation easier. Is this the real reason or is it just an omission ?
Many NASes have broken clocks. Many, many, have broken clocks.
* RFC 2869 Specifies the format of Event-Timestamp to be number of seconds since the Unix Epoch. Yet FR prints it as Event-Timestamp = "Apr 24 2008 20:06:52 BST". Is this FR's interpretation of the integer timestamp as a date string or is the NAS sending the timestamp as a string?
It's being *printed* as a string. The contents of it in the packets are always 32-bit integers.
Ok and it's expanded to the string form with the double quotation marks? why ?
It may be worth adding some logic to the server to double-check for "bad" Event-Timestamps...
* In accounting detail packets, a timestamp attribute is included. But I can't figure out how to access it as an attribute when the detail entries are read back into the server. Any ideas how to ?
Hmm... you can't. It may be useful to add it as something like Packet-Original-Timestamp, to distinguish it from Event-Timestamp.
That's not hard to do.
Indeed, I did something in unlang, but it'd be nice to have it in the server core. Then I can update the SQL queries with %{%{Packet-Original-Timestamp}:-%S} and it should all just work.
It would be better to use this in accounting queries than %S and there will be a delay between the packet arriving and the packet being inserted into the SQL db.
Yes. But the server adds Acct-Delay-Time to the accounting packet, with exactly that time difference.
Hmmm, the Acct Start Time and Acct Start Time are written as the packet is inserted into the database... are you saying if I subtracted Acct-Delay-Time from %S i'd get the equivalent to Packet-Original-Timestamp? Another thing I noticed recently: For file based buffers the server takes the detail file moves it to detail.work, processes all entries in the work file then repeats the process. On one of our servers I made a typo when recreating the symbolic link to start the detail reading server, I didn't notice the error for a number of days, by which time the detail file was ~400mb. Our servers are restarted nightly and the rate of inserts is so slow that the server can't get through 400mb of detail file in under 24hrs. So when it's restarted the whole process starts again. It's not a huge problem, as accounting data isn't massively important to us, but possibly putting an upper limit on the .work file might be useful. Arran
But I see what you mean...
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Arran Cudbard-Bell wrote:
Ok and it's expanded to the string form with the double quotation marks? why ?
Bug. Some things have extra quotation marks. This is fix in 2.0.3, or maybe CVS.
Indeed, I did something in unlang, but it'd be nice to have it in the server core. Then I can update the SQL queries with %{%{Packet-Original-Timestamp}:-%S} and it should all just work.
Done.
Another thing I noticed recently: For file based buffers the server takes the detail file moves it to detail.work, processes all entries in the work file then repeats the process.
Yes. It has to do that for a number of reasons.
On one of our servers I made a typo when recreating the symbolic link to start the detail reading server, I didn't notice the error for a number of days, by which time the detail file was ~400mb. Our servers are restarted nightly and the rate of inserts is so slow that the server can't get through 400mb of detail file in under 24hrs. So when it's restarted the whole process starts again.
It's not a huge problem, as accounting data isn't massively important to us, but possibly putting an upper limit on the .work file might be useful.
It can't, because it's just a renamed "detail" file. If the detail file is 400M, so is detail.work. The larger issue is that it's hard to keep track of which parts of the "detail.work" file have been read && responded to. I've had a few ideas, but nothing that really makes sense. I'll play with some things to see if I can get that last piece fixed. if you're OK with the "detail.work" file being written to, there may be a solution. Or, update it so that it reads *all* of the detail files in a directory. That way, the process writing the detail files can write them every hour, day, etc. Alan DeKok.
Alan DeKok wrote:
Arran Cudbard-Bell wrote:
Ok and it's expanded to the string form with the double quotation marks? why ?
Bug. Some things have extra quotation marks. This is fix in 2.0.3, or maybe CVS.
Hmm running 2.0.3 must be CVS.
Indeed, I did something in unlang, but it'd be nice to have it in the server core. Then I can update the SQL queries with %{%{Packet-Original-Timestamp}:-%S} and it should all just work.
Done.
Thanks. I didn't realise that the server updated the acctdelay stuff too... woo so many options... Did you have time to add the module return codes for authentication success / failure messages ?
Another thing I noticed recently: For file based buffers the server takes the detail file moves it to detail.work, processes all entries in the work file then repeats the process.
Yes. It has to do that for a number of reasons.
On one of our servers I made a typo when recreating the symbolic link to start the detail reading server, I didn't notice the error for a number of days, by which time the detail file was ~400mb. Our servers are restarted nightly and the rate of inserts is so slow that the server can't get through 400mb of detail file in under 24hrs. So when it's restarted the whole process starts again.
It's not a huge problem, as accounting data isn't massively important to us, but possibly putting an upper limit on the .work file might be useful.
It can't, because it's just a renamed "detail" file. If the detail file is 400M, so is detail.work.
Yes I was talking about taking a slice of the detail file, and writing it to the work file, but that's a lot more work that just moving the detail file (in terms of disk I/O).
Or, update it so that it reads *all* of the detail files in a directory. That way, the process writing the detail files can write them every hour, day, etc.
Yep that seems like the most sensible/ flexible solution. So you just specify a directory in the listen section for it to search for detail files in. Thanks, Arran
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Arran Cudbard-Bell wrote:
Hmm running 2.0.3 must be CVS.
Yes.
Did you have time to add the module return codes for authentication success / failure messages ?
It should be there now.
Yep that seems like the most sensible/ flexible solution. So you just specify a directory in the listen section for it to search for detail files in.
Or just use file globbing. Alan DeKok.
participants (2)
-
Alan DeKok -
Arran Cudbard-Bell