Hi,
Have a look at this:
hm, I had a look at this HOWTO, and the FreeRADIUS bits appear a bit odd. He adds a new column to radacct, and fills the gigawords attribute into this seperate column. This does not give a coherent output: you are likely interested in one single number (i.e. the number of input octets), and not in doing binary arithmetic yourself on two seperate columns of the table. I have a few simple suggestions: - (mysql) modify Schema AcctInputOctets to be BIGINT(20) instead of BIGINT(12) [this will make the integer big enough to hold any 64-Bit value] - modify the default query in sql.conf to do the binary arithmetic (merging Gigaword << 32 | inputOctets) into a single number and storing that in radacct. At least for mySQL, this kind of arithmetic in an update or insert query is rather simple: replace AcctInputOctets = '%{Acct-Input-Octets}', with AcctInputOctets = '%{Acct-Input-Gigawords}' << 32 | '%{Acct-Input-Octets}, and you're done: it gives us 64-bit counter handling for SQL backends with no pain (and Output-octets, Input-Packets, Output-Packets equivalently) Something for 2.0? Greetings, Stefan -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
Stefan Winter wrote: ...
At least for mySQL, this kind of arithmetic in an update or insert query is ... and you're done: it gives us 64-bit counter handling for SQL backends with no pain (and Output-octets, Input-Packets, Output-Packets equivalently)
Something for 2.0?
Do you have a patch we can apply? Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
Do you have a patch we can apply?
Hm, I think I can come up with one soon. It's not modifying any real code, so it shouldn't need much testing, but I'm limited in even that tiny bit of testing - I'd need to fake packets to test behaviour, we don't exceed 4G in counters very often. Let's see... Stefan -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
On Tue 29 May 2007, Stefan Winter wrote:
Hi,
Have a look at this:
hm, I had a look at this HOWTO, and the FreeRADIUS bits appear a bit odd. He adds a new column to radacct, and fills the gigawords attribute into this seperate column. This does not give a coherent output: you are likely interested in one single number (i.e. the number of input octets), and not in doing binary arithmetic yourself on two seperate columns of the table.
I have a few simple suggestions:
- (mysql) modify Schema AcctInputOctets to be BIGINT(20) instead of BIGINT(12) [this will make the integer big enough to hold any 64-Bit value] - modify the default query in sql.conf to do the binary arithmetic (merging Gigaword << 32 | inputOctets) into a single number and storing that in radacct.
At least for mySQL, this kind of arithmetic in an update or insert query is rather simple:
replace
AcctInputOctets = '%{Acct-Input-Octets}',
with
AcctInputOctets = '%{Acct-Input-Gigawords}' << 32 | '%{Acct-Input-Octets},
and you're done: it gives us 64-bit counter handling for SQL backends with no pain (and Output-octets, Input-Packets, Output-Packets equivalently)
Erm. This has been part of the default Postgresql queries for eons. Can someone explain to me why the same doesn't "just work" with mysql. See: accounting_update_query = "UPDATE ${acct_table1} \ SET FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, \ AcctSessionTime = '%{Acct-Session-Time}', \ AcctInputOctets = (('%{Acct-Input-Gigawords:-0}'::bigint << 32) + '%{Acct-Input-Octets:-0}'::bigint), \ AcctOutputOctets = (('%{Acct-Output-Gigawords:-0}'::bigint << 32) + '%{Acct-Output-Octets:-0}'::bigint) \ WHERE AcctSessionId = '%{Acct-Session-Id}' AND UserName = '%{SQL-User-Name}' \ AND NASIPAddress= '%{NAS-IP-Address}' AND AcctStopTime IS NULL" Cheers -- Peter Nixon http://www.peternixon.net/ PGP Key: http://www.peternixon.net/public.asc
Hi Peter, maybe just noone ever bothered to update mysql conf file? I'm working on a patch, should be delivered in hours. This *might* even go into 1.1.x, it would just confuse people who are used to getting only 32 bits in eval scripts because it never used to work with 64 bits on mysql. [OT] BTW, you are one of the very few people on fr- lists that get caught by our spam filter. Seems like some blacklists don't like your domain, for whatever reason (see below). Stefan X-Spam-Status: Yes, score=5.9 required=5.0 tests=RAZOR2_CF_RANGE_51_100, RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_SORBS_WEB autolearn=disabled version=3.2.0 -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
Hi, something's attached, patch against current CVS HEAD. Inspired by Peter's postgresql snippet (:-0 is really necessary, the query will otherwise be f*cked up if not both attribs are present). I use this setup in our prod env now, and am shocked about our user's real network usage, now that I see more than the 4GB ceiling. I.e. it appears to work fine. While I was at it, I pretty-formatted one of the queries. I think these queries all-in-one-line are HORRIBLE to read or diff against (I have an x resolution of 3200 pixels and STILL there are multiple line-wraps in some of the queries!). Don't know if you like the style, if you do, I can also create a patch that beautifies all the other queries. Stefan -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
Stefan Winter wrote:
something's attached, patch against current CVS HEAD.
$ cvs diff -u dir1/file1 dir2/file2 > patch Please. The patch doesn't include directory prefixes, and isn't in a format we can apply. Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
$ cvs diff -u dir1/file1 dir2/file2 > patch
Please. The patch doesn't include directory prefixes, and isn't in a format we can apply.
Sorry. Attached a proper one. Note that except for the Gigawords thing, all other changes are whitespace changes. No tabs, just quite a few \ linebreaks and filler spaces to hopefully make stuff easier read- and diffable. This version is not tested at all because today morning's CVS build doesn't compile. Will do tests as soon as I get a working "make install". Three compiling issues so far: - fresh this morning: gmake[4]: Entering directory `/home/swinter/Development/radiusd/src/lib' /home/swinter/Development/radiusd/libtool --mode=compile gcc -g -O2 -D_LIBRADIUS -I/home/swinter/Development/radiusd/src -c print.c gcc -g -O2 -D_LIBRADIUS -I/home/swinter/Development/radiusd/src -c print.c -fPIC -DPIC -o .libs/print.o print.c: In function 'vp_prints_value': print.c:220: error: 'VALUE_PAIR' has no member named 'vp_ether' - I ran into this sed -i thing on "make install" yesterday - NEW? make -j 3 fails to pick up a dependency of sources and breaks during rlm_eap_ttls. A second make -j 3 try then compiles successfully, so it's probably just the build system not getting compile orders correct. Unfortunately I have *no* clue how to fix build systems. Will post make output when point 1 above is fixed and I can get to this point at all :-) Greetings, Stefan -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
Stefan Winter wrote:
Sorry. Attached a proper one. Note that except for the Gigawords thing, all other changes are whitespace changes. No tabs, just quite a few \ linebreaks and filler spaces to hopefully make stuff easier read- and diffable.
Thanks.
This version is not tested at all because today morning's CVS build doesn't compile. Will do tests as soon as I get a working "make install".
I've committed the fix. I have 3-4 things on the go in one sandbox, so I often miss committing something when I'm committing a number of files.
Three compiling issues so far:
- fresh this morning: ...
Fixed.
- I ran into this sed -i thing on "make install" yesterday
- NEW? make -j 3 fails to pick up a dependency of sources and breaks during rlm_eap_ttls. A second make -j 3 try then compiles successfully, so it's probably just the build system not getting compile orders correct.
I've seen that for a long time, too. It has to do with the recursive nature of the Makefiles, which means Make doesn't know about inter-dependencies. Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
- NEW? make -j 3 fails to pick up a dependency of sources and breaks during rlm_eap_ttls. A second make -j 3 try then compiles successfully, so it's probably just the build system not getting compile orders correct.
I've seen that for a long time, too. It has to do with the recursive nature of the Makefiles, which means Make doesn't know about inter-dependencies.
Hm, I see. I tried to dig out my old KDE-dev Makefile knowledge, which isn't much. I was able to slightly modify the Makefile in rlm_eap/types so that tls comes before everything else, i.e. before ttls and peap, which depend on it. That part of serialisation worked fine, but I got stuck somewhere else: rlm_eap main dir. There, the subdir libeap must be compiled before the directory itself. The include of ../rules.mak appears to break this. I'm pretty confident I could solve this if I put all the sources that are in the rlm_eap dir itself into an own subdir ("main"?) and then declare that subdir libeap gets priority over subdir main. The way it is now can't do that AFAICT since I don't dare touching rules.mak. Before I locally work on relocating the files into a subdir, do you think this is a good way forward? If the answer is no, i'll save the time digging into this. I think that the speedup due to parallelisation would be quite nice, since almost all modules are mostly independent. I guess all except eap and sql. Greetings, Stefan -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
Stefan Winter wrote:
Hm, I see. I tried to dig out my old KDE-dev Makefile knowledge, which isn't much. I was able to slightly modify the Makefile in rlm_eap/types so that tls comes before everything else, i.e. before ttls and peap, which depend on it.
Ok...
That part of serialisation worked fine, but I got stuck somewhere else: rlm_eap main dir. There, the subdir libeap must be compiled before the directory itself. The include of ../rules.mak appears to break this.
That's more a side effect of invoking Make as a child process, inside of "for" loops in shell script. That will be difficult to fix.
I think that the speedup due to parallelisation would be quite nice, since almost all modules are mostly independent. I guess all except eap and sql.
The way to fix it cleanly is non-recursive make: http://www.xs4all.nl/~evbergen/nonrecursive-make.html But that requires major changes to the Makefile. Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
Hi,
That's more a side effect of invoking Make as a child process, inside of "for" loops in shell script. That will be difficult to fix.
The way to fix it cleanly is non-recursive make:
http://www.xs4all.nl/~evbergen/nonrecursive-make.html
But that requires major changes to the Makefile.
played around with it a bit. I have a version running -j 3 cleanly on my system now. Compilation time: make -j 3 real 0m48.303s user 0m50.199s sys 0m11.493s make real 1m5.771s user 0m49.847s sys 0m11.149s The idea was to - move rlm_eap/*.{c|h}, configure.in to its own new subdir ("main") - change all references to these header files in libeap/ and types/* - change Makefiles in types and libeap to reference ../main/ - change some .c files that included ../../eap.h - the ugly part: new Makefile for rlm_eap, and new one for main/ - delete Makefile.in and configure* in rlm_eap These two Makefile in the end were basically stolen from other subdirs and halfways hacked to make it work. I'm sure they are ugly and probably don't work everywhere. They will need someone knowledgable looking into it. So, if this is of interest for anyone, I'll open a bug report with the diff against existing files, a tar with the new "main/" subdir and the Makefile for rlm_eap. The basic trick lies in rlm_eap Makefile: target "common" depends on new target "libeap-dep", which just descends into subdir libeap. For concurrency, the dependency means that the recursion via common (which includes the main/ dir) will only start after libeap-dep is finished. I think it's worth noting that rlm_eap was the *only* neuralgic point in the whole build process - the idea about types/ttls being dependend on tls proved to either be not true or was not a problem on my specific workstation's timing. Greetings, Stefan -- Stefan WINTER Stiftung RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche Ingenieur Forschung & Entwicklung 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg E-Mail: stefan.winter@restena.lu Tel.: +352 424409-1 http://www.restena.lu Fax: +352 422473
On Tue 29 May 2007, Stefan Winter wrote:
Hi,
something's attached, patch against current CVS HEAD. Inspired by Peter's postgresql snippet (:-0 is really necessary, the query will otherwise be f*cked up if not both attribs are present).
Yes. They ARE necessary ;-)
I use this setup in our prod env now, and am shocked about our user's real network usage, now that I see more than the 4GB ceiling. I.e. it appears to work fine.
heh.
While I was at it, I pretty-formatted one of the queries. I think these queries all-in-one-line are HORRIBLE to read or diff against (I have an x resolution of 3200 pixels and STILL there are multiple line-wraps in some of the queries!). Don't know if you like the style, if you do, I can also create a patch that beautifies all the other queries.
I suggest you "beautify" them in a similar method to the postgresql queries. Please make sure to use spaces and not tabs on the multi-line indents as tabs come out as garbage in debug logs which means you can't just copy and paste a query from a debug log to the sql command prompt to see how it works/why its failing. Also, please resubmit in a decent diff format as per the docs :-) Cheers -- Peter Nixon http://www.peternixon.net/ PGP Key: http://www.peternixon.net/public.asc
participants (3)
-
Alan Dekok -
Peter Nixon -
Stefan Winter