FreeRadius 3.0.4

[Fajar A. Nugraha]

On Fri, Nov 20, 2015 at 1:48 PM, Oscar Jofre <oscar at jofre.com> wrote:

> Thanks for answer Fajar:
>
> >>
> >> The database MariaDB 10.x is running on a different server with more
> >> RAM and CPU.
> >>
> >>
> >Do you have a monitoring system setup on THAT server? In particular,
> something to monitor its disk IOPS usage?
> >If you don't, a realtime "iostat -mx 3" on that server will suffice
> during tests.
> >
>
> Yes I setup this tool
> https://github.com/major/MySQLTuner-perl/tarball/master to setup best
> performance on database. I will check the one you told.
>
>
Note that I did NOT ask about tuning. I specifically asked about
MONITORING. That's not a typo. There's a reason for that.



>
> >
> >> I have 10.000 test users connectes with PPoE with 5 minutes timeout
> >> and interim every 1 minute.
> >>
> >>
> >
> >Interim 1 minute? really? Is that how you REAL production environment is
> going to be, or is it just a method to test db scalability? 15 minutes - 1
> hour interim should generally be acceptable in production environment.
>
> This is test scalability and to know limits from database and to make grow
> fast radacct and see how works database with 10.000.000 rows on radacct.
>  I will have on production more than 1.000 routers with average 20 users
> on each router connected during 1h (always 10 connected) and all this
> routers will be 15 min interim.
>
>

Then setup your test system to be similar with your future-to-be-production.

If your focus is "10M rows on radacct", then make sure you have that much
entry in your test setup. It should be easy enough to generate dummy acct
rows even when without a real radius server.

Then again, if you keep 10M rows in your "live" radacct table for just 20k
users, then you might need to fix that. Using a custom system where radacct
will only keep absolute-minimum-needed (e.g. only this month and the last),
with old acct records moved to another archive table, would get you better
result.

Note that I said CUSTOM. Meaning you need to have some level of expertise
to able to modify it yourself (or ask an expert to do that for you).



>
> >
> >DB write operations (including one caused by interim updates) are
> generally much more expensive than read operations. 1 minute interim for
> 10k connected users would roughly equal to average of 167 insert/update
> operations on the DB. That would translate to several times the number of
> that in disk random write IOPS, probably to around 300-1k IOPS. An average
> consumer SSD can handle that easily, but a HDD will not be able to provide
> that number of random write IOPS.
> >
>
> Yes there is SSD disk and at the moment i got on test 140.000 rows on
> radacct every hour (that was with 12.000 users 5 minutes timeout and
> internim 1 minute - that will be: 12.000 x (60/5) = 144.000)
>
>
Yet you still mention nothing about current system load :)

iostat is basically a simple tool to measure current disk usage, as I
believed that would be the bottleneck in your case. But since you say you
already use SSD, it might not be so. In any case, performance monitoring
tools (even simple commands like "top" or "iostat") will help you find the
bottleneck, and fix it.



> >
> >
> >
> >>
> >> As Alan Dekok told this duplicate key error is because FreeRadius 3.0.4
> .
> >> but I'm not able to compile and install newer version because is not
> >> on the
> >> Centos_7 repository ..
> >>
> >>
> >You're not able to COMPILE? Why?
> >Is there a reason that prevents you from building FR RPM from latest 3.x
> source? If you encounter an error when building it, post the errors so that
> it can be fixed.
> >
> >
>
> I've tried but because I'm not developer and not expert on servers and
> linux is hard to me to build other version thant the one I can download
> from "official" repository.
>


I believe the current state of freeradius centos builds is that "no
up-to-date repository available, but you can compile it easily". You don't
need to be an expert, nor a developer. You just need basic linux packaging
knowledge, time, and willingness.

The following steps should work (tested on a minimal centos-7 x86_64
container):

yum -y install wget tar bzip2 openssl rpm-build yum-utils
yum -y install
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
wget -c
ftp://ftp.freeradius.org/pub/freeradius/freeradius-server-3.0.10.tar.bz2
tar xfj freeradius-server-3.0.10.tar.bz2
mkdir -p ~/rpmbuild/SOURCES
cp freeradius-server-3.0.10.tar.bz2 freeradius-server-3.0.10/redhat/*
~/rpmbuild/SOURCES/
rpmbuild -bs freeradius-server-3.0.10/redhat/freeradius.spec
yum-builddep ~/rpmbuild/SRPMS/freeradius-3.0.10-2.el7.centos.src.rpm
rpmbuild --rebuild ~/rpmbuild/SRPMS/freeradius-3.0.10-2.el7.centos.src.rpm
yum -y install ~/rpmbuild/RPMS/x86_64/freeradius-{3,config,utils,mysql}*.rpm
sed -i "s/allow_vulnerable_openssl = no/allow_vulnerable_openssl = yes/g"
/etc/raddb/radiusd.conf
systemctl start radiusd
systemctl status radiusd


And even I've read that 3.0.10 still on test on some environments.
>
>

It depends on who you trust the most, really.

If you trust redhat most, and get support from redhat (or the centos
list/forum/whatever), then use whatever version is in the default centos
repository. But you should also treat redhat as your primary source of
support (i.e. "ask there first"), as there might be problems fixed in later
versions that is NOT in centos repository yet.

If you trust FR authors, use whatever version they mark as stable, and ask
FR-related problems on this list first.



> So for my production I really one to use the most stable version. And I
> prefere to use the one that comes with the centos 7 at the moment 3.0.4.
>
>
See the line about "support" above.

>> I modify max_spare_server to 400 and still you probably need to
> >> increase spare . wich value do you think could be ok ?
> >>
> >>
> >>
> >>
> >>
> >> Do I need to modify any other value ?
> >>
> >>
> >
>


Basically you need to know how long the db would take in average to process
each request. Then find out how many requests (including interim) that you
need to handle in a second. Then use those two numbers to adjust radius
config.

Note that the bottleneck is most often in the db, so the monitoring result
in db server is extremely essential before you can move further. There's
really no point in increasing max number of connections if your db is
already too busy handling current ones. ONLY if you ABSOLUTELY sure that
your db is still "idle", and that the long processing times is caused by
something like network delays, THEN you should try increasing concurrent
connections to the db.


Since you also wrote

Fri Nov 20 00:16:25 2015 : Warning: rlm_sql (sql): 22 of 22
connections in use.
You probably need to increase "spare"


it makes me wonder, what other files did you edit? Did you perhaps MANUALLY
edit the sql limit in /etc/raddb/mods-available/sql?

The default value in that file should be

max = ${thread[pool].max_servers}
spare = ${thread[pool].max_spare_servers}

... meaning it should use whatever value in radiusd.conf. However your log
above says it's limited to 22. Have you changed it manually? Or did your db
server perhaps limit max connection to 22?

Try increasing min_spare_servers to something higher (e.g. "64") and see if
the error changes.



> >I'd say leave FR alone for now, and check your db disk performance. Then
> either adjust your test scenario, or adjust your hardware.
>
>
> I'm working and will keep testing to setup the best performance on
> database (last month I've been doing this... ) at the beginig database was
> able to handle only 50.000 new rows on radacct every hour, now it's writing
> 140.000 so new versions of mariaDB and increasing RAM, disk and performance
> has made tha database works beter.
>
> But I will like to fix this errors on freeRadius (that is working really
> perfecte).
>
> Is there any possibility to fix this errors on 3.0.4 ?
>
>
> Error: rlm_sql_mysql: MySQL error 'Duplicate entry
> 'e35620bfd04391bc9c6f5afe3628449d' for key 'acctuniqueid''
>
> Fri Nov 20 00:10:14 2015 : Error: rlm_sql (sql): Duplicate entry
> 'ab0c9cb893d6af4de3388749d7004c2b' for key 'acctuniqueid'
>


See the line about "support" earlier.

*sorry, I'm not willing to spend time chasing other users's problems if
Alan already said the problem is fixed in later versions.


-- 
Fajar