Very large environment depending on FreeRadius
Greetings guys, Need advice before going live: I have deployed a FreeRadius server in an environment with 2,491,000 subscribers. GGSN 2,491,000 MSISDN subscribers. 3 APN(s). 1 APN will be authenticated locally: 830 000 subscribers 1 APN will be proxied for: 1,660,000 subscribers 1 APN another will be proxied for: 1,660,000 subsicbers. BRAS for PPPOE is 1000 subscribers. We tested the configuration and it was working, with attribute filters on proxy etc.. for all our requirements. Using MYSQL Backend and SQL IP Pool to hand out IP addresses for localally authenticated. Also have a secondary standby FreeRadius, with mysql replication using a bunch of additional scripts I wrote to handle redundancy and consistency checks. The system is running live for the 1000 subscribers. We about to swing over the 2,491,000 mobile subscribers... Can you please advice me on anything I should watch out for or plan for? Should I be worried about tuning the ports? listening on multiple ports for auth+acct? anything strange? Any tips??? :-) go mad, i need to ensure this runs smoothly... System is running on Ubuntu Linux Server. -- Christiaan Rademan - JNCIE #661 Mobile: +27 83 419 2078 E-mail: christiaan.rademan@gmail.com
On 03/15/2012 07:38 AM, Christiaan Rademan wrote:
Can you please advice me on anything I should watch out for or plan for?
I'm sure others will chip in, but basically: don't worry about FreeRADIUS, worry about your SQL database. FreeRADIUS itself can handle a truly enormous rate of authentication and accounting packets. The problem people seem to run into at scale is the SQL database they're using for authentication (i.e. to read password) or accounting (i.e. write accounting records) is too slow, which means FreeRADIUS becomes slow. Then people get in a muddle and think adding hundreds of threads to the thread pool will help ("My database is slow... I know, I'll add MORE concurrent queries, that'll speed it up"). It's a particular problem if, after a couple of weeks, they've got 100million rows in their accounting table and accounting takes seconds to complete, so ensure you're archiving regularly. Assuming you're not doing any SQL activity for proxied packets, I don't think you need to worry too much about the, but DO ENSURE you are running 2.1.12, and not some earlier version. With regards the local auth, you say you're using MySQL and sqlippool; you might want to check the list archives for this, there has been some discussion in the past. I don't use MySQL, but my understanding was that the required locking (to avoid handing the same IP out twice) was problematic in some fashion in MySQL. Basically: run some test auths through the server and dump the SQL queries it generates. Then think about how those SQL queries will perform in a month, when your SQL DB is full of accounting records, or when 100 queries/sec come in.
Thank you so much for the tips Phil Mayers. I have optimised everything, except archiving of the accounting messages. That would be a good idea... On 15/03/2012 10:58, Phil Mayers wrote:
On 03/15/2012 07:38 AM, Christiaan Rademan wrote:
Can you please advice me on anything I should watch out for or plan for?
I'm sure others will chip in, but basically: don't worry about FreeRADIUS, worry about your SQL database.
FreeRADIUS itself can handle a truly enormous rate of authentication and accounting packets.
The problem people seem to run into at scale is the SQL database they're using for authentication (i.e. to read password) or accounting (i.e. write accounting records) is too slow, which means FreeRADIUS becomes slow. Then people get in a muddle and think adding hundreds of threads to the thread pool will help ("My database is slow... I know, I'll add MORE concurrent queries, that'll speed it up").
It's a particular problem if, after a couple of weeks, they've got 100million rows in their accounting table and accounting takes seconds to complete, so ensure you're archiving regularly.
Assuming you're not doing any SQL activity for proxied packets, I don't think you need to worry too much about the, but DO ENSURE you are running 2.1.12, and not some earlier version.
With regards the local auth, you say you're using MySQL and sqlippool; you might want to check the list archives for this, there has been some discussion in the past. I don't use MySQL, but my understanding was that the required locking (to avoid handing the same IP out twice) was problematic in some fashion in MySQL.
Basically: run some test auths through the server and dump the SQL queries it generates. Then think about how those SQL queries will perform in a month, when your SQL DB is full of accounting records, or when 100 queries/sec come in. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- Christiaan Rademan - JNCIE #661 Mobile: +27 83 419 2078 E-mail: christiaan.rademan@gmail.com
Christiaan Rademan wrote:
I have deployed a FreeRadius server in an environment with 2,491,000 subscribers.
That's a fairly high number.
We tested the configuration and it was working, with attribute filters on proxy etc.. for all our requirements.
Using MYSQL Backend and SQL IP Pool to hand out IP addresses for localally authenticated.
Have you done performance tests? If not, who knows what will happen.
Can you please advice me on anything I should watch out for or plan for?
Test it before switching over 3M users. Alan DeKok.
participants (3)
-
Alan DeKok -
Christiaan Rademan -
Phil Mayers