FreeRadius crashed on accounting load tests with 1000 concurrent clients

[Alan DeKok]

Dinh Pham Cong wrote:
> I saw that when 1000 client was created at the same time and send
> requests to Radius server concurrently there were about 2000+ radiusd
> threads were created too. I thought that 1000 concurrent client could be
> translated to more than 1000 packets/s.

  Maybe.  It depends on a number of factors.

> I have done another load tests with smaller concurrent clients: 100,
> 200, 500. However, Radiusd can not handle more than 1000 request/s
> (total requests/total time used to sent all the requests). For example
> Radiusd can handle 20 000 request in more than 20 seconds, 40 000
> requests in more than 43 seconds.

  That is usually because the back-end database can't handle more than
those loads.

  Try replacing the "sql" entry in the "accounting" section with "ok".
This means it won't log to the DB, but it will run faster.

  You SHOULD see it handle many 1000's of accounting requests/s.  This
means that FreeRADIUS can handle that load all by itself.  The problem
would then be the interaction between FreeRADIUS && MySQL.

> I thought that num_sql_socks = 1000 meaned that Radiusd would create a
> connection pool of 1000 MySQL connections.

  Yes.  But you do NOT need 1000 MySQL connections to do 1000 packets/s.
   I've seen systems with 50 mySQL connections getting 2K packets/s.

>      See also "radperf" (http://networkradius.com/radperf.html).  It can do
>     load testing and user login simulations.
>  
> Thanks I will try.

  Let me know of any issues you find.

> I have done separate load tests of INSERT and DELETE on radacct table
> (using MEMORY engine) and found that MySQL 5.0 can handle about 6000 -
> 8700 qps. I have used mysqlslap to do that.

  FreeRADIUS does a mix of selects, inserts and updates for accounting.
 It may be that the DB can only handle 1000 transactions/s with that mix.

  See also "sqltrace" config in sql.conf.  You can try the following tests:

- start off with a clean (i.e. empty) DB
- enable sqltrace
- run 10's of 1000's of packets through the server
- wipe the DB
- use a command-line SQL client to re-play SQL queries from the sqltrace
file

  The performance of the last step SHOULD be within a factor of 2 of the
performance of the RADIUS server.  The performance will be higher than a
real RADIUS server, because the RADIUS server has to send && receive
packets over the network.  The CLI client just has to read a file.

  Alan DeKok.