FreeRadius high availability lab

[Norman Elton]

It's interesting to read this, as the suggestions mirror what I've
learned over the years.

This is on a single university campus, so everything is geographically
local. But we have two DCs, a FreeRADIUS server in each. The servers
each have a floating "primary" address, which magically floats to the
other box using keepalived. So if Server A fails, Server B adopts
failed IP address. NAS's (in our case, switches & APs) never have to
deal with failover. Makes things SO much smoother. Holy cow was this a
"lightbulb moment".

We have a database as the "authoritative record", but everything is
Redis-cached on each FreeRADIUS server. It's a bit of a chore to keep
the cache synchronized, but it means that FreeRADIUS is completely
self-sufficient. We use rlm_perl to read this cache when responding to
requests, and to log accounting updates back to the Redis cache. There
are helper scripts (on other servers) which grab those updates and log
them to the database. If there is a flurry of updates, then the Redis
cache queues up for a few seconds while the database catches up. But
FreeRADIUS chugs along, even if the database server completely blows
up. This happened for the first time 2 weeks ago. RAID controller
died. The Redis cache queued up all the requests until we got the
backup database server online. Fun times.

I'm curious if rlm_redis + unlang would be more performant than
rlm_perl + perl script. Haven't had time to test this yet.

Norman

On Tue, Nov 7, 2017 at 8:54 AM, Alan DeKok <aland at deployingradius.com> wrote:
> On Nov 7, 2017, at 6:49 AM, Nathan Ward <lists+freeradius at daork.net> wrote:
>> First thing I’d say is don’t have both load balancers sending traffic to all the RADIUS servers. You don’t gain much resiliency there except in the load balancer - if the pool of RADIUS servers dies for some reason (it will, at some point) you lose your service. Split the backends in to two pools, if you don’t have enough in each pool, get more - don’t merge the pools.
>
>   The approach I often take is:
>
> - two data centres.  Each NAS points to one DC as primary, and the other as secondary
> - each DC is the same
>   - two RADIUS servers, sharing an IP via VRRP
>   - two DBs.
>   - each RADIUS server can write to both SQL DBs, in a fail-over configuration.
>
>   That's good enough for most common uses.  If people need higher performance, the front-end RADIUS servers become proxies, and they load-balance to a set of back-ends, which are all identical (RADIUS + SQL).  The load-balancing is done on a hash of User-Name, for simple sharing and consistency.
>
>   The main issue here is merging all of the data which is split into different SQL DBs.  But that can be done off-line, i.e. *not* live, and *not* by the RADIUS server.
>
>   Much of "high performance" system design ends up working around performance limitations with databases.
>
>> Other than that, there are so, so many questions that need to be asked before you start drawing this sort of picture. In addition to the above:
>> - What is RADIUS doing? Auth? Accounting? Both?
>> - Can you accept lag in accounting getting to the DB? How much?
>> - Can you accept lag in password/user changes? How much?
>> - Can you accept accounting black spots? How long/often?
>> - What conditions cause your NAS to use the secondary server?
>> - Are you doing anything that requires shared state (i.e. IP address assignment, multiple login prevention, etc.)? How important is integrity here if you have to trade it for availability?
>
>   i.e. What problem are you trying to solve?  Knowing that will lead you to a solution.
>
>> These are just some that I came up with off the top of my head.. there are many, many more that would come from the answers to these questions.. Don’t try and design your solution to cater for failure modes that don’t cause problems in your specific environment by adding needless complexity, as adding complexity will cause failures of its own.
>
>   Very true.
>
>   Alan DeKok.
>
>
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html