Cannot control attribute ordering via "rlm_perl"

[Alan DeKok]

claude.brown wrote:
> Our new module was designed to replace "rlm_sql" and meet these goals:
> - Be roughly equivalent to "rlm_files" in terms of speed
> - Utilise all the features of "rlm_files" - avoid re-inventing that wheel
> - Allow high rate of user-by-user updates; i.e. avoid config re-write as per
> "rlm_fastfile"

  ?  The "fastusers" module is deprecated, because the "files" module is
just as fast.  The "files" module also can be HUP'd, so it can be
reloaded on the fly.

  Just use: radmin -e "hup files"

  and it will reload *only* the "files" module.  I've tested it at
loading 100K+ users/s off of disk.

> - Simple for stability: no shared in-memory state (avoid locking and races)

  The server core takes care of that when the "files" module is reloaded.

> - Simple for stability: avoid complex on-disk structures like databases with
> dubious libraries
> - Simple for stability: easy mechanism to re-write entire config (say daily)
> to iron our errors

  Daily config reloads are easy.

> - Simple for stability: re-use as much of freeRADIUS as possible; avoid
> writing lots of new code
> 
> We acheived all these goals and can now process bring all our customers back
> onto our service in about
> five minutes. The price is a lot of i-nodes - we end up with one file per
> user in a dir tree.

  5 minutes for what, exactly?

  Say you have a format similar to the "users" file, with one user per
file.  Loading 100K users will mean 100K file reads, and that can take a
long time.  So, do that in a "cron" job.  Have it collect the individual
user files into one large file.  That might take 5 minutes, but who
cares?  It's once a day.

  Then, point the "files" module at the collected file.  It shouldn't
take longer than a second or two to reload it.

> With "rlm_sql" it would take an hour or two only then with careful (and
> human driven) rate management.

  I'm not sure what that means.  An hour or two to load SQL?  What is it
doing?

> The main issues driving this delay were:
> - "rlm_sql" calls during EAP negotation instead of just at the end of EAP

  That can be fixed without a new module.

> - Performance issues on our MySQL backend that we didn't have budget to
> resolve
> - Thread lock-up's inside MySQL library yet no MySQL server queries were
> active

  I've seen lots of people running MySQL with 300K+ users, and no
problems.  The system needs to be designed carefully, but it *does* work.

> If this module is of interest to the community we are happy to contribute
> it.

  I'd first want to know how many users you have.  And why it's taking
so long to get a system up and running.  It sounds like something is
seriously wrong.

  What does it mean to "bring customers into service in 5 minutes"?
With SQL, you should be able to keep the RADIUS server at 100% uptime.
Then, re-write individual user entries via another administration
process.  Rewriting one user entry should take ~10ms at MOST with any
SQL server.  And when the server starts up, it just connects to SQL.  It
doesn't need to read all of the users from SQL.

  So there's no reason for any downtime, and having 10 users in SQL is
just as fast as having 10M users in SQL.

  It really sounds like your *architecture* is wrong.  Find that and fix
it.  Writing a new module should *not* be necessary.

  Alan DeKok.