IP pools with rlm_sql_mongo + some general testing with MongoDB

[Alan DeKok]

On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <b.thompson at latera.ru> wrote:
> 
> Firstly, many thanks to all concerned for creatiing a MongoDB module.

  That would largely be me.  :)

  The MongoC API is pretty good.  Mongo itself is rather more confusing.

> I have been doing a bit of testing of IP address allocation from pools
> in MongoDB and I wanted to share some thoughts and notes (in no
> particular order) in case they are useful:

  Yes, they are.

> 1) I noticed that rlm_sql_mongo expects any returned data to be in a
> field called "value". This means that for example for a document which
> defines an IP address entry in the pool the actual IP must be stored
> in the "value" field instead of something like "framed_ip_address":
> {
>    pool_key: "foo"
>    pool_name: "bar"
>    expiry_time: xxx
>    value: "192.168.1.1"
> }
> I don't see any problem with this, but I mention it in case it is
> useful to anyone else who tests the module.

  I'll make a note.

> 2) MongoDB requests generally use a lot of curly brackets so when
> callied via %{sql: xx} it is necessary to escape them with an extra
> percent symbol. So in the actual request any "}" should be replaced
> with "%}". This also maybe useful info to anyone testing for the first
> time.

  Sure.  I'll add some text around that,

> 3) In my case I wanted to implement an IP pool with "sticky addresses"
> based on a unique PPP username. One point to note is that the method
> findAndModify() does not support sorting by an exact value. So for
> example it cannot simulate something like "ORDER BY foo <> bar" which
> is possible with other databases. In my case I decided to work around
> this using two seperate requests: the first one does a lookup by exact
> string to see if we can allocate the same IP, then if this does not
> return anything do a second lookup and sort by expiry_time to find the
> oldest free IP. Due to this I implemented all the database requests
> directly in unlang directly in the authorize section instead of using
> the sqlippool module.

  Good!  FreeRADIUS can do just about anything. :)

> 4) I did at one point test the sqlippool module with a variation of
> the above logic by calling one request in allocate_clear and another
> in allocate_find. However for some reason I noticed that
> allocate_clear was not geting called every time and somehow the server
> was skipping it and calling allocate_find immediately. It could have
> been a problem with my configuration so I will try to repocude the
> problem if I get time.

  Hmm... IIRC allocate_clear isn't called every time?  I think it's once a second.

> 5) In general I am very pleased with the performance. For a pool
> database with 20000 entries on a modest 6 core virtual machine I
> reached 6000 requests per second with radperf.

  Very nice!

  The ISC "kea" DHCP server does 6K/s to it's in-memory store.  And less than 1K per second with databases.

  To me, that is a huge validation of the design of FreeRADIUS.  We write software that *isn't* crap.  And, we realize we're not smart enough to write our own database.  So we just use existing ones.

  The result is that our *untuned* performance to a database is as fast as the KEA DHCP *tuned* performance of an in-memory store.

  I think it's time to start pushing DHCP rather more aggressively. :)  We are faster than ISC, support more databases, have more configurable policies, etc.  The only thing we're missing is the ability to read the ISC config file format.

  Anyone want to back-port rlm_isc_dhcp from the "master" branch to v3. ? :)

  Alan DeKok.