IP pools with rlm_sql_mongo + some general testing with MongoDB

[Geaaru]

Hi,
just my cent.
On Fri, 2020-02-14 at 10:54 -0500, Alan DeKok wrote:
> On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <b.thompson at latera.ru
> > wrote:
> > Firstly, many thanks to all concerned for creatiing a MongoDB
> > module.
> 
>   That would largely be me.  :)
>   The MongoC API is pretty good.  Mongo itself is rather more
> confusing.
> > I have been doing a bit of testing of IP address allocation from
> > poolsin MongoDB and I wanted to share some thoughts and notes (in
> > noparticular order) in case they are useful:
> 
>   Yes, they are.
> > 1) I noticed that rlm_sql_mongo expects any returned data to be in
> > afield called "value". This means that for example for a document
> > whichdefines an IP address entry in the pool the actual IP must be
> > storedin the "value" field instead of something like
> > "framed_ip_address":{   pool_key: "foo"   pool_name:
> > "bar"   expiry_time: xxx   value: "192.168.1.1"}I don't see any
> > problem with this, but I mention it in case it isuseful to anyone
> > else who tests the module.
> 
>   I'll make a note.

The choice to use 'value' attribute is related to the fact that in
MongoDB doesn't exist a scalar response. It's always a document and
this is been a good compromise that could be used in a different use
case. This is better than implement a static response for sqlippool and
the driver is more simple.


> > 2) MongoDB requests generally use a lot of curly brackets so
> > whencallied via %{sql: xx} it is necessary to escape them with an
> > extrapercent symbol. So in the actual request any "}" should be
> > replacedwith "%}". This also maybe useful info to anyone testing
> > for the firsttime.
> 
>   Sure.  I'll add some text around that,
> > 3) In my case I wanted to implement an IP pool with "sticky
> > addresses"based on a unique PPP username. One point to note is that
> > the methodfindAndModify() does not support sorting by an exact
> > value. So forexample it cannot simulate something like "ORDER BY
> > foo <> bar" whichis possible with other databases. In my case I
> > decided to work aroundthis using two seperate requests: the first
> > one does a lookup by exactstring to see if we can allocate the same
> > IP, then if this does notreturn anything do a second lookup and
> > sort by expiry_time to find theoldest free IP. Due to this I
> > implemented all the database requestsdirectly in unlang directly in
> > the authorize section instead of usingthe sqlippool module.
> 
>   Good!  FreeRADIUS can do just about anything. :)

If you need to sort your data probably the better solution is to use
the aggregate and create a pipeline.

> > 4) I did at one point test the sqlippool module with a variation
> > ofthe above logic by calling one request in allocate_clear and
> > anotherin allocate_find. However for some reason I noticed
> > thatallocate_clear was not geting called every time and somehow the
> > serverwas skipping it and calling allocate_find immediately. It
> > could havebeen a problem with my configuration so I will try to
> > repocude theproblem if I get time.
> 
>   Hmm... IIRC allocate_clear isn't called every time?  I think it's
> once a second.
> > 5) In general I am very pleased with the performance. For a
> > pooldatabase with 20000 entries on a modest 6 core virtual machine
> > Ireached 6000 requests per second with radperf.
> 
>   Very nice!
>   The ISC "kea" DHCP server does 6K/s to it's in-memory store.  And
> less than 1K per second with databases.
>   To me, that is a huge validation of the design of FreeRADIUS.  We
> write software that *isn't* crap.  And, we realize we're not smart
> enough to write our own database.  So we just use existing ones.
>   The result is that our *untuned* performance to a database is as
> fast as the KEA DHCP *tuned* performance of an in-memory store.
>   I think it's time to start pushing DHCP rather more aggressively.
> :)  We are faster than ISC, support more databases, have more
> configurable policies, etc.  The only thing we're missing is the
> ability to read the ISC config file format.
>   Anyone want to back-port rlm_isc_dhcp from the "master" branch to
> v3. ? :)
>   Alan DeKok.
> 
> -List info/subscribe/unsubscribe? See 
> http://www.freeradius.org/list/users.html
>