IP pools with rlm_sql_mongo + some general testing with MongoDB
Firstly, many thanks to all concerned for creatiing a MongoDB module. I have been doing a bit of testing of IP address allocation from pools in MongoDB and I wanted to share some thoughts and notes (in no particular order) in case they are useful: 1) I noticed that rlm_sql_mongo expects any returned data to be in a field called "value". This means that for example for a document which defines an IP address entry in the pool the actual IP must be stored in the "value" field instead of something like "framed_ip_address": { pool_key: "foo" pool_name: "bar" expiry_time: xxx value: "192.168.1.1" } I don't see any problem with this, but I mention it in case it is useful to anyone else who tests the module. 2) MongoDB requests generally use a lot of curly brackets so when callied via %{sql: xx} it is necessary to escape them with an extra percent symbol. So in the actual request any "}" should be replaced with "%}". This also maybe useful info to anyone testing for the first time. 3) In my case I wanted to implement an IP pool with "sticky addresses" based on a unique PPP username. One point to note is that the method findAndModify() does not support sorting by an exact value. So for example it cannot simulate something like "ORDER BY foo <> bar" which is possible with other databases. In my case I decided to work around this using two seperate requests: the first one does a lookup by exact string to see if we can allocate the same IP, then if this does not return anything do a second lookup and sort by expiry_time to find the oldest free IP. Due to this I implemented all the database requests directly in unlang directly in the authorize section instead of using the sqlippool module. 4) I did at one point test the sqlippool module with a variation of the above logic by calling one request in allocate_clear and another in allocate_find. However for some reason I noticed that allocate_clear was not geting called every time and somehow the server was skipping it and calling allocate_find immediately. It could have been a problem with my configuration so I will try to repocude the problem if I get time. 5) In general I am very pleased with the performance. For a pool database with 20000 entries on a modest 6 core virtual machine I reached 6000 requests per second with radperf. -- Ben Thompson
On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <b.thompson@latera.ru> wrote:
Firstly, many thanks to all concerned for creatiing a MongoDB module.
That would largely be me. :) The MongoC API is pretty good. Mongo itself is rather more confusing.
I have been doing a bit of testing of IP address allocation from pools in MongoDB and I wanted to share some thoughts and notes (in no particular order) in case they are useful:
Yes, they are.
1) I noticed that rlm_sql_mongo expects any returned data to be in a field called "value". This means that for example for a document which defines an IP address entry in the pool the actual IP must be stored in the "value" field instead of something like "framed_ip_address": { pool_key: "foo" pool_name: "bar" expiry_time: xxx value: "192.168.1.1" } I don't see any problem with this, but I mention it in case it is useful to anyone else who tests the module.
I'll make a note.
2) MongoDB requests generally use a lot of curly brackets so when callied via %{sql: xx} it is necessary to escape them with an extra percent symbol. So in the actual request any "}" should be replaced with "%}". This also maybe useful info to anyone testing for the first time.
Sure. I'll add some text around that,
3) In my case I wanted to implement an IP pool with "sticky addresses" based on a unique PPP username. One point to note is that the method findAndModify() does not support sorting by an exact value. So for example it cannot simulate something like "ORDER BY foo <> bar" which is possible with other databases. In my case I decided to work around this using two seperate requests: the first one does a lookup by exact string to see if we can allocate the same IP, then if this does not return anything do a second lookup and sort by expiry_time to find the oldest free IP. Due to this I implemented all the database requests directly in unlang directly in the authorize section instead of using the sqlippool module.
Good! FreeRADIUS can do just about anything. :)
4) I did at one point test the sqlippool module with a variation of the above logic by calling one request in allocate_clear and another in allocate_find. However for some reason I noticed that allocate_clear was not geting called every time and somehow the server was skipping it and calling allocate_find immediately. It could have been a problem with my configuration so I will try to repocude the problem if I get time.
Hmm... IIRC allocate_clear isn't called every time? I think it's once a second.
5) In general I am very pleased with the performance. For a pool database with 20000 entries on a modest 6 core virtual machine I reached 6000 requests per second with radperf.
Very nice! The ISC "kea" DHCP server does 6K/s to it's in-memory store. And less than 1K per second with databases. To me, that is a huge validation of the design of FreeRADIUS. We write software that *isn't* crap. And, we realize we're not smart enough to write our own database. So we just use existing ones. The result is that our *untuned* performance to a database is as fast as the KEA DHCP *tuned* performance of an in-memory store. I think it's time to start pushing DHCP rather more aggressively. :) We are faster than ISC, support more databases, have more configurable policies, etc. The only thing we're missing is the ability to read the ISC config file format. Anyone want to back-port rlm_isc_dhcp from the "master" branch to v3. ? :) Alan DeKok.
Hi, just my cent. On Fri, 2020-02-14 at 10:54 -0500, Alan DeKok wrote:
On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <b.thompson@latera.ru
wrote: Firstly, many thanks to all concerned for creatiing a MongoDB module.
That would largely be me. :) The MongoC API is pretty good. Mongo itself is rather more confusing.
I have been doing a bit of testing of IP address allocation from poolsin MongoDB and I wanted to share some thoughts and notes (in noparticular order) in case they are useful:
Yes, they are.
1) I noticed that rlm_sql_mongo expects any returned data to be in afield called "value". This means that for example for a document whichdefines an IP address entry in the pool the actual IP must be storedin the "value" field instead of something like "framed_ip_address":{ pool_key: "foo" pool_name: "bar" expiry_time: xxx value: "192.168.1.1"}I don't see any problem with this, but I mention it in case it isuseful to anyone else who tests the module.
I'll make a note.
The choice to use 'value' attribute is related to the fact that in MongoDB doesn't exist a scalar response. It's always a document and this is been a good compromise that could be used in a different use case. This is better than implement a static response for sqlippool and the driver is more simple.
2) MongoDB requests generally use a lot of curly brackets so whencallied via %{sql: xx} it is necessary to escape them with an extrapercent symbol. So in the actual request any "}" should be replacedwith "%}". This also maybe useful info to anyone testing for the firsttime.
Sure. I'll add some text around that,
3) In my case I wanted to implement an IP pool with "sticky addresses"based on a unique PPP username. One point to note is that the methodfindAndModify() does not support sorting by an exact value. So forexample it cannot simulate something like "ORDER BY foo <> bar" whichis possible with other databases. In my case I decided to work aroundthis using two seperate requests: the first one does a lookup by exactstring to see if we can allocate the same IP, then if this does notreturn anything do a second lookup and sort by expiry_time to find theoldest free IP. Due to this I implemented all the database requestsdirectly in unlang directly in the authorize section instead of usingthe sqlippool module.
Good! FreeRADIUS can do just about anything. :)
If you need to sort your data probably the better solution is to use the aggregate and create a pipeline.
4) I did at one point test the sqlippool module with a variation ofthe above logic by calling one request in allocate_clear and anotherin allocate_find. However for some reason I noticed thatallocate_clear was not geting called every time and somehow the serverwas skipping it and calling allocate_find immediately. It could havebeen a problem with my configuration so I will try to repocude theproblem if I get time.
Hmm... IIRC allocate_clear isn't called every time? I think it's once a second.
5) In general I am very pleased with the performance. For a pooldatabase with 20000 entries on a modest 6 core virtual machine Ireached 6000 requests per second with radperf.
Very nice! The ISC "kea" DHCP server does 6K/s to it's in-memory store. And less than 1K per second with databases. To me, that is a huge validation of the design of FreeRADIUS. We write software that *isn't* crap. And, we realize we're not smart enough to write our own database. So we just use existing ones. The result is that our *untuned* performance to a database is as fast as the KEA DHCP *tuned* performance of an in-memory store. I think it's time to start pushing DHCP rather more aggressively. :) We are faster than ISC, support more databases, have more configurable policies, etc. The only thing we're missing is the ability to read the ISC config file format. Anyone want to back-port rlm_isc_dhcp from the "master" branch to v3. ? :) Alan DeKok.
-List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On 14/02/2020, Alan DeKok <aland@deployingradius.com> wrote:
On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <b.thompson@latera.ru> wrote:
Firstly, many thanks to all concerned for creatiing a MongoDB module.
That would largely be me. :)
Thanks Alan.
3) In my case I wanted to implement an IP pool with "sticky addresses" based on a unique PPP username. One point to note is that the method findAndModify() does not support sorting by an exact value. So for example it cannot simulate something like "ORDER BY foo <> bar" which is possible with other databases. In my case I decided to work around this using two seperate requests: the first one does a lookup by exact string to see if we can allocate the same IP, then if this does not return anything do a second lookup and sort by expiry_time to find the oldest free IP. Due to this I implemented all the database requests directly in unlang directly in the authorize section instead of using the sqlippool module.
Good! FreeRADIUS can do just about anything. :)
In case it is useful to anyone else, I ended up with something like this: update reply { &Framed-IP-Address := "%{sql: db.radippool.findAndModify({'query': {'$and': [{'pool_name': '%{control:Pool-Name}'%}, {'pool_key': '%{User-Name}'%}]%}, 'update': {'$set': {'expiry_time': {'$date': {'$numberLong': '%{expr: (%l + 3600) * 1000}'%}%}, 'nas_ip': '%{NAS-IP-Address}', 'calling_station_id': '%{Calling-Station-Id}'%}%}, 'fields': {'_id': 0, 'value': 1%}%})}" } if (&reply:Framed-IP-Address !~ /^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$/) { update reply { &Framed-IP-Address := "%{sql: db.radippool.findAndModify({'query': {'$and': [{'pool_name': '%{control:Pool-Name}'%}, {'expiry_time': {'$lt': {'$date': {'$numberLong': '%{expr: %l * 1000}'%}%}%}%}]%}, 'update': {'$set': {'expiry_time': {'$date': {'$numberLong': '%{expr: (%l + 3600) * 1000}'%}%}, 'pool_key': '%{User-Name}', 'nas_ip': '%{NAS-IP-Address}', 'calling_station_id': '%{Calling-Station-Id}'%}%}, 'fields': {'_id': 0, 'value': 1%}%})}" } }
4) I did at one point test the sqlippool module with a variation of the above logic by calling one request in allocate_clear and another in allocate_find. However for some reason I noticed that allocate_clear was not geting called every time and somehow the server was skipping it and calling allocate_find immediately. It could have been a problem with my configuration so I will try to repocude the problem if I get time.
Hmm... IIRC allocate_clear isn't called every time? I think it's once a second.
Ahh that would explain it then. Thanks.
I think it's time to start pushing DHCP rather more aggressively. :) We are faster than ISC, support more databases, have more configurable policies, etc. The only thing we're missing is the ability to read the ISC config file format.
I do recommend FreeRADIUS to others (including for DHCP) and can't see that anyone would go back to ISC after trying it. One thing though which our clients are asking for more and more is DHCPv6 so on our wishlist we are hoping that you will be get this in (v4) at some point. -- Ben Thompson
On Feb 17, 2020, at 5:39 AM, Benjamin Thompson <b.thompson@latera.ru> wrote:
In case it is useful to anyone else, I ended up with something like this:
Not trivial, but definitely powerful.
I do recommend FreeRADIUS to others (including for DHCP) and can't see that anyone would go back to ISC after trying it.
The things which are easy in ISC can be difficult in FreeRADIUS. i.e. basic rules around static IPs, etc. The things which are impossible in FreeRADIUS are only slightly more difficult in FreeRADIUS. FreeRADIUS has higher performance than ISC, is more flexible, supports more database back-ends, etc. In some sites we're seeing 8K DORAs per second with a Redis cluster back-end, and minimal logic. When we add multiple round-trips to databases for various other policy checks, that drops to "only" 2K DORAs per second.
One thing though which our clients are asking for more and more is DHCPv6 so on our wishlist we are hoping that you will be get this in (v4) at some point.
Hmm... $ cd ~/git/v4 $ ls src/protocols/dhcpv6 all.mk attrs.h base.c decode.c dhcpv6.h encode.c packet.c $ ls src/tests/unit/protocols/dhcpv6 addresses.txt packet_domain-list.txt rfc6225.txt bools.txt packet_ia-na.txt rfc6355.txt dates.txt packet_ia-pd.txt rfc6939.txt dictionary packet_ia-ta.txt rfc7078.txt dns_wire_format.txt packet_ntp-server.txt rfc7600.txt fixed_element_arrays.txt packet_sip-server-d.txt rfc8415.txt integers.txt rfc3315.txt strings.txt microsoft.txt rfc3319.txt tlvs.txt packet.txt rfc3633.txt variable_element_arrays.txt packet_AFTR-Name-rfc6334.txt rfc3646.txt packet_client_server.txt rfc4704.txt I have no idea what that's about. :) The main difficulty with DHCPv6 is that it doesn't do TLVs. It allocates all attributes from the same global attribute space. Including sub-attributes. Allowing this in v4 requires some fairly substantial rework of core pieces. So that's taking time. Alan DeKok.
participants (3)
-
Alan DeKok -
Benjamin Thompson -
Geaaru