sqlippool additional information

[Alan DeKok]

On Dec 5, 2018, at 10:35 PM, Nathan Ward <lists+freeradius at daork.net> wrote:
> Yeah. The LDAP thing would be good - we’d need a way to do longest prefix match though. Both redis and SQL are just going to tell us an IP, not a prefix (or, some other label like a prefix name or group or something). I don’t think LDAP can do that itself.
> We could create an LDAP entry per IP, but, that’s a drag to maintain..

  You don't want to do that.

> How have the systems you’ve seen handle this?

  It should just be putting an IP prefix into LDAP, and then selecting a subtree based on that.

> I’m staying away from redis right now - it doesn’t give me much advantage over SQL, I don’t think, and is a bit of a learning curve for the rest of the team. Both just give me an IP, and work like a grab bag, yeah?

  Yes.  The main difference is that Redis is faster.  Much, much, faster.

> We’re not (necessarily*), sorry if that was implied in some way. Based on IP from sqlippool we need to get:
> - Router
> - Subnet Mask
> - Primary DNS
> - Secondary DNS
> - Domain name (ick. for now.)

  Yeah... those should all be keyed off of the prefix.  It should be:

- get IP
- look up IP to get prefix
 - prefix table contains routing, etc. information

> My understanding is that the sql xlat can get one of those at a time, so, we’d do something like:
> update reply {
>  Router-Attribute-Whatever = “%{sql:SELECT router FROM prefixes WHERE prefix >>= '%{reply:Framed-IP-Address}';"
>  Framed-IP-Netmask = “%{sql:SELECT netmask FROM prefixes WHERE prefix >>= '%{reply:Framed-IP-Address}’;”
>  .. etc ..
> }

  You could get them all as one string, and then parse them in FreeRADIUS.  But that's ugly.

> We actually don’t assign them to those reply attributes directly - we have some internal control attributes, and then later we turn the control attributes in to vendor/BNG specific attributes - but that’s just an example.
> 
> I can optimise the above in to the query cache, aside for one query, by doing (pseudocode):
> - control:Internal-Attr-Prefix = SELECT prefix FROM prefixes WHERE prefix >>= ‘%{reply:Framed-IP-Address}’;
> - reply:Attribute-1 = SELECT attribute1 FROM prefixes WHERE prefix = ‘%{control:Internal-Attr-Prefix}’;
> - reply:Attribute-2 = SELECT attribute2 FROM prefixes WHERE prefix = ‘%{control:Internal-Attr-Prefix}’;
> - reply:Attribute-3 = SELECT attribute2 FROM prefixes WHERE prefix = ‘%{control:Internal-Attr-Prefix}’;
> 
> But, it’s still not ideal - I’d rather avoid the round trip to the DB over and over, and just get all the attributes in one hit. I suppose I could just stick an encoded DHCP options string directly in to the DB, pull it out in one query then unpack it with the dhcp_options xlat.. but it’s pretty ugly.

  Yeah.  That's why we added "map" in v4:

	map sql select {
		radius-attribute = sql-column
		...
	}

  Which does multiple things in one go.

> I could also do the above first query, then call LDAP for the attributes based on the prefix - but, data in 2 DBs seems like a design bug.

  Yeah.

> Yep. I’ve done quite a lot of experimentation with it in the last few weeks, too, and got a pretty significant performance improvement over the stock queries, in my environment. RAND() really slows down large pools, and some modern SELECT parameters can really help concurrency (SELECT .. FOR UPDATE SKIP LOCKED). MariaDB doesn’t support SKIP LOCKED, recent MySQL does, Postgres does. Without it, FreeRADIUS effectively locks all the rows in the pool, as the radippool_poolname_expire index record is locked, even with innodb, even with postgres - add the RAND() sort in there and your request latency goes through the roof for high request rates and pools with lots of unused addresses (think /16 RFC6498 pools, for example).

  Sounds good.

> I’ll write some of this up when I’ve got this production proven, and maybe submit a PR or two.

  Thanks.  Performance improvements are always nice. :)

  Alan DeKok.