rlm_ippool memory footprint

[Alan DeKok]

On Oct 14, 2020, at 4:19 PM, James Chapman via Freeradius-Users <freeradius-users at lists.freeradius.org> wrote:
> 
> I sometimes see freeradius killed by OOM-killer at boot time. On
> investigating this, freeradius seems to allocate a lot of RAM to manage
> IP address pools (rlm_ippool). Is it possible to reduce this footprint?

  The module uses gdbm for it's database.  So essentially all memory used is by the gdbm library.

> I have a large IP pool configured (64k addresses) using rlm_ippool.
> 
>         range_start = 172.28.0.1
>         range_stop = 172.28.255.254
>         netmask = 255.255.0.0
>         cache_size = 65535
> 
> freeradius allocates 600M with this configuration:

  Hmm... that's about 10K per IP, which seems excessive.

> Reducing cache_size in the pool config to 255 results in far less memory
> being allocated (92M) :

  Yeah.  Still about 10K per IP.

  I'm not sure why.  The entries stored by the module are less than 64 bytes.  And the key is 16 bytes.  So 10K per entry seems _way_ excessive.

  But in the FreeRADIUS tradition... I can blame something else.  :)   The module uses gdbm, and all of the memory usage is in gdbm.  Not in FreeRADIUS.  You know this is true because changing the cache size directly affects the memory usage.

> But freeradius docs say that cache_size should be set to the number of
> entries in the pool. In a test setup, using a cache_size that is much
> less than the pool size seems to work. Other than for performance
> reasons, is there a reason why cache_size should be set to the number of
> entries in the pool? It seems like a lot of memory is otherwise needed
> for managing an IP address pool...

  The cache size is passed directly to gdbm.  So I don't think it needs to be set to the number of IPs in the pool.  it can be anything which makes sense for you.

  TBH, the solution here is to just use an SQL database.  Something like sqlite beats gdbm in pretty much every way.  In the v4 "master" branch, we've even deleted rlm_ippool.  There's just not enough reasons to use a custom DB when SQL exists.

  If you take a look at the v3.0.x branch in GitHub, there's a bunch of new tools to manage IP ranges in SQL.  That makes it much easier.

  Or if you want to stay with gdbm, use a small value for cache_size.  The only affect is that database lookups will be a little slower.

  Alan DeKok.