Minor but fun changes today

[Arran Cudbard-Bell]

> On 11 Feb 2015, at 12:32, Matthew Newton <mcn4 at LEICESTER.AC.UK> wrote:
> 
> On Wed, Feb 11, 2015 at 11:15:28AM -0500, Arran Cudbard-Bell wrote:
>> 
>>> On 11 Feb 2015, at 11:08, Phil Mayers <p.mayers at IMPERIAL.AC.UK> wrote:
>>> 
>>> Sorry, not what I meant - rather the figure "32kb" for each REQUEST. Some sites might allocate a lot of crap in their configs, others little, so it might be something to vary.
>>> 
>>> But it's probably pointless optimisation!
>> 
>> It's already configurable (shhh, don't tell anyone). But it's one of those knobs that really shouldn't exist, and should be determined automatically by the server.
> 
> Is there any reason why the server can't keep track of the average
> size of the last <n> requests, and dynamically adjust the amount
> of memory it allocates for the request pool?

That's what I was suggesting (in a previous email). But the talloc API
doesn't track or expose such information currently.

> Not sure if there is a way in, say, talloc free to find out how
> much memory was actually freed in total? Then it would be an easy
> job at free time to keep track of how much was used for that
> request, and feed back into a loop to determine the allocation
> size.

Right but there's significant churn in the context of the request
as it runs through all the different modules.

i.e. one module could allocate a 1M chunk in the context, then
free it, and at cleanup time you'd never know that had happened.

Well you hope you wouldn't know that had happened. To use contexts
efficiently you have to do try and stick to LAFF
(Last Allocated First Freed). That's not always possible, and in those
cases the context will leak memory (in a controlled way).

i.e. if you allocate chunk A, then chunk B, then free chunk A, the
memory isn't returned to the context.

The talloc code ain't got the smarts to deal with memory fragmentation,
or doesn't want the overhead.

Calling talloc_free_children() on a pool, will reset it back to its
initial state. So we could in theory maintain a pool of pools, and
avoid all runtime mallocs(). The issue is whether the overhead of
managing the pool of pools is greater than the overhead of calling
malloc.

> But maybe it's just overcomplicating things. Worst case you don't
> want the situation that most requests go 10 octets over 32k - so
> some debug or similar to tell the admin this has happened would be
> useful.

The only way to do that currently would be to set a memlimit on the
context, which, when exceeded would cause allocations to fail, and the
server to exit. Not ideal.

-Arran

Arran Cudbard-Bell <a.cudbardb at freeradius.org>
FreeRADIUS development team

FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 872 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freeradius.org/pipermail/freeradius-devel/attachments/20150211/ade22184/attachment.sig>