On 11 Feb 2015, at 12:32, Matthew Newton <mcn4@LEICESTER.AC.UK> wrote:
On Wed, Feb 11, 2015 at 11:15:28AM -0500, Arran Cudbard-Bell wrote:
On 11 Feb 2015, at 11:08, Phil Mayers <p.mayers@IMPERIAL.AC.UK> wrote:
Sorry, not what I meant - rather the figure "32kb" for each REQUEST. Some sites might allocate a lot of crap in their configs, others little, so it might be something to vary.
But it's probably pointless optimisation!
It's already configurable (shhh, don't tell anyone). But it's one of those knobs that really shouldn't exist, and should be determined automatically by the server.
Is there any reason why the server can't keep track of the average size of the last <n> requests, and dynamically adjust the amount of memory it allocates for the request pool?
That's what I was suggesting (in a previous email). But the talloc API doesn't track or expose such information currently.
Not sure if there is a way in, say, talloc free to find out how much memory was actually freed in total? Then it would be an easy job at free time to keep track of how much was used for that request, and feed back into a loop to determine the allocation size.
Right but there's significant churn in the context of the request as it runs through all the different modules. i.e. one module could allocate a 1M chunk in the context, then free it, and at cleanup time you'd never know that had happened. Well you hope you wouldn't know that had happened. To use contexts efficiently you have to do try and stick to LAFF (Last Allocated First Freed). That's not always possible, and in those cases the context will leak memory (in a controlled way). i.e. if you allocate chunk A, then chunk B, then free chunk A, the memory isn't returned to the context. The talloc code ain't got the smarts to deal with memory fragmentation, or doesn't want the overhead. Calling talloc_free_children() on a pool, will reset it back to its initial state. So we could in theory maintain a pool of pools, and avoid all runtime mallocs(). The issue is whether the overhead of managing the pool of pools is greater than the overhead of calling malloc.
But maybe it's just overcomplicating things. Worst case you don't want the situation that most requests go 10 octets over 32k - so some debug or similar to tell the admin this has happened would be useful.
The only way to do that currently would be to set a memlimit on the context, which, when exceeded would cause allocations to fail, and the server to exit. Not ideal. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS development team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2