Proxying accounting to create a 'tee'

[John Morrissey]

On Tue, Aug 25, 2009 at 10:49:21AM +0100, Arran Cudbard-Bell wrote:
> I know it sounds a little clunky, but another option could be to use a
> chain of detail readers/writers? If you set the primary detail reader load
> factor to 100% the actual delay is likely to be pretty minimal...
> 
> So you'd have:
> 
> NAS->Outer Server->Detail Writer (Primary)->Detail Reader->Detail Writer Queue 1
> 							 ->Detail Writer Queue 2
> 							 ->Detail Writer Queue n.
> 
> Detail Reader Queue 1 -> Proxy Server
> Detail Reader Queue 2 -> Proxy Server
> Detail Reader Queue n -> Proxy Server
> 
> That way the NAS always receives a response, and you get pseudo parallel
> Accounting requests going to the proxy server.
> 
> To balance between the detail writers you can use the load-balance unlang
> stanza, or just the expressions module with the modulo operator.

I settled on something similar to this. The outer server (processing
requests from the NAS) uses redundant-load-balance to write round-robin
across several (currently 5) detail files.

Five detail listeners (one for each detail file) then feed data to their
final destinations (remote proxies, SQL databases, etc.).

It turned out a bit neater than having the outer server write a single file
that's exploded by a dedicated detail listener into several files that, in
turn, each have a detail listener that actually processes the detail. FWIW,
it turned out to be impossible to implement that way since detail listeners
won't write to a detail files, even if the output file is different from the
input file.

Thanks again for the idea Arran, I'm glad it worked out.

john
-- 
John Morrissey          _o            /\         ----  __o
jwm at horde.net        _-< \_          /  \       ----  <  \,
www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__