rlm_cache_redis

[Gabriel Blanchard]

Oh that's a really good news. Thanks for taking the time to do this

> On May 20, 2015, at 19:14, "Arran Cudbard-Bell" <a.cudbardb at freeradius.org> wrote:
> 
> Hi all,
> 
> There's now a rlm_cache_redis driver that supports caching information in a Redis k/v store in the v3.1.x branch.
> 
> It behaves pretty much how you'd expect in terms of Redis. The cache entry format is a list built with RPUSH made up of map/op/value triplets. I did experiment with HMSET, but in the end figured it wasn't hugely useful, and that the overhead of building the hash wasn't worth the small functionality increase.
> 
> In terms of marshalling/unmarshalling:
> - Octet strings stay as raw binary strings instead of getting converted to hex.
> - Integers are... weird. I tried writing a mini implementation of the redis protocol so I could insert them (hiredis doesn't allow it with redisCommandArgv) but the Redis server kept throwing back errors. So for now, integers are strings.
> - Everything else is the string form you'd see in the debug logs.
> 
> As part of this work rlm_cache was modified to use vp_map_t as its internal cached attribute storage struct, with the LHS being a TMPL_TYPE_ATTR and the RHS being TMPL_TYPE_DATA.
> 
> This exposes the full range of lists (including CoA/DM) and request refs (outer.) as destinations. It also exposes all the filtering operators.
> 
> The behaviour of the cache module is now also more consistent with how you'd expect it to operate. The RHS of the update section is evaluated to create the cache entry, and the LHS is evaluated when merging the cache entry. That's it. No partial merges, no other crazy logic.
> 
> The next few bits of development work related to this are:
> 
>    - Implement redis 3.0 clustering support. This is a priority.
>    - Expose the session state blobs as attributes (couldn't do this in v2.x.x as the attribute buffers weren't large enough)
>    - Remove the code from rlm_eap that deals with storing the session state blobs on disk.
>    - Write a rlm_cache_files driver (to provide similar functionality to that which rlm_eap previously provided)
> 
> Once that's complete, ultra scalable EAP-TLS/TTLS/PEAP deployments become a possibility.
> 
> Feel free to play with/break it, and let me know if you have any feature requests.
> 
> -Arran
> 
> radiusd -Xx
> 
> Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND TMPL XLAT
> Wed May 20 17:55:43 2015 : Debug: %{User-Name}
> Wed May 20 17:55:43 2015 : Debug: Parsed xlat tree:
> Wed May 20 17:55:43 2015 : Debug: attribute --> User-Name
> Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND %{User-Name}
> Wed May 20 17:55:43 2015 : Debug: (0) cache:    --> foo
> Wed May 20 17:55:43 2015 : Debug: rlm_cache (cache): Reserved connection (0)
> Wed May 20 17:55:43 2015 : Debug: (0) cache: LRANGE foo 0 -1
> Wed May 20 17:55:43 2015 : Debug: (0) cache: No cache entry found for "foo"
> Wed May 20 17:55:43 2015 : Debug: (0) cache: Creating new cache entry
> Wed May 20 17:55:43 2015 : Debug: Cache last updated at %t
> Wed May 20 17:55:43 2015 : Debug: Parsed xlat tree:
> Wed May 20 17:55:43 2015 : Debug: literal --> Cache last updated at
> Wed May 20 17:55:43 2015 : Debug: percent --> t
> Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND Cache last updated at %t
> Wed May 20 17:55:43 2015 : Debug: (0) cache:    --> Cache last updated at Wed May 20 17:55:43 2015
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   reply:Reply-Message += Cache last updated at Wed May 20 17:55:43 2015
> Wed May 20 17:55:43 2015 : Debug: %{randstr:ssssssssssssssssssssssssssssssss}
> Wed May 20 17:55:43 2015 : Debug: Parsed xlat tree:
> Wed May 20 17:55:43 2015 : Debug: xlat --> randstr
> Wed May 20 17:55:43 2015 : Debug: {
> Wed May 20 17:55:43 2015 : Debug:    literal --> ssssssssssssssssssssssssssssssss
> Wed May 20 17:55:43 2015 : Debug: }
> Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND %{randstr:ssssssssssssssssssssssssssssssss}
> Wed May 20 17:55:43 2015 : Debug: (0) cache:    --> es/uaTeD0fk3Up8mSgUMDCMJfWgDQAxt
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   reply:Class := 0x65732f756154654430666b335570386d5367554d44434d4a6657674451417874
> Wed May 20 17:55:43 2015 : Debug: (0) cache: Pipelining commands
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   MULTI
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   DEL "foo"
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   argv command
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     RPUSH
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     foo
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     &Cache-Created
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     :=
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     May 20 2015 17:55:43 EDT
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     &Cache-Expires
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     :=
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     May 20 2015 17:55:53 EDT
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     &reply:Reply-Message
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     +=
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     Cache last updated at Wed May 20 17:55:43 2015
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     &reply:Class
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     :=
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     es/uaTeD0fk3Up8mSgUMDCMJfWgDQAxt
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   EXPIREAT "foo" 1432158953
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   EXEC
> Wed May 20 17:55:43 2015 : Debug: (0) cache: Command results
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   (0) status  : OK
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   (1) status  : QUEUED
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   (2) status  : QUEUED
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   (3) status  : QUEUED
> Wed May 20 17:55:43 2015 : Debug: (0) cache:   (4) array[3]
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     (0) integer : 0
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     (1) integer : 12
> Wed May 20 17:55:43 2015 : Debug: (0) cache:     (2) integer : 1
> Wed May 20 17:55:43 2015 : Debug: (0) cache: Commited entry, TTL 10 seconds
> Wed May 20 17:55:43 2015 : Debug: rlm_cache (cache): Released connection (0)
> 
> Wed May 20 17:55:47 2015 : Debug: (1)     modsingle[authorize]: calling cache (rlm_cache) for request 1
> Wed May 20 17:55:47 2015 : Debug: (1) cache: EXPAND TMPL XLAT
> Wed May 20 17:55:47 2015 : Debug: %{User-Name}
> Wed May 20 17:55:47 2015 : Debug: Parsed xlat tree:
> Wed May 20 17:55:47 2015 : Debug: attribute --> User-Name
> Wed May 20 17:55:47 2015 : Debug: (1) cache: EXPAND %{User-Name}
> Wed May 20 17:55:47 2015 : Debug: (1) cache:    --> foo
> Wed May 20 17:55:47 2015 : Debug: rlm_cache (cache): Reserved connection (1)
> Wed May 20 17:55:47 2015 : Debug: (1) cache: LRANGE foo 0 -1
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &Cache-Created
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : :=
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : May 20 2015 17:55:43 EDT
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &Cache-Expires
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : :=
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : May 20 2015 17:55:53 EDT
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &reply:Reply-Message
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : +=
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : Cache last updated at Wed May 20 17:55:43 2015
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &reply:Class
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : :=
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : es/uaTeD0fk3Up8mSgUMDCMJfWgDQAxt
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Found entry for "foo"
> Wed May 20 17:55:47 2015 : Debug: (1) cache: Merging cache entry into request
> Wed May 20 17:55:47 2015 : Debug: (1) cache:   &reply:Reply-Message += Cache last updated at Wed May 20 17:55:43 2015
> Wed May 20 17:55:47 2015 : Debug: (1) cache:   &reply:Class := 0x65732f756154654430666b335570386d5367554d44434d4a6657674451417874
> Wed May 20 17:55:47 2015 : Debug: rlm_cache (cache): Released connection (1)
> Wed May 20 17:55:47 2015 : Debug: (1)     modsingle[authorize]: returned from cache (rlm_cache) for request 1
> Wed May 20 17:55:47 2015 : Debug: (1)     [cache] = updated
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html