rlm_cache_redis

[Arran Cudbard-Bell]

Hi all,

There's now a rlm_cache_redis driver that supports caching information in a Redis k/v store in the v3.1.x branch.

It behaves pretty much how you'd expect in terms of Redis. The cache entry format is a list built with RPUSH made up of map/op/value triplets. I did experiment with HMSET, but in the end figured it wasn't hugely useful, and that the overhead of building the hash wasn't worth the small functionality increase.

In terms of marshalling/unmarshalling:
- Octet strings stay as raw binary strings instead of getting converted to hex.
- Integers are... weird. I tried writing a mini implementation of the redis protocol so I could insert them (hiredis doesn't allow it with redisCommandArgv) but the Redis server kept throwing back errors. So for now, integers are strings.
- Everything else is the string form you'd see in the debug logs.

As part of this work rlm_cache was modified to use vp_map_t as its internal cached attribute storage struct, with the LHS being a TMPL_TYPE_ATTR and the RHS being TMPL_TYPE_DATA.

This exposes the full range of lists (including CoA/DM) and request refs (outer.) as destinations. It also exposes all the filtering operators.

The behaviour of the cache module is now also more consistent with how you'd expect it to operate. The RHS of the update section is evaluated to create the cache entry, and the LHS is evaluated when merging the cache entry. That's it. No partial merges, no other crazy logic.

The next few bits of development work related to this are:

	- Implement redis 3.0 clustering support. This is a priority.
	- Expose the session state blobs as attributes (couldn't do this in v2.x.x as the attribute buffers weren't large enough)
	- Remove the code from rlm_eap that deals with storing the session state blobs on disk.
	- Write a rlm_cache_files driver (to provide similar functionality to that which rlm_eap previously provided)

Once that's complete, ultra scalable EAP-TLS/TTLS/PEAP deployments become a possibility.

Feel free to play with/break it, and let me know if you have any feature requests.

-Arran

radiusd -Xx

Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND TMPL XLAT
Wed May 20 17:55:43 2015 : Debug: %{User-Name}
Wed May 20 17:55:43 2015 : Debug: Parsed xlat tree:
Wed May 20 17:55:43 2015 : Debug: attribute --> User-Name
Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND %{User-Name}
Wed May 20 17:55:43 2015 : Debug: (0) cache:    --> foo
Wed May 20 17:55:43 2015 : Debug: rlm_cache (cache): Reserved connection (0)
Wed May 20 17:55:43 2015 : Debug: (0) cache: LRANGE foo 0 -1
Wed May 20 17:55:43 2015 : Debug: (0) cache: No cache entry found for "foo"
Wed May 20 17:55:43 2015 : Debug: (0) cache: Creating new cache entry
Wed May 20 17:55:43 2015 : Debug: Cache last updated at %t
Wed May 20 17:55:43 2015 : Debug: Parsed xlat tree:
Wed May 20 17:55:43 2015 : Debug: literal --> Cache last updated at
Wed May 20 17:55:43 2015 : Debug: percent --> t
Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND Cache last updated at %t
Wed May 20 17:55:43 2015 : Debug: (0) cache:    --> Cache last updated at Wed May 20 17:55:43 2015
Wed May 20 17:55:43 2015 : Debug: (0) cache:   reply:Reply-Message += Cache last updated at Wed May 20 17:55:43 2015
Wed May 20 17:55:43 2015 : Debug: %{randstr:ssssssssssssssssssssssssssssssss}
Wed May 20 17:55:43 2015 : Debug: Parsed xlat tree:
Wed May 20 17:55:43 2015 : Debug: xlat --> randstr
Wed May 20 17:55:43 2015 : Debug: {
Wed May 20 17:55:43 2015 : Debug: 	literal --> ssssssssssssssssssssssssssssssss
Wed May 20 17:55:43 2015 : Debug: }
Wed May 20 17:55:43 2015 : Debug: (0) cache: EXPAND %{randstr:ssssssssssssssssssssssssssssssss}
Wed May 20 17:55:43 2015 : Debug: (0) cache:    --> es/uaTeD0fk3Up8mSgUMDCMJfWgDQAxt
Wed May 20 17:55:43 2015 : Debug: (0) cache:   reply:Class := 0x65732f756154654430666b335570386d5367554d44434d4a6657674451417874
Wed May 20 17:55:43 2015 : Debug: (0) cache: Pipelining commands
Wed May 20 17:55:43 2015 : Debug: (0) cache:   MULTI
Wed May 20 17:55:43 2015 : Debug: (0) cache:   DEL "foo"
Wed May 20 17:55:43 2015 : Debug: (0) cache:   argv command
Wed May 20 17:55:43 2015 : Debug: (0) cache:     RPUSH
Wed May 20 17:55:43 2015 : Debug: (0) cache:     foo
Wed May 20 17:55:43 2015 : Debug: (0) cache:     &Cache-Created
Wed May 20 17:55:43 2015 : Debug: (0) cache:     :=
Wed May 20 17:55:43 2015 : Debug: (0) cache:     May 20 2015 17:55:43 EDT
Wed May 20 17:55:43 2015 : Debug: (0) cache:     &Cache-Expires
Wed May 20 17:55:43 2015 : Debug: (0) cache:     :=
Wed May 20 17:55:43 2015 : Debug: (0) cache:     May 20 2015 17:55:53 EDT
Wed May 20 17:55:43 2015 : Debug: (0) cache:     &reply:Reply-Message
Wed May 20 17:55:43 2015 : Debug: (0) cache:     +=
Wed May 20 17:55:43 2015 : Debug: (0) cache:     Cache last updated at Wed May 20 17:55:43 2015
Wed May 20 17:55:43 2015 : Debug: (0) cache:     &reply:Class
Wed May 20 17:55:43 2015 : Debug: (0) cache:     :=
Wed May 20 17:55:43 2015 : Debug: (0) cache:     es/uaTeD0fk3Up8mSgUMDCMJfWgDQAxt
Wed May 20 17:55:43 2015 : Debug: (0) cache:   EXPIREAT "foo" 1432158953
Wed May 20 17:55:43 2015 : Debug: (0) cache:   EXEC
Wed May 20 17:55:43 2015 : Debug: (0) cache: Command results
Wed May 20 17:55:43 2015 : Debug: (0) cache:   (0) status  : OK
Wed May 20 17:55:43 2015 : Debug: (0) cache:   (1) status  : QUEUED
Wed May 20 17:55:43 2015 : Debug: (0) cache:   (2) status  : QUEUED
Wed May 20 17:55:43 2015 : Debug: (0) cache:   (3) status  : QUEUED
Wed May 20 17:55:43 2015 : Debug: (0) cache:   (4) array[3]
Wed May 20 17:55:43 2015 : Debug: (0) cache:     (0) integer : 0
Wed May 20 17:55:43 2015 : Debug: (0) cache:     (1) integer : 12
Wed May 20 17:55:43 2015 : Debug: (0) cache:     (2) integer : 1
Wed May 20 17:55:43 2015 : Debug: (0) cache: Commited entry, TTL 10 seconds
Wed May 20 17:55:43 2015 : Debug: rlm_cache (cache): Released connection (0)

Wed May 20 17:55:47 2015 : Debug: (1)     modsingle[authorize]: calling cache (rlm_cache) for request 1
Wed May 20 17:55:47 2015 : Debug: (1) cache: EXPAND TMPL XLAT
Wed May 20 17:55:47 2015 : Debug: %{User-Name}
Wed May 20 17:55:47 2015 : Debug: Parsed xlat tree:
Wed May 20 17:55:47 2015 : Debug: attribute --> User-Name
Wed May 20 17:55:47 2015 : Debug: (1) cache: EXPAND %{User-Name}
Wed May 20 17:55:47 2015 : Debug: (1) cache:    --> foo
Wed May 20 17:55:47 2015 : Debug: rlm_cache (cache): Reserved connection (1)
Wed May 20 17:55:47 2015 : Debug: (1) cache: LRANGE foo 0 -1
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &Cache-Created
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : :=
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : May 20 2015 17:55:43 EDT
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &Cache-Expires
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : :=
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : May 20 2015 17:55:53 EDT
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &reply:Reply-Message
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : +=
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : Cache last updated at Wed May 20 17:55:43 2015
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got key   : &reply:Class
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got op    : :=
Wed May 20 17:55:47 2015 : Debug: (1) cache: Got value : es/uaTeD0fk3Up8mSgUMDCMJfWgDQAxt
Wed May 20 17:55:47 2015 : Debug: (1) cache: Found entry for "foo"
Wed May 20 17:55:47 2015 : Debug: (1) cache: Merging cache entry into request
Wed May 20 17:55:47 2015 : Debug: (1) cache:   &reply:Reply-Message += Cache last updated at Wed May 20 17:55:43 2015
Wed May 20 17:55:47 2015 : Debug: (1) cache:   &reply:Class := 0x65732f756154654430666b335570386d5367554d44434d4a6657674451417874
Wed May 20 17:55:47 2015 : Debug: rlm_cache (cache): Released connection (1)
Wed May 20 17:55:47 2015 : Debug: (1)     modsingle[authorize]: returned from cache (rlm_cache) for request 1
Wed May 20 17:55:47 2015 : Debug: (1)     [cache] = updated
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freeradius.org/pipermail/freeradius-devel/attachments/20150520/26787d4a/attachment.sig>