Hi All, I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS. If anyone has any suggestions regarding other fields they'd like to see included, let me know. Outbound (request): { "<attribute>":{ type:"<type>" , # RADIUS or internal attribute type value:[<values>] # Always an array, may contain string, integer, float, JSON object (nested VP) }, "User-Name":{ type:"string", value:["user@example.org"] } } Inbound (response): { "[qualifiers:]<attribute>":{ op:"<operator>", # One of the standard update construct operators value:<value> # May be array (multivalued, or nested vps (JSON object)), value or JSON object (nested VP) }, # Alternative "[qualifiers:]<attribute>":<value> # Value may be anything but a JSON object, array values may still hold multiple JSON objects } All values will be XLATd. I'm thinking for the response, maybe two optional bools (for the expanded JSON object format): no_xlat - where no_xlat:true disables xlat expansion is_json - where is_json:true disables JSON parsing of the value (you can write JSON structure to an attribute for later parsing) Thoughts? If we can decide on a standard format, future modules will be required to implement it if they're just bridges to an external service (and the external service does not have predefined JSON structures). Note: The outbound request will only use attributes from the request list. If you think attributes should be included from multiple lists, please explain why. -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
1. It is also necessory to standartize representation of primitive values, i.e.. a) IP4, IP6 [variants: string which can be passet to net_pton] b) IP6Prefix [variant ??? -- some standart text representation] c) Mac address [ variant: canonical IEE 802 string representation, i.e. http://en.wikipedia.org/wiki/IEEE_802 ] d) Octets [ base64 string ? ] e) IFID [base64 ? hex string ?, integer ?] ) Note: The outbound request will only use attributes from the request list. If you think attributes should be ) included from multiple lists, please explain why. I.e. this is mean, that if we want to receive value of some attribute from json application server, we must send one at first ? Let we have client, which send DHCP Offer message to radius. Radius must response with IP address, which allocated from pool. Of course incoming message does not contains YOU_IP_ADDRESS attribute. So, it will be impossible to request ip address from json server, or I miss something ? On Tue, Nov 8, 2011 at 3:32 PM, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
Hi All, I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS. If anyone has any suggestions regarding other fields they'd like to see included, let me know.
Outbound (request): { "<attribute>":{ type:"<type>" , # RADIUS or internal attribute type value:[<values>] # Always an array, may contain string, integer, float, JSON object (nested VP) }, "User-Name":{ type:"string", value:["user@example.org"] } } Inbound (response): { "[qualifiers:]<attribute>":{ op:"<operator>", # One of the standard update construct operators value:<value> # May be array (multivalued, or nested vps (JSON object)), value or JSON object (nested VP) }, # Alternative "[qualifiers:]<attribute>":<value> # Value may be anything but a JSON object, array values may still hold multiple JSON objects } All values will be XLATd. I'm thinking for the response, maybe two optional bools (for the expanded JSON object format): no_xlat - where no_xlat:true disables xlat expansion is_json - where is_json:true disables JSON parsing of the value (you can write JSON structure to an attribute for later parsing) Thoughts? If we can decide on a standard format, future modules will be required to implement it if they're just bridges to an external service (and the external service does not have predefined JSON structures). Note: The outbound request will only use attributes from the request list. If you think attributes should be included from multiple lists, please explain why. -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com
Technical consultant and solutions architect
15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
On 8 Nov 2011, at 14:18, Ruslan Shevchenko wrote:
1. It is also necessory to standartize representation of primitive values, i.e..
a) IP4, IP6 [variants: string which can be passet to net_pton] b) IP6Prefix [variant ??? -- some standart text representation] c) Mac address [ variant: canonical IEE 802 string representation, i.e. http://en.wikipedia.org/wiki/IEEE_802 ] d) Octets [ base64 string ? ] e) IFID [base64 ? hex string ?, integer ?]
We only really need to standardise primitives that are interpreted by FreeRADIUS directly. The format should be whatever FreeRADIUS can interpret or encode as RADIUS attributes. a) IPv4 Standard dotted quad without CIDR suffix b) IPv6 Suggestions? c) IPv6-Prefix Suggestions? d) MAC-Address - User/Application defined (not used internally by FreeRADIUS) - Most likely a JSON string though, as I don't believe JSON supports 64bit integers... e) Octets - These are currently encoded as a hex string by vp_prints, but BASE64 would be more efficient. It should be possible to add support for 'automagical' BASE64 parsing for the response, i.e. the server can determine if it's hexits or BASE64. For clarity/debugging I'm still leaning towards hexits in the request. But good point, i'll look into this... f) IFID( do you mean ifIndex?) Integer or String, FreeRADIUS does the conversion automatically
) Note: The outbound request will only use attributes from the request list. If you think attributes should be ) included from multiple lists, please explain why.
I.e. this is mean, that if we want to receive value of some attribute from json application server, we must send one at first ?
No, it means we won't send the contents of the reply, control, proxy*, coa*, dm* lists. Only request. -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On Tue, Nov 8, 2011 at 4:39 PM, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
On 8 Nov 2011, at 14:18, Ruslan Shevchenko wrote:
1. It is also necessory to standartize representation of primitive values, i.e..
a) IP4, IP6 [variants: string which can be passet to net_pton] b) IP6Prefix [variant ??? -- some standart text representation] c) Mac address [ variant: canonical IEE 802 string representation, i.e. http://en.wikipedia.org/wiki/IEEE_802 ] d) Octets [ base64 string ? ] e) IFID [base64 ? hex string ?, integer ?]
We only really need to standardise primitives that are interpreted by FreeRADIUS directly. The format should be whatever FreeRADIUS can interpret or encode as RADIUS attributes.
a) IPv4 Standard dotted quad without CIDR suffix b) IPv6 Suggestions?
Text Representation of Addresses, specified in rfc 1884 (http://tools.ietf.org/html/rfc1884) in paragraph 2.2, chapter 1 (or 2 or 3 -- i.e. three possible representations are defined there, we can choose any)
c) IPv6-Prefix Suggestions?
part of choosed ipv6 representation.
d) MAC-Address - User/Application defined (not used internally by FreeRADIUS) - Most likely a JSON string though, as I don't believe JSON supports 64bit integers...
//Btw, json specs says nothing about possible size of number, so this is implementation depended. Popular json-c library from metaware supports 64-bit integers (and does not support 32 ;) ) But yes, If ip-s are string, than mac-address also must be string [IEE 802].
e) Octets - These are currently encoded as a hex string by vp_prints, but BASE64 would be more efficient. It should be possible to add support for 'automagical' BASE64 parsing for the response, i.e. the server can determine if it's hexits or BASE64. For clarity/debugging I'm still leaning towards hexits in the request. But good point, i'll look into this... f) IFID( do you mean ifIndex?) Integer or String, FreeRADIUS does the conversion automatically
yes.
) Note: The outbound request will only use attributes from the request list. If you think attributes should be ) included from multiple lists, please explain why.
I.e. this is mean, that if we want to receive value of some attribute from json application server, we must send one at first ?
No, it means we won't send the contents of the reply, control, proxy*, coa*, dm* lists. Only request.
Ohh, undestood. Thanks.
-Arran
Arran Cudbard-Bell a.cudbardb@networkradius.com
Technical consultant and solutions architect
15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
Ruslan Shevchenko <ruslan@shevchenko.kiev.ua> writes:
b) IPv6 Suggestions? Text Representation of Addresses, specified in rfc 1884
Which is obsoleted by rfc2373, which again is obsoleted by..... well, it's a rather long and boring story. The current standard is rfc4291. But please don't use that directly. It has far too many options, and too few recommendations. Read rfc5952 instead. http://tools.ietf.org/html/rfc5952 Bjørn
On 08/11/11 12:32, Arran Cudbard-Bell wrote:
Hi All,
I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS. If anyone has any suggestions regarding other fields they'd like to see included, let me know.
What is the application for this? I'll note this syntax doesn't preserve the original inter-attribute ordering in the packet (though I assume it will preserve intra-attribute ordering). Whether this matters or not, I don't know (but what about tagged/group attrs?)
Phil Mayers wrote:
I'll note this syntax doesn't preserve the original inter-attribute ordering in the packet
RFC 2865 mandates that inter-attribute order doesn't matter.
(though I assume it will preserve intra-attribute ordering).
Yes. RFC 2865 mandates that intra-attribute ordering is preserved.
Whether this matters or not, I don't know (but what about tagged/group attrs?)
Tags could be another field in an "attribute" struct. But semantically, attributes with the same tag are grouped together. Tags are really an atrocious hack. My $0.02 is to represent them as fields in an attribute: "<attribute>":{ type:"<type>", value:[<values>] tag:"<integer>" }, Alan DeKok.
On 8 Nov 2011, at 14:34, Alan DeKok wrote:
Phil Mayers wrote:
I'll note this syntax doesn't preserve the original inter-attribute ordering in the packet
RFC 2865 mandates that inter-attribute order doesn't matter.
(though I assume it will preserve intra-attribute ordering).
Yes. RFC 2865 mandates that intra-attribute ordering is preserved.
Whether this matters or not, I don't know (but what about tagged/group attrs?)
Tags could be another field in an "attribute" struct. But semantically, attributes with the same tag are grouped together.
Tags are really an atrocious hack.
My $0.02 is to represent them as fields in an attribute:
"<attribute>":{ type:"<type>", value:[<values>] tag:"<integer>" },
Hmm, no, i'd rather have something directly tied to the value seeing as multiple values can be encoded, and i'd rather have roughly similar formats for request and response... What about: "<attribute>":{ "value":[[<value>,<tag>],<value>,[<value>,<tag>]] } I don't see that there's any other possible use for nested arrays in the value key? -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On Tue, Nov 08, 2011 at 03:04:23PM +0100, Arran Cudbard-Bell wrote:
What about:
"<attribute>":{ "value":[[<value>,<tag>],<value>,[<value>,<tag>]] }
I don't see that there's any other possible use for nested arrays in the value key?
Possibly the update operator (=, +=, :=, !* etc). As for tags, the first option which springs to mind is to put them in the attribute label, as they are in FreeRadius today. However with tags, the update operators don't behave how I'd expect. If I write Tunnel-Server-Endpoint:0 := 1.2.3.4 Tunnel-Server-Endpoint:1 := 5.6.7.8 then the second := erases all previous instances of Tunnel-Server-Endpoint. The tag behaves not as part of the attribute, but as part of the value (which in fact it is). This isn't a problem with standard FreeRadius where the AV pairs are always returned in some order - apart from the limitation that there's no facility to replace just Tunnel-Server-Endpoint:1. But it would be a problem for this JSON: { "Tunnel-Server-Endpoint:0": [["1.2.3.4",":="]], "Tunnel-Server-Endpoint:1": [["5.6.7.8",":="]] } because many JSON consumers don't guarantee the order in which object members are returned. A more convenient way to handle tags would be as object keys: { "Tunnel-Server-Endpoint": {"0"=>"1.2.3.4", "1"=>"5.6.7.8"} } But this still has the problem of element ordering given the same example as above: { "Tunnel-Server-Endpoint": {"0"=>["1.2.3.4",":="], "1"=>["5.6.7.8",":="]} } Semantically, having repeated := is not very useful. Normally you'd have := on the first value and += on the others. Can we rely on this? That is, apply := to the whole attribute and have it expand to := on the first value and += on the rest? Then += would apply as += to all values. But when you start to consider other update operators, it would prohibit cases like this: Reply-Message += "foo" Reply-Message -= "bar" and the only way I can see to support that is that firstly every value needs to be tagged with an operator, and secondly the values must be in an array so that they are processed in a deterministic sequence. If not, you get non-deterministic behaviour in this sort of edge case (admittedly useless): Reply-Message += "foo" Reply-Message -= "foo" Final point: if you are returning a bundle of attributes, currently these are always *merged* into the reply packet. You can delete individual attributes using -= or !* operators, but there is no way to say "replace the entire reply list built so far" without invoking something like attr_filter in unlang. This is no worse for rlm_rest than we have today, but I'd really like some way to return maybe a magic attribute which either deletes all attributes, or all attributes apart from those listed in the reply, or triggers a specific entry in rlm_attr_filter. But this is going outside the realm of JSON formatting. Regards, Brian.
Brian Candler wrote:
As for tags, the first option which springs to mind is to put them in the attribute label, as they are in FreeRadius today.
I'd really like to avoid that. It makes it much more difficult to handle attributes.
However with tags, the update operators don't behave how I'd expect. If I write
Tunnel-Server-Endpoint:0 := 1.2.3.4 Tunnel-Server-Endpoint:1 := 5.6.7.8
then the second := erases all previous instances of Tunnel-Server-Endpoint. The tag behaves not as part of the attribute, but as part of the value (which in fact it is).
Yes, that is a problem.
A more convenient way to handle tags would be as object keys:
{ "Tunnel-Server-Endpoint": {"0"=>"1.2.3.4", "1"=>"5.6.7.8"} }
That might be better.
Final point: if you are returning a bundle of attributes, currently these are always *merged* into the reply packet. You can delete individual attributes using -= or !* operators, but there is no way to say "replace the entire reply list built so far" without invoking something like attr_filter in unlang.
Yeah. It would be nice to have some kind of language describing how to process the two lists (current, new). But that's hard.
This is no worse for rlm_rest than we have today, but I'd really like some way to return maybe a magic attribute which either deletes all attributes, or all attributes apart from those listed in the reply, or triggers a specific entry in rlm_attr_filter.
But this is going outside the realm of JSON formatting.
Yes. Suggest a syntax... Alan DeKok.
On Wed, Nov 09, 2011 at 02:30:11PM +0100, Alan DeKok wrote:
Yeah. It would be nice to have some kind of language describing how to process the two lists (current, new). But that's hard.
This is no worse for rlm_rest than we have today, but I'd really like some way to return maybe a magic attribute which either deletes all attributes, or all attributes apart from those listed in the reply, or triggers a specific entry in rlm_attr_filter.
But this is going outside the realm of JSON formatting.
Yes. Suggest a syntax...
Well... modules currently return lists of <attribute> <op> <value> and I think we should stick with that, so that this feature could be used from all existing modules like rlm_sql and rlm_files. Modules can set control attributes as part of the 'check' phase, since at that point operators := = += are treated as applying to the control list. So it could be as simple as: steve Huntgroup =~ "foo", Pre-Apply-Module := attr_filter.deleteall Tunnel-Server-Endpoint:0 := 1.2.3.4 steve Framed-Protocol := PPP That is: if any control:Pre-Apply-Module attributes exist then those modules are invoked before the packet is updated by merging in the control and reply lists. Some care is needed to prevent infinite recursion, and maybe this feature needs to be enabled individually per module for security. This does raise an additional thought I have about rlm_rest. Is it going to be able to return updates to both control and reply lists? The simplest approach would be to tag attributes as control: and request:. But otherwise it could return separate members, i.e. hash of {listname:{attribute:value}} Also, to be a full replacement for rlm_sql I think it would need to be able to return a sequence of control/reply lists, with the control elements tested in turn and the first match used (unless Fall-Through = Yes is set). That is, ignoring the details, something like [{ "control":{ "Huntgroup": [["foo", "=~"]], "Pre-Apply-Module:=": [["attr_filter.deleteall", ":="]] }, "reply":{ "Tunnel-Server-Endpoint": {"0" => [["1.2.3.4", ":=']]} } }, { "reply":{ "Framed-Protocol": "PPP" } }] Using a control list in this way, where the attributes are in an unordered JSON object, implies that it doesn't matter in which order the tests are applied. If you want this to be deterministic you'd have to go back to arrays of [attr,op,value]. Or is it conceived that logic like this should exist entirely in the HTTP server which rlm_rest submits to, which just returns one set of updates to be applied? I think it would be nice to be able to point to a dumb HTTP or CouchDB server like this though. Regards, Brian.
On 10 Nov 2011, at 14:40, Brian Candler wrote:
On Wed, Nov 09, 2011 at 02:30:11PM +0100, Alan DeKok wrote:
Yeah. It would be nice to have some kind of language describing how to process the two lists (current, new). But that's hard.
This is no worse for rlm_rest than we have today, but I'd really like some way to return maybe a magic attribute which either deletes all attributes, or all attributes apart from those listed in the reply, or triggers a specific entry in rlm_attr_filter.
But this is going outside the realm of JSON formatting.
Yes. Suggest a syntax...
Well... modules currently return lists of <attribute> <op> <value> and I think we should stick with that, so that this feature could be used from all existing modules like rlm_sql and rlm_files.
Modules can set control attributes as part of the 'check' phase, since at that point operators := = += are treated as applying to the control list.
So it could be as simple as:
steve Huntgroup =~ "foo", Pre-Apply-Module := attr_filter.deleteall Tunnel-Server-Endpoint:0 := 1.2.3.4
steve Framed-Protocol := PPP
That is: if any control:Pre-Apply-Module attributes exist then those modules are invoked before the packet is updated by merging in the control and reply lists.
Some care is needed to prevent infinite recursion, and maybe this feature needs to be enabled individually per module for security.
This does raise an additional thought I have about rlm_rest. Is it going to be able to return updates to both control and reply lists?
Yes
The simplest approach would be to tag attributes as control: and request:. But otherwise it could return separate members, i.e. hash of {listname:{attribute:value}}
Also, to be a full replacement for rlm_sql I think it would need to be able to return a sequence of control/reply lists, with the control elements tested in turn and the first match used (unless Fall-Through = Yes is set). That is, ignoring the details, something like
[{ "control":{ "Huntgroup": [["foo", "=~"]], "Pre-Apply-Module:=": [["attr_filter.deleteall", ":="]] }, "reply":{ "Tunnel-Server-Endpoint": {"0" => [["1.2.3.4", ":=']]} } }, { "reply":{ "Framed-Protocol": "PPP" } }]
Using a control list in this way, where the attributes are in an unordered JSON object, implies that it doesn't matter in which order the tests are applied. If you want this to be deterministic you'd have to go back to arrays of [attr,op,value].
Or is it conceived that logic like this should exist entirely in the HTTP server which rlm_rest submits to, which just returns one set of updates to be applied? I think it would be nice to be able to point to a dumb HTTP or CouchDB server like this though.
Mmm no. The server sends a complete request list to the remote application, the application can figure out what it wants to modify. If users want to admins want to expose more than the request list they can copy the values across... In the reply allowing the outer JSON object to specify lists does make sense though, shouldn't be too hard to do for rlm_rest. -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On 10/11/11 14:54, Arran Cudbard-Bell wrote:
If users want to admins want to expose more than the request list they can copy the values across...
Ugh. So rlm_sql all over again... please no!
In the reply allowing the outer JSON object to specify lists does make sense though, shouldn't be too hard to do for rlm_rest.
On 10 Nov 2011, at 16:48, Phil Mayers wrote:
On 10/11/11 14:54, Arran Cudbard-Bell wrote:
If users want to admins want to expose more than the request list they can copy the values across...
Ugh. So rlm_sql all over again... please no!
*if admins want to So you want to do what? Include the entire control list with the password hashes? -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On 10/11/11 16:23, Arran Cudbard-Bell wrote:
On 10 Nov 2011, at 16:48, Phil Mayers wrote:
On 10/11/11 14:54, Arran Cudbard-Bell wrote:
If users want to admins want to expose more than the request list they can copy the values across...
Ugh. So rlm_sql all over again... please no!
*if admins want to
So you want to do what? Include the entire control list with the password hashes?
A common use case for similar modules is to EXTRACT the password hash from some database, and give it to FreeRADIUS. So there's a good chance a lot of password hashes will be flying around in the returned JSON. If they're doing PAP, the User-Password field will be in the JSON. And so on. So unless you're going to provide an attribute filtering mechanism (good luck supporting that on the mailing list!) you're going to have to face attribute privacy issues anyway. Does rlm_rest support HTTPS? Anyway, no - that's not what I want. What I *don't* want is to have to do this: update request { Var = "%{reply:Var}" Var2 = "%{control:Var2}" } How about a module config item: modules { rest myserver { # which attributes do we send to the server request_pairs = yes control_pairs = no reply_pairs = no } } However: I'm bikeshedding. There's precisely zero chance I'd use this module, so feel free to ignore me. Cheers, Phil
On 10 Nov 2011, at 17:49, Phil Mayers wrote:
On 10/11/11 16:23, Arran Cudbard-Bell wrote:
On 10 Nov 2011, at 16:48, Phil Mayers wrote:
On 10/11/11 14:54, Arran Cudbard-Bell wrote:
If users want to admins want to expose more than the request list they can copy the values across...
Ugh. So rlm_sql all over again... please no!
*if admins want to
So you want to do what? Include the entire control list with the password hashes?
A common use case for similar modules is to EXTRACT the password hash from some database, and give it to FreeRADIUS. So there's a good chance a lot of password hashes will be flying around in the returned JSON.
If they're doing PAP, the User-Password field will be in the JSON. And so on.
So unless you're going to provide an attribute filtering mechanism (good luck supporting that on the mailing list!) you're going to have to face attribute privacy issues anyway. Does rlm_rest support HTTPS?
Yes, I was planning to use attrfilter. Yes it will support HTTPS. And I disagree with most of what you said above... This is not like rlm_sql which calls a relatively simple dumb backend database. This module is meant to bridge FreeRADIUS with a webservice, a webservice is as smart or dumb as its author makes it.
Anyway, no - that's not what I want. What I *don't* want is to have to do this:
Why do you want to include reply items in the request body? Why do you want to include control items in the request body?
update request { Var = "%{reply:Var}" Var2 = "%{control:Var2}" }
How about a module config item:
modules { rest myserver { # which attributes do we send to the server request_pairs = yes control_pairs = no reply_pairs = no } }
No. -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On Thu, Nov 10, 2011 at 09:59:02PM +0100, Arran Cudbard-Bell wrote:
Why do you want to include reply items in the request body?
I can give you a concrete usage case for this. You've built up a partial reply so far, by looking up attributes for a user in some database. Now you want to supplement that with a policy decision, implemented by rlm_rest. The policy says: if this user has a static IP address, then tunnel them to a farm of static LNSes (unless the request was from one of those LNSes).
Why do you want to include control items in the request body?
That one I don't have a case for, but the control list is sometimes a convenient dumping ground for temporary variables. These could go in the request list instead. But I also think there is value in making the input and output JSON formats identical. If the reply can set attributes in control and reply lists, then I think it's reasonable that the request should be able to carry those lists too. Regards, Brian.
On 11 Nov 2011, at 00:13, Brian Candler wrote:
On Thu, Nov 10, 2011 at 09:59:02PM +0100, Arran Cudbard-Bell wrote:
Why do you want to include reply items in the request body?
I can give you a concrete usage case for this.
You've built up a partial reply so far, by looking up attributes for a user in some database. Now you want to supplement that with a policy decision, implemented by rlm_rest.
The policy says: if this user has a static IP address, then tunnel them to a farm of static LNSes (unless the request was from one of those LNSes).
Hmm fair enough. Well i've been discussing some changes with Alan that would allow VALUE_PAIR to carry a 'source/target' list. I.e. you can create a temporary list of VPs, and then pass that to pairmove with a request, which'll sort them into the correct list. But you could also write the source list in there, and then include that as either an encapsulating JSON object "request":{} or just as part of the attribute name "request:<attr>". Then attrfilter needs to be modified to take calls from other modules, and the attributes you want to include from other lists go in there. reply:<name> *= ANY User-Name *= ANY Tmp-String-0 := "My extra special web API attribute that you only see here"
Why do you want to include control items in the request body?
That one I don't have a case for, but the control list is sometimes a convenient dumping ground for temporary variables. These could go in the request list instead.
They can also go in header data which is what I was hinting at before. For rlm_rest this would be something like: uri = "http://example.org/user/%{User-Name}?auth-type=%{control:Auth-Type}"
But I also think there is value in making the input and output JSON formats identical. If the reply can set attributes in control and reply lists, then I think it's reasonable that the request should be able to carry those lists too.
Yes. -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On 8 Nov 2011, at 14:25, Phil Mayers wrote:
On 08/11/11 12:32, Arran Cudbard-Bell wrote:
Hi All,
I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS. If anyone has any suggestions regarding other fields they'd like to see included, let me know.
What is the application for this?
https://github.com/arr2036/freeradius-server/tree/rlm_rest/src/modules/rlm_r... And future transports that may use JSON like ZMQ. Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On Tue, Nov 08, 2011 at 01:32:12PM +0100, Arran Cudbard-Bell wrote:
I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS. If anyone has any suggestions regarding other fields they'd like to see included, let me know.
I think we should keep it simple, and I would also like to see it symmetrical for data in and out of a module. If a regular FreeRadius output has Service-Type = Framed-User Framed-Protocol = PPP then a starting point would be: #1 {"Service-Type": "Framed-User", "Framed-Protocol": "PPP"} That is, I don't see a need to include an explicit Type, since the receiver will presumably have a dictionary; and the value can be whatever format is used when FreeRadius normally inputs or outputs a string of that type (e.g. radclient, rlm_sql etc) Then there is the requirement for keeping ordering between repeated attributes. Simple options like #2 [{"Service-Type": "Framed-User"}, {"Framed-Protocol": "PPP"}, {"Reply-Message": "Foo"}, {"Reply-Message": "Bar"}] #3 [["Service-Type": "Framed-User"], ["Framed-Protocol": "PPP"], ["Reply-Message": "Foo"], ["Reply-Message": "Bar"]] are direct but inconvenient when you want to look up the value for a particular attribute. So I would go as you suggest: #4 {"Service-Type":["Framed-User"], "Framed-Protocol":["PPP"], "Reply-Message":["Foo","Bar"]} Aside: it would be convenient to permit modules which generate this format to be allowed to omit the [array] for single-valued attributes. But the above would be the normalised form which FreeRadius itself would always generate.
Inbound (response):
RADIUS packets themselves of course don't include "+=" or ":=" or "=", but I guess this format needs to work for internal communication between modules and FR. So there are several options I can think of: #5 - Non-extensible {"Service-Type": [{"+=": "Framed-User"}], "Framed-Protocol": [{":=": "PPP"}]}, "Reply-Message": [{"+=": "Foo"}, {"+=": "Bar"}]} #6 - Verbose but extensible {"Service-Type": [{"op":"+=","value":"Framed-User"}], "Framed-Protocol": [{"op":":=","value":"PPP"}]} "Reply-Message": [{"op":"+=","value":"Foo"}, {"op":"+=","value":"Bar"}]} 5 and 6 are butt-ugly and should be discounted for that reason alone. #7 - Like #3 with third value. Missing third value implies "+=" [["Service-Type", "Framed-User", "+="], ["Framed-Protocol", "PPP", ":="], ["Reply-Message", "Foo", "+="], ["Reply-Message", "Bar", "+="]] Easy to use when generating responses, but I hate the asymmetry with #4. So how about including the update operator in the label? #8 {"Service-Type+=": "Framed-User", "Framed-Protocol:=": "PPP", "Reply-Message+=": ["Foo","Bar"]} So basically, "+=" would always add (and is perhaps the default?), ":=" would always erase the list first and then add, and = would do nothing if the attribute already exists. And so {"Reply-Message:=",[]} could be used to remove an attribute; perhaps also value null could do that. Finally, I'd also like to be able to update request/control/reply lists in one go, and the simplest way I can think of is: #9 {"request:Huntgroup-Name:=", "Foo", "reply:Framed-Protocol+=", "PPP"} The list: prefix would be optional and could be implied from context normally (e.g. default to request: list for queries and reply: for responses) Final idea: if you want the labels to be easier to parse, split them on a character which is unlikely to occur. #10 {"request|Huntgroup-Name|:=", "Foo", "reply|Framed-Protocol|+=", "PPP"} Just a few random thoughts :-) Regards, Brian.
On 8 Nov 2011, at 16:30, Brian Candler wrote:
On Tue, Nov 08, 2011 at 01:32:12PM +0100, Arran Cudbard-Bell wrote:
I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS. If anyone has any suggestions regarding other fields they'd like to see included, let me know.
I think we should keep it simple, and I would also like to see it symmetrical for data in and out of a module.
If a regular FreeRadius output has
Service-Type = Framed-User Framed-Protocol = PPP
then a starting point would be:
#1 {"Service-Type": "Framed-User", "Framed-Protocol": "PPP"}
That is, I don't see a need to include an explicit Type, since the receiver will presumably have a dictionary; and the value can be whatever format is used when FreeRadius normally inputs or outputs a string of that type (e.g. radclient, rlm_sql etc)
Then there is the requirement for keeping ordering between repeated attributes. Simple options like
#2 [{"Service-Type": "Framed-User"}, {"Framed-Protocol": "PPP"}, {"Reply-Message": "Foo"}, {"Reply-Message": "Bar"}]
#3 [["Service-Type": "Framed-User"], ["Framed-Protocol": "PPP"], ["Reply-Message": "Foo"], ["Reply-Message": "Bar"]]
are direct but inconvenient when you want to look up the value for a particular attribute. So I would go as you suggest:
#4 {"Service-Type":["Framed-User"], "Framed-Protocol":["PPP"], "Reply-Message":["Foo","Bar"]}
Aside: it would be convenient to permit modules which generate this format to be allowed to omit the [array] for single-valued attributes. But the above would be the normalised form which FreeRadius itself would always generate.
Inbound (response):
RADIUS packets themselves of course don't include "+=" or ":=" or "=", but I guess this format needs to work for internal communication between modules and FR.
So there are several options I can think of:
#5 - Non-extensible {"Service-Type": [{"+=": "Framed-User"}], "Framed-Protocol": [{":=": "PPP"}]}, "Reply-Message": [{"+=": "Foo"}, {"+=": "Bar"}]}
#6 - Verbose but extensible {"Service-Type": [{"op":"+=","value":"Framed-User"}], "Framed-Protocol": [{"op":":=","value":"PPP"}]} "Reply-Message": [{"op":"+=","value":"Foo"}, {"op":"+=","value":"Bar"}]}
5 and 6 are butt-ugly and should be discounted for that reason alone.
#7 - Like #3 with third value. Missing third value implies "+="
[["Service-Type", "Framed-User", "+="], ["Framed-Protocol", "PPP", ":="], ["Reply-Message", "Foo", "+="], ["Reply-Message", "Bar", "+="]]
Easy to use when generating responses, but I hate the asymmetry with #4.
So how about including the update operator in the label?
#8 {"Service-Type+=": "Framed-User", "Framed-Protocol:=": "PPP", "Reply-Message+=": ["Foo","Bar"]}
So basically, "+=" would always add (and is perhaps the default?), ":=" would always erase the list first and then add, and = would do nothing if the attribute already exists. And so
{"Reply-Message:=",[]}
could be used to remove an attribute; perhaps also value null could do that.
Finally, I'd also like to be able to update request/control/reply lists in one go, and the simplest way I can think of is:
#9 {"request:Huntgroup-Name:=", "Foo", "reply:Framed-Protocol+=", "PPP"}
The list: prefix would be optional and could be implied from context normally (e.g. default to request: list for queries and reply: for responses)
Final idea: if you want the labels to be easier to parse, split them on a character which is unlikely to occur.
#10 {"request|Huntgroup-Name|:=", "Foo", "reply|Framed-Protocol|+=", "PPP"}
Just a few random thoughts :-)
Regards,
Brian. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
Phew... Lots of feedback. Ok. #1 It's useful for validation purposes, I agree, it's not strictly needed but I don't see the harm in it. I don't think servers will have dictionaries, but I think they'll probably know the types of the attributes they're interested in #2 Most JSON decoders don't support duplicate keys, which is the reason behind aggregating multiple values together in one declaration. #3 Hmm, no, attribute names should be in a hash for easy and fast lookup by the receiver. #4 Yes the original format with type removed, but this doesn't allow for tags. The standard is the standard, it makes it easier for the client to parse attributes and allows nested arrays to be used to specify more complex values. If you made the outer array optional, you couldn't have nested arrays. #5, #6 Json-C does support iterating over hash keys so the duplicate keys are not an issue for received data. Using objects in values to specify operators breaks support for nested VPs. I actually like the fact that there's a single op for a collection of values, it makes it really easy to write filters based on -= and the regex ops. For rlm_rest I added some special behaviour for := and = where it'll automatically switch to += after the first value in the list has been processed, this behaviour should be specified somewhere. #7 I guess there's no need for hash based lookups on the server... Possibly. #8 #9 Difficult to parse and a little fuggly #10 Yes in theory it does help if they're trying to do handcrafted JSON responses. Maybe support both? Just substitute pipes for colons in the name. Of course if they're trying to return text strings with commas they're going to run into the same issues ;) -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
Arran Cudbard-Bell <a.cudbardb@freeradius.org> writes:
I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS.
I would suggest studying the unlang and users manual pages in detail, and at least make sure that whatever format you choose can be used to express everything possible within the server today. I.e. DO NOT limit yourself to 2 lists when the server maintains 7... Just my .02 greek euros Bjørn
On 8 Nov 2011, at 16:53, Bjørn Mork wrote:
Arran Cudbard-Bell <a.cudbardb@freeradius.org> writes:
I'm proposing the following JSON structures for VP lists being sent from and parsed by FreeRADIUS.
I would suggest studying the unlang and users manual pages in detail, and at least make sure that whatever format you choose can be used to express everything possible within the server today. I.e. DO NOT limit yourself to 2 lists when the server maintains 7...
Just my .02 greek euros
The request, proxy-reply, coa and dm lists are the only ones which will not contain locally generated content. I guess request can be substituted where appropriate. It's actually quite hard to include multiple lists in the request codewise. Regarding unlang, the format proposed is actually *more* flexible than the current unlang 'update' constructs :) Running the values through XLAT also exposes all kinds of additional awesomeness. For example if you wanted to increase the debug level for a particular request you could just include { "control:Tmp-String-0":"%{debug:3}" } in the response. I guess it'd be nice to have { "void":"%{debug:3}" } Where the expansion is done but the VP is never added... -Arran Arran Cudbard-Bell a.cudbardb@networkradius.com Technical consultant and solutions architect 15 Ave. du Granier, Meylan, France +33 4 69 66 54 50
On Tue, Nov 08, 2011 at 05:18:59PM +0100, Arran Cudbard-Bell wrote:
Regarding unlang, the format proposed is actually *more* flexible than the current unlang 'update' constructs :)
I would hope that changes in future - and if I had enough time I would try to implement it myself (the main issue being getting my head around the two distinct sets of operator code) I think there could be a feature that every attribute could also identify the list, with a default list based on the context for tidiness and backwards compatibility. Example 'users' file: steve control:Huntgroup-Name =~ "foo", reply:Framed-IP-Address =~ "." request:Huntgroup-Name := "bar" reply:Framed-Protocol := PPP Here we would default to request: or control: in the first line, dependent on the operator; and default to reply: for the remaining lines. In rlm_mysql: attributes returned can have a list prefix, but similarly default to request:/control: in authorize_check_query, and to reply: in authorize_reply_query In unlang: if (control:Huntgroup-Name =~ /foo/ && reply:Framed-IP-Address =~ /./) { update { request:Huntgroup-Name := "bar" reply:Service-Type = PPP } } Some of my configs would be hugely simplified if I did this. At the moment I have to copy attributes from the Reply list to the Request list, just so that rlm_sql can test on their presence.
Running the values through XLAT also exposes all kinds of additional awesomeness.
And dangers :-) But allowing that is part of the semantics of rlm_rest I think, rather than the JSON format itself. rlm_rest could have a flag to enable or disable this. Personally I don't like expansions to have side-effects. If the debug level were instead controlled by an attribute on the control: list, then you could just return a regular update. Regards, Brian.
participants (6)
-
Alan DeKok -
Arran Cudbard-Bell -
Bjørn Mork -
Brian Candler -
Phil Mayers -
Ruslan Shevchenko