Regex match on a variable is not working if length is long
Hi, I am using freeradius-4.0.x version. This one works, (6) if ( "%{control:Tmp-String-7}" =~ /vendor-options.*\(number 43\)\(value (([0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) { (6) EXPAND %{control:Tmp-String-7} (6) --> ((ClassName Policy)(name client-policy:1,6,00:A0:BC:6C:7E:64)(v4-reply-options [vendor-encapsulated-options])(vendor-options [((ClassName Option)(number 43)(value 01:03:00:A0:BC:14:44:01:00:02:01:03:11:05:00:44:00:44:07:00:43:00:43:01:01:02:01:03:06:07:01:BD:01:BD:01:01:02:01:07:00:87:00:88:01:01:02:01:06:C0:A8:64:00:18:07:00:16:00:17:01:01:02:01:03:11:06:C0:A8:64:00:18:07:00:A1:00:A1:19:01:01:21:01:01:26:38:68:74:74:70:3A:2F:2F:31:30:2E:34:33:2E:31:39:2E:31:38:3A:39:36:37:35:2F:6C:69:76:65:2F:43:50:45:4D:61:6E:61:67:65:72:2F:43:50:45:73:2F:67:65:6E:65:72:69:63:54:52:36:39:27:02:00:05:28:02:07:D0:36:01:FF:40:01:00)(option-definition-set-name ViasatUT))])) (6) update control { But this one is not, (15) if ( "%{control:Tmp-String-7}" =~ /vendor-options.*\(number 43\)\(value (([0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) { (15) EXPAND %{control:Tmp-String-7} (15) --> ((ClassName Policy)(name client-policy:1,6,00:A0:BC:6C:7D:3A)(v4-reply-options [vendor-encapsulated-options])(vendor-options [((ClassName Option)(number 43)(value 01:03:00:A0:BC:02:09:55:54:00:00:00:00:00:00:07:03:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:04:11:55:54:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:05:04:0A:4F:FF:0F:14:44:01:00:02:01:03:11:05:00:44:00:44:07:00:43:00:43:01:01:02:01:03:06:07:01:BD:01:BD:01:01:02:01:07:00:87:00:88:01:01:02:01:06:C0:A8:64:00:18:07:00:16:00:17:01:01:02:01:03:11:06:C0:A8:64:00:18:07:00:A1:00:A1:19:01:01:26:42:68:74:74:70:3A:2F:2F:61:63:73:2E:64:65:76:2E:63:6D:74:2E:76:69:61:73:61:74:2E:69:6F:3A:39:36:37:35:2F:6C:69:76:65:2F:43:50:45:4D:61:6E:61:67:65:72:2F:43:50:45:73:2F:67:65:6E:65:72:69:63:54:52:36:39:27:02:00:05:28:02:07:D0:2A:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2B:04:0A:4F:FF:0F:2C:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:33:0C:55:54:32:00:00:00:00:00:00:00:29:08:36:01:FF:3B:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:3C:12:55:54:32:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:3D:04:0A:4F:FF:0F:3E:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2E:74:61:72:2E:67:7A:3F:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:2E:74:61:72:2E:67:7A:40:01:00)(option-definition-set-name ViasatUT))])) (15) ERROR: regex failed: regex evaluation failed with code (-27): <INVALID> (15) ERROR: Condition evaluation failed because the value of an operand could not be determined (15) ... (15) } Both are same conditions with different values in them. Why second one is not working? Regards, Nagamani Chinnapaiyan
On Dec 11, 2019, at 4:06 AM, Chinnapaiyan, Nagamani <Nagamani.Chinnapaiyan@viasat.com> wrote:
But this one is not,
(15) if ( "%{control:Tmp-String-7}" =~ /vendor-options.*\(number 43\)\(value (([0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) {
(15) EXPAND %{control:Tmp-String-7}
(15) --> ((ClassName Policy)(name client-policy:1,6,00:A0:BC:6C:7D:3A)(v4-reply-options [vendor-encapsulated-options])(vendor-options [((ClassName Option)(number 43)(value 01:03:00:A0:BC:02:09:55:54:00:00:00:00:00:00:07:03:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:04:11:55:54:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:05:04:0A:4F:FF:0F:14:44:01:00:02:01:03:11:05:00:44:00:44:07:00:43:00:43:01:01:02:01:03:06:07:01:BD:01:BD:01:01:02:01:07:00:87:00:88:01:01:02:01:06:C0:A8:64:00:18:07:00:16:00:17:01:01:02:01:03:11:06:C0:A8:64:00:18:07:00:A1:00:A1:19:01:01:26:42:68:74:74:70:3A:2F:2F:61:63:73:2E:64:65:76:2E:63:6D:74:2E:76:69:61:73:61:74:2E:69:6F:3A:39:36:37:35:2F:6C:69:76:65:2F:43:50:45:4D:61:6E:61:67:65:72:2F:43:50:45:73:2F:67:65:6E:65:72:69:63:54:52:36:39:27:02:00:05:28:02:07:D0:2A:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2B:04:0A:4F:FF:0F:2C:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:33:0C:55:54:32:00:00:00:00:00:00:00:29:08:36:01:FF:3B:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:3C:12:55:54:32:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:3D:04:0A:4F:FF:0F:3E:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2E:74:61:72:2E:67:7A:3F:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:2E:74:61:72:2E:67:7A:40:01:00)(option-definition-set-name ViasatUT))]))
(15) ERROR: regex failed: regex evaluation failed with code (-27): <INVALID>
Hmm... I've pushed a change which should get you an actual error message. That may help.
Both are same conditions with different values in them. Why second one is not working?
There's an internal error inside of PCRE. We don't know what error until we see it in a debug message. My guess is that the internal JIT stack for PCRE is too small. One regex will work for small data, but the same regex will overflow the internal JIT stack for large data. There's a line in src/lib/util/regex.c: tls->jit_stack = pcre2_jit_stack_create(32 * 1024, 512 * 1024, tls->gcontext); Change it to something like: tls->jit_stack = pcre2_jit_stack_create(32 * 1024, 4 * 1024 * 1024, tls->gcontext); That will greatly increase the size of the JIT stack. If that fixes it, then that's the problem. Alan DeKok.
On Dec 11, 2019, at 4:06 AM, Chinnapaiyan, Nagamani <Nagamani.Chinnapaiyan@viasat.com> wrote:
This one works,
(6) if ( "%{control:Tmp-String-7}" =~ /vendor-options.*\(number 43\)\(value (([0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) {
One possibility is to fix the expression. You're not using the capture groups, so there's no need to the regex engine to store them. You should be able to use: if (&control:Tmp-String-7 =~ /vendor-options.*\(number 43\)\(value (?:(?:[0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) { Which tells PCRE to not store the capture groups. But it's also not clear why you're doing *parsing* either. If the vendor options are *always* hex, then you don't need to check that. Just change the regular expression so: if ( &control:Tmp-String-7 =~ /vendor-options.*\(number 43\)\(value (.*)/ ) { On the other hand, if you need to parse the "00:00:00" strings and covert them to hex, then it's probably best to avoid regular expressions. They're just not designed for that. Alan DeKok.
On 11 Dec 2019, at 20:48, Alan DeKok <aland@deployingradius.com> wrote:
On Dec 11, 2019, at 4:06 AM, Chinnapaiyan, Nagamani <Nagamani.Chinnapaiyan@viasat.com> wrote:
This one works,
(6) if ( "%{control:Tmp-String-7}" =~ /vendor-options.*\(number 43\)\(value (([0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) {
One possibility is to fix the expression. You're not using the capture groups, so there's no need to the regex engine to store them.
You should be able to use:
if (&control:Tmp-String-7 =~ /vendor-options.*\(number 43\)\(value (?:(?:[0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) {
or it'd be: if (&control:Tmp-String-7 =~ /vendor-options.*\(number 43\)\(value ((?:[0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) { if you wanted to extract the value still. I've pushed a change to increase the JIT stack size, and added string output for the extra error codes, but disabling the inner capture group as Alan suggested, will likely fix the issue. -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Thanks Alan and Arran,
One possibility is to fix the expression. You're not using the capture groups, so there's no need to the regex engine to store them.
You should be able to use:
if (&control:Tmp-String-7 =~ /vendor-options.*\(number 43\)\(value (?:(?:[0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) {
I am using the outer capture group but not the inner one. After removing inner group from capture group, the existing code itself worked. (0) if ( "%{control:Tmp-String-7}" =~ /vendor-options.*\(number 43\)\(value ((?:[0-9a-fA-F]{2}:)*[0-9a-fA-F]{2})/ ) { (0) EXPAND %{control:Tmp-String-7} (0) --> ((ClassName Policy)(name client-policy:1,6,00:A0:BC:6C:7D:3A)(v4-reply-options [vendor-encapsulated-options])(vendor-options [((ClassName Option)(number 43)(value 01:03:00:A0:BC:02:09:55:54:00:00:00:00:00:00:07:03:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:04:11:55:54:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:05:04:0A:4F:FF:0F:14:44:01:00:02:01:03:11:05:00:44:00:44:07:00:43:00:43:01:01:02:01:03:06:07:01:BD:01:BD:01:01:02:01:07:00:87:00:88:01:01:02:01:06:C0:A8:64:00:18:07:00:16:00:17:01:01:02:01:03:11:06:C0:A8:64:00:18:07:00:A1:00:A1:19:01:01:26:42:68:74:74:70:3A:2F:2F:61:63:73:2E:64:65:76:2E:63:6D:74:2E:76:69:61:73:61:74:2E:69:6F:3A:39:36:37:35:2F:6C:69:76:65:2F:43:50:45:4D:61:6E:61:67:65:72:2F:43:50:45:73:2F:67:65:6E:65:72:69:63:54:52:36:39:27:02:00:05:28:02:07:D0:2A:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2B:04:0A:4F:FF:0F:2C:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:33:0C:55:54:32:00:00:00:00:00:00:00:29:08:36:01:FF:3B:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:3C:12:55:54:32:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:3D:04:0A:4F:FF:0F:3E:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2E:74:61:72:2E:67:7A:3F:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:2E:74:61:72:2E:67:7A:40:01:00)(option-definition-set-name ViasatUT))])) (0) update control { (0) EXPAND %{sub:: } (0) (%{sub:: 01:03:00:A0:BC:02:09:55:54:00:00:00:00:00:00:07:03:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:04:11:55:54:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:05:04:0A:4F:FF:0F:14:44:01:00:02:01:03:11:05:00:44:00:44:07:00:43:00:43:01:01:02:01:03:06:07:01:BD:01:BD:01:01:02:01:07:00:87:00:88:01:01:02:01:06:C0:A8:64:00:18:07:00:16:00:17:01:01:02:01:03:11:06:C0:A8:64:00:18:07:00:A1:00:A1:19:01:01:26:42:68:74:74:70:3A:2F:2F:61:63:73:2E:64:65:76:2E:63:6D:74:2E:76:69:61:73:61:74:2E:69:6F:3A:39:36:37:35:2F:6C:69:76:65:2F:43:50:45:4D:61:6E:61:67:65:72:2F:43:50:45:73:2F:67:65:6E:65:72:69:63:54:52:36:39:27:02:00:05:28:02:07:D0:2A:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2B:04:0A:4F:FF:0F:2C:0E:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:33:0C:55:54:32:00:00:00:00:00:00:00:29:08:36:01:FF:3B:0E:55:54:32:00:00:00:00:00:03:07:03:0A:00:09:3C:12:55:54:32:5F:33:2E:37:2E:33:2E:31:30:2E:39:2E:62:69:6E:3D:04:0A:4F:FF:0F:3E:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:35:30:33:2E:74:61:72:2E:67:7A:3F:3C:68:74:74:70:3A:2F:2F:31:30:2E:37:39:2E:32:35:35:2E:31:36:2F:70:75:62:2F:76:77:61:5F:69:6D:61:67:65:73:2F:76:77:61:5F:55:54:2E:35:2E:32:2E:32:2E:31:30:33:39:33:2E:74:61:72:2E:67:7A:40:01:00}) (0) --> 010300A0BC0209555400000000000007030E55543200000000000307030A0009041155545F332E372E332E31302E392E62696E05040A4FFF0F1444010002010311050044004407004300430101020103060701BD01BD0101020107008700880101020106C0A8640018070016001701010201031106C0A86400180700A100A11901012642687474703A2F2F6163732E6465762E636D742E7669617361742E696F3A393637352F6C6976652F4350454D616E616765722F435045732F67656E657269635452363927020005280207D02A0E55542E352E322E322E31303530332B040A4FFF0F2C0E55542E352E322E322E3130333933330C5554320000000000000029083601FF3B0E55543200000000000307030A00093C125554325F332E372E332E31302E392E62696E3D040A4FFF0F3E3C687474703A2F2F31302E37392E3235352E31362F7075622F7677615F696D616765732F7677615F55542E352E322E322E31303530332E7461722E677A3F3C687474703A2F2F31302E37392E3235352E31362F7075622F7677615F696D616765732F7677615F55542E352E322E322E31303339332E7461722E677A400100 (0) &Tmp-String-8 := "010300A0BC0209555400000000000007030E55543200000000000307030A0009041155545F332E372E332E31302E392E62696E05040A4FFF0F1444010002010311050044004407004300430101020103060701BD01BD0101020107008700880101020106C0A8640018070016001701010201031106C0A86400180700A100A11901012642687474703A2F2F6163732E6465762E636D742E7669617361742E696F3A393637352F6C6976652F4350454D616E616765722F435045732F67656E657269635452363927020005280207D02A0E55542E352E322E322E31303530332B040A4FFF0F2C0E55542E352E322E322E3130333933330C5554320000000000000029083601FF3B0E55543200000000000307030A00093C125554325F332E372E332E31302E392E62696E3D040A4FFF0F3E3C687474703A2F2F31302E37392E3235352E31362F7075622F7677615F696D616765732F7677615F55542E352E322E322E31303530332E7461722E677A3F3C687474703A2F2F31302E37392E3235352E31362F7075622F7677615F696D616765732F7677615F55542E352E322E322E31303339332E7461722E677A400100" ... (0) Sending DHCP-Offer XID 1623a79a from 10.43.18.62:6767 to 10.43.18.44:6767 via eth1 (0) DHCP-Vendor = 0x010300a0bc0209555400000000000007030e55543200000000000307030a0009041155545f332e372e332e31302e392e62696e05040a4fff0f1444010002010311050044004407004300430101020103060701bd01bd0101020107008700880101020106c0a8640018070016001701010201031106c0a86400180700a100a11901012642687474703a2f2f6163732e6465762e636d742e7669617361742e696f3a393637352f6c6976652f4350454d616e616765722f435045732f67656e657269635452363927020005280207d02a0e55542e352e322e322e31303530332b040a4fff0f2c0e55542e352e322e322e3130333933330c5554320000000000000029083601ff3b0e55543200000000000307030a00093c125554325f332e372e332e31302e392e62696e3d040a4fff0f3e3c687474703a2f2f31302e37392e3235352e31362f7075622f7677615f696d616765732f7677615f55542e352e322e322e31303530332e7461722e677a3f3c687474703a2f2f31302e37392e3235352e31362f7075622f7677615f696d616765732f7677615f55542e352e322e322e31303339332e7461722e677a400100 ... When we tried to build with the new changes, we got below error, CC src/lib/util/regex.c src/lib/util/regex.c:840:25: error: 'PCRE_ERROR_UNSET' undeclared here (not in a function); did you mean 'PCRE_ERROR_NULL'? { "PCRE_ERROR_UNSET", PCRE_ERROR_UNSET }, ^~~~~~~~~~~~~~~~ PCRE_ERROR_NULL make[1]: *** [build/objs/src/lib/util/regex.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... Regards, Nagamani Chinnapaiyan
On Dec 12, 2019, at 5:25 AM, Chinnapaiyan, Nagamani <Nagamani.Chinnapaiyan@viasat.com> wrote:
When we tried to build with the new changes, we got below error, CC src/lib/util/regex.c src/lib/util/regex.c:840:25: error: 'PCRE_ERROR_UNSET' undeclared here (not in a function); did you mean 'PCRE_ERROR_NULL'? { "PCRE_ERROR_UNSET", PCRE_ERROR_UNSET }, ^~~~~~~~~~~~~~~~ PCRE_ERROR_NULL make[1]: *** [build/objs/src/lib/util/regex.lo] Error 1 make[1]: *** Waiting for unfinished jobs....
We've pushed a fix, thanks. Alan DeKok.
participants (3)
-
Alan DeKok -
Arran Cudbard-Bell -
Chinnapaiyan, Nagamani