3.0.x rlm_sql mime encoding UTF8 characters
Hello, Long time lurker. This is the first time I've not been able to solve a problem by reading the notes in config or looking at answers on this list. Apologies if I've missed a thread/explanation/fix and this wastes someones time, or if my understanding is lacking. I'm having a problem with the safe chars check in rlm_sql (postgresql driver) mime encoding characters that are not in it's safe_characters list (i.e. it's doing what it's supposed to). I've run into this recently with users from the European continent using accented characters (or umlauts, cyrillic text etc). The usernames are stored on my server in UTF8 encoding, but the SQL the module runs is obviously not matching the username if it contains a character not in the safe_characters list. I note several old threads on this subject, one of which suggests that the safe chars check is not UTF8 capable (at version 2.1.x). My production server is running 2.1.10 so, accordingly, I have installed v3.0.5 to test. I have tried with a default postgresql server config setup and also changing the safe_characters variable in mods-config/sql/main/postgresql/queries.conf to add special characters: safe_characters = "@àáâãäåæçèéêëìíîïðñòóôõöøùúûüýÿabcdefghijklmnopqrstuvwxyzÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝŸABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /" … and tested with: User-Name = 'êllô@foo.com' … but in both cases the ê and ô are still mime encoded in the SQL select: (0) sql: SQL-User-Name set to 'êllô@foo.com' rlm_sql (sql): Reserved connection (4) (0) sql: EXPAND SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id (0) sql: --> SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id rlm_sql (sql): Executing query: 'SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id' rlm_sql_postgresql: Status: PGRES_TUPLES_OK rlm_sql_postgresql: query affected rows = 0 , fields = 5 (0) sql: EXPAND SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority (0) sql: --> SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority rlm_sql (sql): Executing query: 'SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority' rlm_sql_postgresql: Status: PGRES_TUPLES_OK rlm_sql_postgresql: query affected rows = 0 , fields = 1 (0) sql: User not found in any groups My questions are: Is there a way for me to get rlm_sql to accept UTF8 characters? Or even ignore that check (at my own risk)? Am I expecting too much from this module and should I instead look at encoding the username prior to it arriving on the server so it passes the safe chars check? Or maybe preprocessing the username to escape/encode it before it is parsed by the rlm_sql module? Any help greatly appreciated, Adam FULL DEBUG OUTPUT: radiusd: FreeRADIUS Version 3.0.5 (git #aa75660), for host x86_64-unknown-linux-gnu, built on Oct 24 2014 at 18:00:57 Copyright (C) 1999-2014 The FreeRADIUS server project and contributors There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE You may redistribute copies of FreeRADIUS under the terms of the GNU General Public License For more information about these matters, see the file named COPYRIGHT Starting - reading configuration files ... including dictionary file /usr/local/share/freeradius/dictionary including dictionary file /usr/local/share/freeradius/dictionary.dhcp including dictionary file /usr/local/share/freeradius/dictionary.vqp including dictionary file /etc/raddb/dictionary including configuration file /etc/raddb/radiusd.conf including configuration file /etc/raddb/proxy.conf including configuration file /etc/raddb/clients.conf including files in directory /etc/raddb/mods-enabled/ including configuration file /etc/raddb/mods-enabled/mschap including configuration file /etc/raddb/mods-enabled/cache_eap including configuration file /etc/raddb/mods-enabled/expr including configuration file /etc/raddb/mods-enabled/ntlm_auth including configuration file /etc/raddb/mods-enabled/unpack including configuration file /etc/raddb/mods-enabled/replicate including configuration file /etc/raddb/mods-enabled/soh including configuration file /etc/raddb/mods-enabled/digest including configuration file /etc/raddb/mods-enabled/always including configuration file /etc/raddb/mods-enabled/logintime including configuration file /etc/raddb/mods-enabled/files including configuration file /etc/raddb/mods-enabled/attr_filter including configuration file /etc/raddb/mods-enabled/echo including configuration file /etc/raddb/mods-enabled/sradutmp including configuration file /etc/raddb/mods-enabled/dynamic_clients including configuration file /etc/raddb/mods-enabled/chap including configuration file /etc/raddb/mods-enabled/unix including configuration file /etc/raddb/mods-enabled/detail.log including configuration file /etc/raddb/mods-enabled/preprocess including configuration file /etc/raddb/mods-enabled/radutmp including configuration file /etc/raddb/mods-enabled/pap including configuration file /etc/raddb/mods-enabled/dhcp including configuration file /etc/raddb/mods-enabled/exec including configuration file /etc/raddb/mods-enabled/passwd including configuration file /etc/raddb/mods-enabled/realm including configuration file /etc/raddb/mods-enabled/detail including configuration file /etc/raddb/mods-enabled/linelog including configuration file /etc/raddb/mods-enabled/expiration including configuration file /etc/raddb/mods-enabled/sql including configuration file /etc/raddb/mods-config/sql/main/postgresql/queries.conf including configuration file /etc/raddb/mods-enabled/eap including configuration file /etc/raddb/mods-enabled/utf8 including files in directory /etc/raddb/policy.d/ including configuration file /etc/raddb/policy.d/control including configuration file /etc/raddb/policy.d/debug including configuration file /etc/raddb/policy.d/abfab-tr including configuration file /etc/raddb/policy.d/canonicalization including configuration file /etc/raddb/policy.d/dhcp including configuration file /etc/raddb/policy.d/operator-name including configuration file /etc/raddb/policy.d/accounting including configuration file /etc/raddb/policy.d/cui including configuration file /etc/raddb/policy.d/filter including configuration file /etc/raddb/policy.d/eap including files in directory /etc/raddb/sites-enabled/ including configuration file /etc/raddb/sites-enabled/default main { security { allow_core_dumps = no } } main { name = "radiusd" prefix = "/usr/local" localstatedir = "/usr/local/var" sbindir = "/usr/local/sbin" logdir = "/usr/local/var/log/radius" run_dir = "/usr/local/var/run/radiusd" libdir = "/usr/local/lib" radacctdir = "/usr/local/var/log/radius/radacct" hostname_lookups = no max_request_time = 30 cleanup_delay = 5 max_requests = 1024 pidfile = "/usr/local/var/run/radiusd/radiusd.pid" checkrad = "/usr/local/sbin/checkrad" debug_level = 0 proxy_requests = yes log { stripped_names = no auth = no auth_badpass = no auth_goodpass = no colourise = yes msg_denied = "You are already logged in - access denied" } security { max_attributes = 200 reject_delay = 1 status_server = yes allow_vulnerable_openssl = "yes" } } radiusd: #### Loading Realms and Home Servers #### proxy server { retry_delay = 5 retry_count = 3 default_fallback = no dead_time = 120 wake_all_if_all_dead = no } home_server localhost { ipaddr = 127.0.0.1 port = 1812 type = "auth" secret = <<< secret >>> response_window = 20.000000 response_timeouts = 1 max_outstanding = 65536 zombie_period = 40 status_check = "status-server" ping_interval = 30 check_interval = 30 check_timeout = 4 num_answers_to_alive = 3 revive_interval = 120 coa { irt = 2 mrt = 16 mrc = 5 mrd = 30 } limit { max_connections = 16 max_requests = 0 lifetime = 0 idle_timeout = 0 } } home_server_pool my_auth_failover { type = fail-over home_server = localhost } realm example.com { auth_pool = my_auth_failover } realm LOCAL { } radiusd: #### Loading Clients #### client 0.0.0.0/0 { ipaddr = * require_message_authenticator = no secret = <<< secret >>> limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } radiusd: #### Instantiating modules #### instantiate { } modules { # Loaded module rlm_mschap # Instantiating module "mschap" from file /etc/raddb/mods-enabled/mschap mschap { use_mppe = yes require_encryption = no require_strong = no with_ntdomain_hack = yes passchange { } allow_retry = yes } # Loaded module rlm_cache # Instantiating module "cache_eap" from file /etc/raddb/mods-enabled/cache_eap cache cache_eap { key = "%{%{control:State}:-%{%{reply:State}:-%{State}}}" ttl = 15 max_entries = 16384 epoch = 0 add_stats = no } # Loaded module rlm_expr # Instantiating module "expr" from file /etc/raddb/mods-enabled/expr expr { safe_characters = "@øabcdefghijklmnopqrstuvwxyzØABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /" } # Loaded module rlm_exec # Instantiating module "ntlm_auth" from file /etc/raddb/mods-enabled/ntlm_auth exec ntlm_auth { wait = yes program = "/path/to/ntlm_auth --request-nt-key --domain=MYDOMAIN --username=%{mschap:User-Name} --password=%{User-Password}" shell_escape = yes } # Loaded module rlm_unpack # Instantiating module "unpack" from file /etc/raddb/mods-enabled/unpack # Loaded module rlm_replicate # Instantiating module "replicate" from file /etc/raddb/mods-enabled/replicate # Loaded module rlm_soh # Instantiating module "soh" from file /etc/raddb/mods-enabled/soh soh { dhcp = yes } # Loaded module rlm_digest # Instantiating module "digest" from file /etc/raddb/mods-enabled/digest # Loaded module rlm_always # Instantiating module "reject" from file /etc/raddb/mods-enabled/always always reject { rcode = "reject" simulcount = 0 mpp = no } # Instantiating module "fail" from file /etc/raddb/mods-enabled/always always fail { rcode = "fail" simulcount = 0 mpp = no } # Instantiating module "ok" from file /etc/raddb/mods-enabled/always always ok { rcode = "ok" simulcount = 0 mpp = no } # Instantiating module "handled" from file /etc/raddb/mods-enabled/always always handled { rcode = "handled" simulcount = 0 mpp = no } # Instantiating module "invalid" from file /etc/raddb/mods-enabled/always always invalid { rcode = "invalid" simulcount = 0 mpp = no } # Instantiating module "userlock" from file /etc/raddb/mods-enabled/always always userlock { rcode = "userlock" simulcount = 0 mpp = no } # Instantiating module "notfound" from file /etc/raddb/mods-enabled/always always notfound { rcode = "notfound" simulcount = 0 mpp = no } # Instantiating module "noop" from file /etc/raddb/mods-enabled/always always noop { rcode = "noop" simulcount = 0 mpp = no } # Instantiating module "updated" from file /etc/raddb/mods-enabled/always always updated { rcode = "updated" simulcount = 0 mpp = no } # Loaded module rlm_logintime # Instantiating module "logintime" from file /etc/raddb/mods-enabled/logintime logintime { minimum_timeout = 60 } # Loaded module rlm_files # Instantiating module "files" from file /etc/raddb/mods-enabled/files files { filename = "/etc/raddb/mods-config/files/authorize" usersfile = "/etc/raddb/mods-config/files/authorize" acctusersfile = "/etc/raddb/mods-config/files/accounting" preproxy_usersfile = "/etc/raddb/mods-config/files/pre-proxy" compat = "cistron" } reading pairlist file /etc/raddb/mods-config/files/authorize [/etc/raddb/mods-config/files/authorize]:180 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:187 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:194 Cistron compatibility checks for entry DEFAULT ... reading pairlist file /etc/raddb/mods-config/files/authorize [/etc/raddb/mods-config/files/authorize]:180 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:187 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:194 Cistron compatibility checks for entry DEFAULT ... reading pairlist file /etc/raddb/mods-config/files/accounting reading pairlist file /etc/raddb/mods-config/files/pre-proxy # Loaded module rlm_attr_filter # Instantiating module "attr_filter.post-proxy" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.post-proxy { filename = "/etc/raddb/mods-config/attr_filter/post-proxy" key = "%{Realm}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/post-proxy # Instantiating module "attr_filter.pre-proxy" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.pre-proxy { filename = "/etc/raddb/mods-config/attr_filter/pre-proxy" key = "%{Realm}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/pre-proxy # Instantiating module "attr_filter.access_reject" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.access_reject { filename = "/etc/raddb/mods-config/attr_filter/access_reject" key = "%{User-Name}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/access_reject # Instantiating module "attr_filter.access_challenge" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.access_challenge { filename = "/etc/raddb/mods-config/attr_filter/access_challenge" key = "%{User-Name}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/access_challenge # Instantiating module "attr_filter.accounting_response" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.accounting_response { filename = "/etc/raddb/mods-config/attr_filter/accounting_response" key = "%{User-Name}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/accounting_response # Instantiating module "echo" from file /etc/raddb/mods-enabled/echo exec echo { wait = yes program = "/bin/echo %{User-Name}" input_pairs = "request" output_pairs = "reply" shell_escape = yes } # Loaded module rlm_radutmp # Instantiating module "sradutmp" from file /etc/raddb/mods-enabled/sradutmp radutmp sradutmp { filename = "/usr/local/var/log/radius/sradutmp" username = "%{User-Name}" case_sensitive = yes check_with_nas = yes permissions = 420 caller_id = no } # Loaded module rlm_dynamic_clients # Instantiating module "dynamic_clients" from file /etc/raddb/mods-enabled/dynamic_clients # Loaded module rlm_chap # Instantiating module "chap" from file /etc/raddb/mods-enabled/chap # Loaded module rlm_unix # Instantiating module "unix" from file /etc/raddb/mods-enabled/unix unix { radwtmp = "/usr/local/var/log/radius/radwtmp" } # Loaded module rlm_detail # Instantiating module "auth_log" from file /etc/raddb/mods-enabled/detail.log detail auth_log { filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/auth-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } rlm_detail (auth_log): 'User-Password' suppressed, will not appear in detail output # Instantiating module "reply_log" from file /etc/raddb/mods-enabled/detail.log detail reply_log { filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/reply-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Instantiating module "pre_proxy_log" from file /etc/raddb/mods-enabled/detail.log detail pre_proxy_log { filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/pre-proxy-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Instantiating module "post_proxy_log" from file /etc/raddb/mods-enabled/detail.log detail post_proxy_log { filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/post-proxy-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Loaded module rlm_preprocess # Instantiating module "preprocess" from file /etc/raddb/mods-enabled/preprocess preprocess { huntgroups = "/etc/raddb/mods-config/preprocess/huntgroups" hints = "/etc/raddb/mods-config/preprocess/hints" with_ascend_hack = no ascend_channels_per_line = 23 with_ntdomain_hack = no with_specialix_jetstream_hack = no with_cisco_vsa_hack = no with_alvarion_vsa_hack = no } reading pairlist file /etc/raddb/mods-config/preprocess/huntgroups reading pairlist file /etc/raddb/mods-config/preprocess/hints # Instantiating module "radutmp" from file /etc/raddb/mods-enabled/radutmp radutmp { filename = "/usr/local/var/log/radius/radutmp" username = "%{User-Name}" case_sensitive = yes check_with_nas = yes permissions = 384 caller_id = yes } # Loaded module rlm_pap # Instantiating module "pap" from file /etc/raddb/mods-enabled/pap pap { normalise = yes } # Loaded module rlm_dhcp # Instantiating module "dhcp" from file /etc/raddb/mods-enabled/dhcp # Instantiating module "exec" from file /etc/raddb/mods-enabled/exec exec { wait = no input_pairs = "request" shell_escape = yes timeout = 10 } # Loaded module rlm_passwd # Instantiating module "etc_passwd" from file /etc/raddb/mods-enabled/passwd passwd etc_passwd { filename = "/etc/passwd" format = "*User-Name:Crypt-Password:" delimiter = ":" ignore_nislike = no ignore_empty = yes allow_multiple_keys = no hash_size = 100 } rlm_passwd: nfields: 3 keyfield 0(User-Name) listable: no # Loaded module rlm_realm # Instantiating module "IPASS" from file /etc/raddb/mods-enabled/realm realm IPASS { format = "prefix" delimiter = "/" ignore_default = no ignore_null = no } # Instantiating module "suffix" from file /etc/raddb/mods-enabled/realm realm suffix { format = "suffix" delimiter = "@" ignore_default = no ignore_null = no } # Instantiating module "realmpercent" from file /etc/raddb/mods-enabled/realm realm realmpercent { format = "suffix" delimiter = "%" ignore_default = no ignore_null = no } # Instantiating module "ntdomain" from file /etc/raddb/mods-enabled/realm realm ntdomain { format = "prefix" delimiter = "\\" ignore_default = no ignore_null = no } # Instantiating module "detail" from file /etc/raddb/mods-enabled/detail detail { filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Loaded module rlm_linelog # Instantiating module "linelog" from file /etc/raddb/mods-enabled/linelog linelog { filename = "/usr/local/var/log/radius/linelog" permissions = 384 format = "This is a log message for %{User-Name}" reference = "messages.%{%{Packet-Type}:-default}" } # Instantiating module "log_accounting" from file /etc/raddb/mods-enabled/linelog linelog log_accounting { filename = "/usr/local/var/log/radius/linelog-accounting" permissions = 384 format = "" reference = "Accounting-Request.%{%{Acct-Status-Type}:-unknown}" } # Loaded module rlm_expiration # Instantiating module "expiration" from file /etc/raddb/mods-enabled/expiration # Loaded module rlm_sql # Instantiating module "sql" from file /etc/raddb/mods-enabled/sql sql { driver = "rlm_sql_postgresql" server = "10.0.10.4" port = "5432" login = "jt_radius" password = <<< secret >>> radius_db = "jt" read_groups = yes read_profiles = yes read_clients = no delete_stale_sessions = yes sql_user_name = "%{%{Stripped-User-Name}:-%{%{User-Name}:-none}}" default_user_profile = "" client_query = "SELECT id, nasname, shortname, type, secret, server FROM nas" authorize_check_query = "SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id" authorize_reply_query = "SELECT id, UserName, Attribute, Value, Op FROM radreply WHERE Username = '%{SQL-User-Name}' ORDER BY id" authorize_group_check_query = "SELECT id, GroupName, Attribute, Value, op FROM radgroupcheck WHERE GroupName = '%{Sql-Group}' ORDER BY id" authorize_group_reply_query = "SELECT id, GroupName, Attribute, Value, op FROM radgroupreply WHERE GroupName = '%{Sql-Group}' ORDER BY id" group_membership_query = "SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority" simul_count_query = "" simul_verify_query = "" safe_characters = "@àáâãäåæçèéêëìíîïðñòóôõöøùúûüýÿabcdefghijklmnopqrstuvwxyzÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝŸABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /" accounting { reference = "%{tolower:type.%{%{Acct-Status-Type}:-none}.query}" type { accounting-on { query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime))), AcctTerminateCause = '%{%{Acct-Terminate-Cause}:-NAS-Reboot}', WHERE AcctStopTime IS NULL AND NASIPAddress= '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}' AND AcctStartTime <= '%S'::timestamp" } accounting-off { query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime))), AcctTerminateCause = '%{%{Acct-Terminate-Cause}:-NAS-Reboot}', WHERE AcctStopTime IS NULL AND NASIPAddress= '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}' AND AcctStartTime <= '%S'::timestamp" } start { query = "INSERT INTO radacct (AcctSessionId, AcctUniqueId, UserName, Realm, NASIPAddress, NASPortId, NASPortType, AcctStartTime, AcctUpdateTime, AcctStopTime, AcctSessionTime, AcctAuthentic, ConnectInfo_start, ConnectInfo_Stop, AcctInputOctets, AcctOutputOctets, CalledStationId, CallingStationId, AcctTerminateCause, ServiceType, FramedProtocol, FramedIpAddress) VALUES('%{Acct-Session-Id}', '%{Acct-Unique-Session-Id}', '%{SQL-User-Name}', NULLIF('%{Realm}', ''), '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}', %{%{NAS-Port}:-NULL}, '%{NAS-Port-Type}', TO_TIMESTAMP(%{integer:Event-Timestamp}), TO_TIMESTAMP(%{integer:Event-Timestamp}), NULL, 0, '%{Acct-Authentic}', '%{Connect-Info}', NULL, 0, 0, '%{Called-Station-Id}', '%{Calling-Station-Id}', NULL, '%{Service-Type}', '%{Framed-Protocol}', NULLIF('%{Framed-IP-Address}', '')::inet)" } interim-update { query = "UPDATE radacct SET FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, AcctSessionTime = %{%{Acct-Session-Time}:-NULL}, AcctInterval = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM (COALESCE(AcctUpdateTime, AcctStartTime)))), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctInputOctets = (('%{%{Acct-Input-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Input-Octets}:-0}'::bigint), AcctOutputOctets = (('%{%{Acct-Output-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Output-Octets}:-0}'::bigint) WHERE (AcctUniqueId = '%{Acct-Unique-Session-Id}') AND AcctStopTime IS NULL" } stop { query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = COALESCE(%{%{Acct-Session-Time}:-NULL}, (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime)))), AcctInputOctets = (('%{%{Acct-Input-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Input-Octets}:-0}'::bigint), AcctOutputOctets = (('%{%{Acct-Output-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Output-Octets}:-0}'::bigint), AcctTerminateCause = '%{Acct-Terminate-Cause}', FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, ConnectInfo_stop = '%{Connect-Info}' WHERE (AcctUniqueId = '%{Acct-Unique-Session-Id}') AND AcctStopTime IS NULL" } } } post-auth { reference = ".query" query = "INSERT INTO radpostauth (username, pass, reply, authdate) VALUES('%{User-Name}', '%{%{User-Password}:-Chap-Password}', '%{reply:Packet-Type}', NOW())" } } postgresql { send_application_name = no } rlm_sql (sql): Driver rlm_sql_postgresql (module rlm_sql_postgresql) loaded and linked rlm_sql (sql): Attempting to connect to database "jt" rlm_sql (sql): Initialising connection pool pool { start = 5 min = 4 max = 32 spare = 3 uses = 0 lifetime = 0 cleanup_interval = 30 idle_timeout = 60 retry_delay = 1 spread = no } rlm_sql (sql): Opening additional connection (0) rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4' Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10252 rlm_sql (sql): Opening additional connection (1) rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4' Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10253 rlm_sql (sql): Opening additional connection (2) rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4' Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10254 rlm_sql (sql): Opening additional connection (3) rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4' Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10255 rlm_sql (sql): Opening additional connection (4) rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4' Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10256 # Loaded module rlm_eap # Instantiating module "eap" from file /etc/raddb/mods-enabled/eap eap { default_eap_type = "md5" timer_expire = 60 ignore_unknown_eap_types = no mod_accounting_username_bug = no max_sessions = 1024 } # Linked to sub-module rlm_eap_md5 # Linked to sub-module rlm_eap_leap # Linked to sub-module rlm_eap_gtc gtc { challenge = "Password: " auth_type = "PAP" } # Linked to sub-module rlm_eap_tls tls { tls = "tls-common" } tls-config tls-common { rsa_key_exchange = no dh_key_exchange = yes rsa_key_length = 512 dh_key_length = 512 verify_depth = 0 ca_path = "/etc/raddb/certs" pem_file_type = yes private_key_file = "/etc/raddb/certs/server.pem" certificate_file = "/etc/raddb/certs/server.pem" ca_file = "/etc/raddb/certs/ca.pem" private_key_password = <<< secret >>> dh_file = "/etc/raddb/certs/dh" fragment_size = 1024 include_length = yes check_crl = no cipher_list = "DEFAULT" ecdh_curve = "prime256v1" cache { enable = yes lifetime = 24 max_entries = 255 } verify { } ocsp { enable = no override_cert_url = yes url = "http://127.0.0.1/ocsp/" use_nonce = yes timeout = 0 softfail = yes } } # Linked to sub-module rlm_eap_ttls ttls { tls = "tls-common" default_eap_type = "md5" copy_request_to_tunnel = no use_tunneled_reply = no virtual_server = "inner-tunnel" include_length = yes require_client_cert = no } Using cached TLS configuration from previous invocation # Linked to sub-module rlm_eap_peap peap { tls = "tls-common" default_method = "mschapv2" copy_request_to_tunnel = no use_tunneled_reply = no proxy_tunneled_request_as_eap = yes virtual_server = "inner-tunnel" soh = no require_client_cert = no } Using cached TLS configuration from previous invocation # Linked to sub-module rlm_eap_mschapv2 mschapv2 { with_ntdomain_hack = no send_error = no } # Loaded module rlm_utf8 # Instantiating module "utf8" from file /etc/raddb/mods-enabled/utf8 } # modules radiusd: #### Loading Virtual Servers #### server { # from file /etc/raddb/radiusd.conf } # server server default { # from file /etc/raddb/sites-enabled/default # Loading authenticate {...} # Loading authorize {...} # Loading accounting {...} # Loading post-proxy {...} # Loading post-auth {...} } # server default radiusd: #### Opening IP addresses and Ports #### listen { type = "auth" ipaddr = * port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "acct" ipaddr = * port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "auth" ipv6addr = :: port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "acct" ipv6addr = :: port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } Listening on auth address * port 1812 as server default Listening on acct address * port 1813 as server default Listening on auth address :: port 1812 as server default Listening on acct address :: port 1813 as server default Opening new proxy socket 'proxy address * port 0' Listening on proxy address * port 60147 Ready to process requests Received Access-Request Id 179 from 193.35.56.122:36176 to 10.0.10.5:1812 length 140 NAS-Port-Type = Wireless-802.11 Calling-Station-Id = '55-44-33-22-11-00' Called-Station-Id = '00-11-22-33-44-55' User-Name = 'êllô@foo.com' User-Password = 'bakingsoda' NAS-Port = 2153775108 Acct-Session-Id = '12345qwert' Framed-IP-Address = 192.168.1.1 Service-Type = Login-User NAS-Identifier = 'test' NAS-IP-Address = 1.2.3.4 (0) Received Access-Request packet from host 193.35.56.122 port 36176, id=179, length=140 (0) NAS-Port-Type = Wireless-802.11 (0) Calling-Station-Id = '55-44-33-22-11-00' (0) Called-Station-Id = '00-11-22-33-44-55' (0) User-Name = 'êllô@foo.com' (0) User-Password = 'bakingsoda' (0) NAS-Port = 2153775108 (0) Acct-Session-Id = '12345qwert' (0) Framed-IP-Address = 192.168.1.1 (0) Service-Type = Login-User (0) NAS-Identifier = 'test' (0) NAS-IP-Address = 1.2.3.4 (0) # Executing section authorize from file /etc/raddb/sites-enabled/default (0) authorize { (0) sql: EXPAND %{%{Stripped-User-Name}:-%{%{User-Name}:-none}} (0) sql: --> êllô@foo.com (0) sql: SQL-User-Name set to 'êllô@foo.com' rlm_sql (sql): Reserved connection (4) (0) sql: EXPAND SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id (0) sql: --> SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id rlm_sql (sql): Executing query: 'SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id' rlm_sql_postgresql: Status: PGRES_TUPLES_OK rlm_sql_postgresql: query affected rows = 0 , fields = 5 (0) sql: EXPAND SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority (0) sql: --> SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority rlm_sql (sql): Executing query: 'SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority' rlm_sql_postgresql: Status: PGRES_TUPLES_OK rlm_sql_postgresql: query affected rows = 0 , fields = 1 (0) sql: User not found in any groups rlm_sql (sql): Released connection (4) (0) [sql] = notfound (0) pap: WARNING: No "known good" password found for the user. Not setting Auth-Type (0) pap: WARNING: Authentication will fail unless a "known good" password is available (0) [pap] = noop (0) } # authorize = noop (0) ERROR: No Auth-Type found: rejecting the user via Post-Auth-Type = Reject (0) Failed to authenticate the user
On 27/10/14 14:31, Adam Hammond wrote:
Is there a way for me to get rlm_sql to accept UTF8 characters? Or even ignore that check (at my own risk)?
I suspect the problem is that rlm_sql safe_characters check treats the input and safe string as 1-octet wide strings. UTF8 is, of course, multibyte. So, passing: safe_characters = "£" ...is actually passing safe_characters = "\xC2\xA3" ...and will be interpreted as such, a 2-char string. Likewise, a radius value containing that will search safe_characters for \xC2 then \xA3 as separate characters. I'm kind of surprised it hasn't worked given the above, but I don't know what strchr() does with utf8 input to the "s" argument, and I suspect it's locale-dependent. Rather than fix safe_characters, there should be a way to per-instance escape using functions like PQescapeLiteral and PQescapeStringConn, not something hardcoded into FreeRADIUS. I had patches to to this, but they bit-rotted.
Phil Mayers wrote:
I'm kind of surprised it hasn't worked given the above, but I don't know what strchr() does with utf8 input to the "s" argument, and I suspect it's locale-dependent.
Yes.
Rather than fix safe_characters, there should be a way to per-instance escape using functions like PQescapeLiteral and PQescapeStringConn, not something hardcoded into FreeRADIUS. I had patches to to this, but they bit-rotted.
That's probably worth re-doing. In the mean time, I think it's worth just allowing UTF-8 characters. Alan DeKok.
Adam Hammond wrote:
I'm having a problem with the safe chars check in rlm_sql (postgresql driver) mime encoding characters that are not in it's safe_characters list (i.e. it's doing what it's supposed to). I've run into this recently with users from the European continent using accented characters (or umlauts, cyrillic text etc). The usernames are stored on my server in UTF8 encoding, but the SQL the module runs is obviously not matching the username if it contains a character not in the safe_characters list.
I note several old threads on this subject, one of which suggests that the safe chars check is not UTF8 capable (at version 2.1.x). My production server is running 2.1.10 so, accordingly, I have installed v3.0.5 to test.
The simplest solution would be to treat the input as UTF-8. For backwards compatibility, this would have to be configurable. Alan DeKok.
Hi, On Mon, Oct 27, 2014 at 4:31 PM, Adam Hammond <adam.hammond@wicoms.com> wrote: ...
My questions are:
Is there a way for me to get rlm_sql to accept UTF8 characters? Or even ignore that check (at my own risk)?
I was faced with the same problem today (version 2.5.5). I managed to get it working by replacing the function 'sql_escape_func' in 'rlm_sql.c' with the function 'sql_utf8_escape_func' from 'rlm_sql_log.c' file. This seems to pass basic tests, but I am not sure what are the implications of this change. HTH, Isaac B.
Isaac Boukris wrote:
I was faced with the same problem today (version 2.5.5).
I managed to get it working by replacing the function 'sql_escape_func' in 'rlm_sql.c' with the function 'sql_utf8_escape_func' from 'rlm_sql_log.c' file.
This seems to pass basic tests, but I am not sure what are the implications of this change.
Not much. The best thing is probably to treat the "safe-characters" string as ASCII. Then, allow any multi-byte UTF-89 character. I'll put a fix in before the next release. Alan DeKok.
On 2 Nov 2014, at 08:39, Alan DeKok <aland@deployingradius.com> wrote:
Isaac Boukris wrote:
I was faced with the same problem today (version 2.5.5).
I managed to get it working by replacing the function 'sql_escape_func' in 'rlm_sql.c' with the function 'sql_utf8_escape_func' from 'rlm_sql_log.c' file.
This seems to pass basic tests, but I am not sure what are the implications of this change.
Not much. The best thing is probably to treat the "safe-characters" string as ASCII. Then, allow any multi-byte UTF-89 character.
I'll put a fix in before the next release.
I've pushed a fix to process UTF8 chars correctly for the %{escape:} and %{unescape:} xlats. I'm guessing the decision to pass through UTF8 chars here is more a matter of convenience for the administrator than not wanting to write the code :) -Arran Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS development team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Arran Cudbard-Bell wrote:
I've pushed a fix to process UTF8 chars correctly for the %{escape:} and %{unescape:} xlats.
OK.
I'm guessing the decision to pass through UTF8 chars here is more a matter of convenience for the administrator than not wanting to write the code :)
O(N^2) string comparisons are bad. The problematic SQL characters are ASCII. They need to be escaped. The rest can be passed through to SQL. Alan DeKok.
On 2 Nov 2014, at 17:40, Alan DeKok <aland@deployingradius.com> wrote:
Arran Cudbard-Bell wrote:
I've pushed a fix to process UTF8 chars correctly for the %{escape:} and %{unescape:} xlats.
OK.
I'm guessing the decision to pass through UTF8 chars here is more a matter of convenience for the administrator than not wanting to write the code :)
O(N^2) string comparisons are bad.
Well, yes.
The problematic SQL characters are ASCII. They need to be escaped. The rest can be passed through to SQL.
It might be a good idea add an additional (optional) callback for the drivers, to call driver specific escape functions. If one of those functions is registered for a driver, we don't call sql_escape_func. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS development team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Adam Hammond wrote:
I note several old threads on this subject, one of which suggests that the safe chars check is not UTF8 capable (at version 2.1.x). My production server is running 2.1.10 so, accordingly, I have installed v3.0.5 to test.
I've pushed a fix for v2 and v3. The "safe-characters" check is now only for ASCII characters. All non-ASCII UTF-8 characters are allowed. Alan DeKok.
participants (5)
-
Adam Hammond -
Alan DeKok -
Arran Cudbard-Bell -
Isaac Boukris -
Phil Mayers