3.0.x rlm_sql mime encoding UTF8 characters

27 Oct 2014

      Hello,

Long time lurker. This is the first time I've not been able to solve a problem by reading the notes in config or looking at answers on this list. Apologies if I've missed a thread/explanation/fix and this wastes someones time, or if my understanding is lacking.

I'm having a problem with the safe chars check in rlm_sql (postgresql driver) mime encoding characters that are not in it's safe_characters list (i.e. it's doing what it's supposed to). I've run into this recently with users from the European continent using accented characters (or umlauts, cyrillic text etc). The usernames are stored on my server in UTF8 encoding, but the SQL the module runs is obviously not matching the username if it contains a character not in the safe_characters list.

I note several old threads on this subject, one of which suggests that the safe chars check is not UTF8 capable (at version 2.1.x). My production server is running 2.1.10 so, accordingly, I have installed v3.0.5 to test.

I have tried with a default postgresql server config setup and also changing the safe_characters variable in mods-config/sql/main/postgresql/queries.conf to add special characters:

safe_characters = "@àáâãäåæçèéêëìíîïðñòóôõöøùúûüýÿabcdefghijklmnopqrstuvwxyzÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝŸABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /"

… and tested with:

User-Name = 'êllô@foo.com'

… but in both cases the ê and ô are still mime encoded in the SQL select:

(0) sql: SQL-User-Name set to 'êllô@foo.com'
rlm_sql (sql): Reserved connection (4)
(0) sql: EXPAND SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id
(0) sql:    --> SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id
rlm_sql (sql): Executing query: 'SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id'
rlm_sql_postgresql: Status: PGRES_TUPLES_OK
rlm_sql_postgresql: query affected rows = 0 , fields = 5
(0) sql: EXPAND SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority
(0) sql:    --> SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority
rlm_sql (sql): Executing query: 'SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority'
rlm_sql_postgresql: Status: PGRES_TUPLES_OK
rlm_sql_postgresql: query affected rows = 0 , fields = 1
(0) sql: User not found in any groups

My questions are: 

Is there a way for me to get rlm_sql to accept UTF8 characters? Or even ignore that check (at my own risk)?

Am I expecting too much from this module and should I instead look at encoding the username prior to it arriving on the server so it passes the safe chars check? Or maybe preprocessing the username to escape/encode it before it is parsed by the rlm_sql module?

Any help greatly appreciated,
Adam

FULL DEBUG OUTPUT:

radiusd: FreeRADIUS Version 3.0.5 (git #aa75660), for host x86_64-unknown-linux-gnu, built on Oct 24 2014 at 18:00:57
Copyright (C) 1999-2014 The FreeRADIUS server project and contributors
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE
You may redistribute copies of FreeRADIUS under the terms of the
GNU General Public License
For more information about these matters, see the file named COPYRIGHT
Starting - reading configuration files ...
including dictionary file /usr/local/share/freeradius/dictionary
including dictionary file /usr/local/share/freeradius/dictionary.dhcp
including dictionary file /usr/local/share/freeradius/dictionary.vqp
including dictionary file /etc/raddb/dictionary
including configuration file /etc/raddb/radiusd.conf
including configuration file /etc/raddb/proxy.conf
including configuration file /etc/raddb/clients.conf
including files in directory /etc/raddb/mods-enabled/
including configuration file /etc/raddb/mods-enabled/mschap
including configuration file /etc/raddb/mods-enabled/cache_eap
including configuration file /etc/raddb/mods-enabled/expr
including configuration file /etc/raddb/mods-enabled/ntlm_auth
including configuration file /etc/raddb/mods-enabled/unpack
including configuration file /etc/raddb/mods-enabled/replicate
including configuration file /etc/raddb/mods-enabled/soh
including configuration file /etc/raddb/mods-enabled/digest
including configuration file /etc/raddb/mods-enabled/always
including configuration file /etc/raddb/mods-enabled/logintime
including configuration file /etc/raddb/mods-enabled/files
including configuration file /etc/raddb/mods-enabled/attr_filter
including configuration file /etc/raddb/mods-enabled/echo
including configuration file /etc/raddb/mods-enabled/sradutmp
including configuration file /etc/raddb/mods-enabled/dynamic_clients
including configuration file /etc/raddb/mods-enabled/chap
including configuration file /etc/raddb/mods-enabled/unix
including configuration file /etc/raddb/mods-enabled/detail.log
including configuration file /etc/raddb/mods-enabled/preprocess
including configuration file /etc/raddb/mods-enabled/radutmp
including configuration file /etc/raddb/mods-enabled/pap
including configuration file /etc/raddb/mods-enabled/dhcp
including configuration file /etc/raddb/mods-enabled/exec
including configuration file /etc/raddb/mods-enabled/passwd
including configuration file /etc/raddb/mods-enabled/realm
including configuration file /etc/raddb/mods-enabled/detail
including configuration file /etc/raddb/mods-enabled/linelog
including configuration file /etc/raddb/mods-enabled/expiration
including configuration file /etc/raddb/mods-enabled/sql
including configuration file /etc/raddb/mods-config/sql/main/postgresql/queries.conf
including configuration file /etc/raddb/mods-enabled/eap
including configuration file /etc/raddb/mods-enabled/utf8
including files in directory /etc/raddb/policy.d/
including configuration file /etc/raddb/policy.d/control
including configuration file /etc/raddb/policy.d/debug
including configuration file /etc/raddb/policy.d/abfab-tr
including configuration file /etc/raddb/policy.d/canonicalization
including configuration file /etc/raddb/policy.d/dhcp
including configuration file /etc/raddb/policy.d/operator-name
including configuration file /etc/raddb/policy.d/accounting
including configuration file /etc/raddb/policy.d/cui
including configuration file /etc/raddb/policy.d/filter
including configuration file /etc/raddb/policy.d/eap
including files in directory /etc/raddb/sites-enabled/
including configuration file /etc/raddb/sites-enabled/default
main {
 security {
 	allow_core_dumps = no
 }
}
main {
	name = "radiusd"
	prefix = "/usr/local"
	localstatedir = "/usr/local/var"
	sbindir = "/usr/local/sbin"
	logdir = "/usr/local/var/log/radius"
	run_dir = "/usr/local/var/run/radiusd"
	libdir = "/usr/local/lib"
	radacctdir = "/usr/local/var/log/radius/radacct"
	hostname_lookups = no
	max_request_time = 30
	cleanup_delay = 5
	max_requests = 1024
	pidfile = "/usr/local/var/run/radiusd/radiusd.pid"
	checkrad = "/usr/local/sbin/checkrad"
	debug_level = 0
	proxy_requests = yes
 log {
 	stripped_names = no
 	auth = no
 	auth_badpass = no
 	auth_goodpass = no
 	colourise = yes
 	msg_denied = "You are already logged in - access denied"
 }
 security {
 	max_attributes = 200
 	reject_delay = 1
 	status_server = yes
 	allow_vulnerable_openssl = "yes"
 }
}
radiusd: #### Loading Realms and Home Servers ####
 proxy server {
 	retry_delay = 5
 	retry_count = 3
 	default_fallback = no
 	dead_time = 120
 	wake_all_if_all_dead = no
 }
 home_server localhost {
 	ipaddr = 127.0.0.1
 	port = 1812
 	type = "auth"
 	secret = <<< secret >>>
 	response_window = 20.000000
 	response_timeouts = 1
 	max_outstanding = 65536
 	zombie_period = 40
 	status_check = "status-server"
 	ping_interval = 30
 	check_interval = 30
 	check_timeout = 4
 	num_answers_to_alive = 3
 	revive_interval = 120
  coa {
  	irt = 2
  	mrt = 16
  	mrc = 5
  	mrd = 30
  }
  limit {
  	max_connections = 16
  	max_requests = 0
  	lifetime = 0
  	idle_timeout = 0
  }
 }
 home_server_pool my_auth_failover {
	type = fail-over
	home_server = localhost
 }
 realm example.com {
	auth_pool = my_auth_failover
 }
 realm LOCAL {
 }
radiusd: #### Loading Clients ####
 client 0.0.0.0/0 {
 	ipaddr = *
 	require_message_authenticator = no
 	secret = <<< secret >>>
  limit {
  	max_connections = 16
  	lifetime = 0
  	idle_timeout = 30
  }
 }
radiusd: #### Instantiating modules ####
 instantiate {
 }
 modules {
  # Loaded module rlm_mschap
  # Instantiating module "mschap" from file /etc/raddb/mods-enabled/mschap
  mschap {
  	use_mppe = yes
  	require_encryption = no
  	require_strong = no
  	with_ntdomain_hack = yes
   passchange {
   }
  	allow_retry = yes
  }
  # Loaded module rlm_cache
  # Instantiating module "cache_eap" from file /etc/raddb/mods-enabled/cache_eap
  cache cache_eap {
  	key = "%{%{control:State}:-%{%{reply:State}:-%{State}}}"
  	ttl = 15
  	max_entries = 16384
  	epoch = 0
  	add_stats = no
  }
  # Loaded module rlm_expr
  # Instantiating module "expr" from file /etc/raddb/mods-enabled/expr
  expr {
  	safe_characters = "@øabcdefghijklmnopqrstuvwxyzØABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /"
  }
  # Loaded module rlm_exec
  # Instantiating module "ntlm_auth" from file /etc/raddb/mods-enabled/ntlm_auth
  exec ntlm_auth {
  	wait = yes
  	program = "/path/to/ntlm_auth --request-nt-key --domain=MYDOMAIN --username=%{mschap:User-Name} --password=%{User-Password}"
  	shell_escape = yes
  }
  # Loaded module rlm_unpack
  # Instantiating module "unpack" from file /etc/raddb/mods-enabled/unpack
  # Loaded module rlm_replicate
  # Instantiating module "replicate" from file /etc/raddb/mods-enabled/replicate
  # Loaded module rlm_soh
  # Instantiating module "soh" from file /etc/raddb/mods-enabled/soh
  soh {
  	dhcp = yes
  }
  # Loaded module rlm_digest
  # Instantiating module "digest" from file /etc/raddb/mods-enabled/digest
  # Loaded module rlm_always
  # Instantiating module "reject" from file /etc/raddb/mods-enabled/always
  always reject {
  	rcode = "reject"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "fail" from file /etc/raddb/mods-enabled/always
  always fail {
  	rcode = "fail"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "ok" from file /etc/raddb/mods-enabled/always
  always ok {
  	rcode = "ok"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "handled" from file /etc/raddb/mods-enabled/always
  always handled {
  	rcode = "handled"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "invalid" from file /etc/raddb/mods-enabled/always
  always invalid {
  	rcode = "invalid"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "userlock" from file /etc/raddb/mods-enabled/always
  always userlock {
  	rcode = "userlock"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "notfound" from file /etc/raddb/mods-enabled/always
  always notfound {
  	rcode = "notfound"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "noop" from file /etc/raddb/mods-enabled/always
  always noop {
  	rcode = "noop"
  	simulcount = 0
  	mpp = no
  }
  # Instantiating module "updated" from file /etc/raddb/mods-enabled/always
  always updated {
  	rcode = "updated"
  	simulcount = 0
  	mpp = no
  }
  # Loaded module rlm_logintime
  # Instantiating module "logintime" from file /etc/raddb/mods-enabled/logintime
  logintime {
  	minimum_timeout = 60
  }
  # Loaded module rlm_files
  # Instantiating module "files" from file /etc/raddb/mods-enabled/files
  files {
  	filename = "/etc/raddb/mods-config/files/authorize"
  	usersfile = "/etc/raddb/mods-config/files/authorize"
  	acctusersfile = "/etc/raddb/mods-config/files/accounting"
  	preproxy_usersfile = "/etc/raddb/mods-config/files/pre-proxy"
  	compat = "cistron"
  }
reading pairlist file /etc/raddb/mods-config/files/authorize
[/etc/raddb/mods-config/files/authorize]:180 Cistron compatibility checks for entry DEFAULT ...
[/etc/raddb/mods-config/files/authorize]:187 Cistron compatibility checks for entry DEFAULT ...
[/etc/raddb/mods-config/files/authorize]:194 Cistron compatibility checks for entry DEFAULT ...
reading pairlist file /etc/raddb/mods-config/files/authorize
[/etc/raddb/mods-config/files/authorize]:180 Cistron compatibility checks for entry DEFAULT ...
[/etc/raddb/mods-config/files/authorize]:187 Cistron compatibility checks for entry DEFAULT ...
[/etc/raddb/mods-config/files/authorize]:194 Cistron compatibility checks for entry DEFAULT ...
reading pairlist file /etc/raddb/mods-config/files/accounting
reading pairlist file /etc/raddb/mods-config/files/pre-proxy
  # Loaded module rlm_attr_filter
  # Instantiating module "attr_filter.post-proxy" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.post-proxy {
  	filename = "/etc/raddb/mods-config/attr_filter/post-proxy"
  	key = "%{Realm}"
  	relaxed = no
  }
reading pairlist file /etc/raddb/mods-config/attr_filter/post-proxy
  # Instantiating module "attr_filter.pre-proxy" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.pre-proxy {
  	filename = "/etc/raddb/mods-config/attr_filter/pre-proxy"
  	key = "%{Realm}"
  	relaxed = no
  }
reading pairlist file /etc/raddb/mods-config/attr_filter/pre-proxy
  # Instantiating module "attr_filter.access_reject" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.access_reject {
  	filename = "/etc/raddb/mods-config/attr_filter/access_reject"
  	key = "%{User-Name}"
  	relaxed = no
  }
reading pairlist file /etc/raddb/mods-config/attr_filter/access_reject
  # Instantiating module "attr_filter.access_challenge" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.access_challenge {
  	filename = "/etc/raddb/mods-config/attr_filter/access_challenge"
  	key = "%{User-Name}"
  	relaxed = no
  }
reading pairlist file /etc/raddb/mods-config/attr_filter/access_challenge
  # Instantiating module "attr_filter.accounting_response" from file /etc/raddb/mods-enabled/attr_filter
  attr_filter attr_filter.accounting_response {
  	filename = "/etc/raddb/mods-config/attr_filter/accounting_response"
  	key = "%{User-Name}"
  	relaxed = no
  }
reading pairlist file /etc/raddb/mods-config/attr_filter/accounting_response
  # Instantiating module "echo" from file /etc/raddb/mods-enabled/echo
  exec echo {
  	wait = yes
  	program = "/bin/echo %{User-Name}"
  	input_pairs = "request"
  	output_pairs = "reply"
  	shell_escape = yes
  }
  # Loaded module rlm_radutmp
  # Instantiating module "sradutmp" from file /etc/raddb/mods-enabled/sradutmp
  radutmp sradutmp {
  	filename = "/usr/local/var/log/radius/sradutmp"
  	username = "%{User-Name}"
  	case_sensitive = yes
  	check_with_nas = yes
  	permissions = 420
  	caller_id = no
  }
  # Loaded module rlm_dynamic_clients
  # Instantiating module "dynamic_clients" from file /etc/raddb/mods-enabled/dynamic_clients
  # Loaded module rlm_chap
  # Instantiating module "chap" from file /etc/raddb/mods-enabled/chap
  # Loaded module rlm_unix
  # Instantiating module "unix" from file /etc/raddb/mods-enabled/unix
  unix {
  	radwtmp = "/usr/local/var/log/radius/radwtmp"
  }
  # Loaded module rlm_detail
  # Instantiating module "auth_log" from file /etc/raddb/mods-enabled/detail.log
  detail auth_log {
  	filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/auth-detail-%Y%m%d"
  	header = "%t"
  	permissions = 384
  	locking = no
  	log_packet_header = no
  }
rlm_detail (auth_log): 'User-Password' suppressed, will not appear in detail output
  # Instantiating module "reply_log" from file /etc/raddb/mods-enabled/detail.log
  detail reply_log {
  	filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/reply-detail-%Y%m%d"
  	header = "%t"
  	permissions = 384
  	locking = no
  	log_packet_header = no
  }
  # Instantiating module "pre_proxy_log" from file /etc/raddb/mods-enabled/detail.log
  detail pre_proxy_log {
  	filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/pre-proxy-detail-%Y%m%d"
  	header = "%t"
  	permissions = 384
  	locking = no
  	log_packet_header = no
  }
  # Instantiating module "post_proxy_log" from file /etc/raddb/mods-enabled/detail.log
  detail post_proxy_log {
  	filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/post-proxy-detail-%Y%m%d"
  	header = "%t"
  	permissions = 384
  	locking = no
  	log_packet_header = no
  }
  # Loaded module rlm_preprocess
  # Instantiating module "preprocess" from file /etc/raddb/mods-enabled/preprocess
  preprocess {
  	huntgroups = "/etc/raddb/mods-config/preprocess/huntgroups"
  	hints = "/etc/raddb/mods-config/preprocess/hints"
  	with_ascend_hack = no
  	ascend_channels_per_line = 23
  	with_ntdomain_hack = no
  	with_specialix_jetstream_hack = no
  	with_cisco_vsa_hack = no
  	with_alvarion_vsa_hack = no
  }
reading pairlist file /etc/raddb/mods-config/preprocess/huntgroups
reading pairlist file /etc/raddb/mods-config/preprocess/hints
  # Instantiating module "radutmp" from file /etc/raddb/mods-enabled/radutmp
  radutmp {
  	filename = "/usr/local/var/log/radius/radutmp"
  	username = "%{User-Name}"
  	case_sensitive = yes
  	check_with_nas = yes
  	permissions = 384
  	caller_id = yes
  }
  # Loaded module rlm_pap
  # Instantiating module "pap" from file /etc/raddb/mods-enabled/pap
  pap {
  	normalise = yes
  }
  # Loaded module rlm_dhcp
  # Instantiating module "dhcp" from file /etc/raddb/mods-enabled/dhcp
  # Instantiating module "exec" from file /etc/raddb/mods-enabled/exec
  exec {
  	wait = no
  	input_pairs = "request"
  	shell_escape = yes
  	timeout = 10
  }
  # Loaded module rlm_passwd
  # Instantiating module "etc_passwd" from file /etc/raddb/mods-enabled/passwd
  passwd etc_passwd {
  	filename = "/etc/passwd"
  	format = "*User-Name:Crypt-Password:"
  	delimiter = ":"
  	ignore_nislike = no
  	ignore_empty = yes
  	allow_multiple_keys = no
  	hash_size = 100
  }
rlm_passwd: nfields: 3 keyfield 0(User-Name) listable: no
  # Loaded module rlm_realm
  # Instantiating module "IPASS" from file /etc/raddb/mods-enabled/realm
  realm IPASS {
  	format = "prefix"
  	delimiter = "/"
  	ignore_default = no
  	ignore_null = no
  }
  # Instantiating module "suffix" from file /etc/raddb/mods-enabled/realm
  realm suffix {
  	format = "suffix"
  	delimiter = "@"
  	ignore_default = no
  	ignore_null = no
  }
  # Instantiating module "realmpercent" from file /etc/raddb/mods-enabled/realm
  realm realmpercent {
  	format = "suffix"
  	delimiter = "%"
  	ignore_default = no
  	ignore_null = no
  }
  # Instantiating module "ntdomain" from file /etc/raddb/mods-enabled/realm
  realm ntdomain {
  	format = "prefix"
  	delimiter = "\\"
  	ignore_default = no
  	ignore_null = no
  }
  # Instantiating module "detail" from file /etc/raddb/mods-enabled/detail
  detail {
  	filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d"
  	header = "%t"
  	permissions = 384
  	locking = no
  	log_packet_header = no
  }
  # Loaded module rlm_linelog
  # Instantiating module "linelog" from file /etc/raddb/mods-enabled/linelog
  linelog {
  	filename = "/usr/local/var/log/radius/linelog"
  	permissions = 384
  	format = "This is a log message for %{User-Name}"
  	reference = "messages.%{%{Packet-Type}:-default}"
  }
  # Instantiating module "log_accounting" from file /etc/raddb/mods-enabled/linelog
  linelog log_accounting {
  	filename = "/usr/local/var/log/radius/linelog-accounting"
  	permissions = 384
  	format = ""
  	reference = "Accounting-Request.%{%{Acct-Status-Type}:-unknown}"
  }
  # Loaded module rlm_expiration
  # Instantiating module "expiration" from file /etc/raddb/mods-enabled/expiration
  # Loaded module rlm_sql
  # Instantiating module "sql" from file /etc/raddb/mods-enabled/sql
  sql {
  	driver = "rlm_sql_postgresql"
  	server = "10.0.10.4"
  	port = "5432"
  	login = "jt_radius"
  	password = <<< secret >>>
  	radius_db = "jt"
  	read_groups = yes
  	read_profiles = yes
  	read_clients = no
  	delete_stale_sessions = yes
  	sql_user_name = "%{%{Stripped-User-Name}:-%{%{User-Name}:-none}}"
  	default_user_profile = ""
  	client_query = "SELECT id, nasname, shortname, type, secret, server FROM nas"
  	authorize_check_query = "SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id"
  	authorize_reply_query = "SELECT id, UserName, Attribute, Value, Op FROM radreply WHERE Username = '%{SQL-User-Name}' ORDER BY id"
  	authorize_group_check_query = "SELECT id, GroupName, Attribute, Value, op FROM radgroupcheck WHERE GroupName = '%{Sql-Group}' ORDER BY id"
  	authorize_group_reply_query = "SELECT id, GroupName, Attribute, Value, op FROM radgroupreply WHERE GroupName = '%{Sql-Group}' ORDER BY id"
  	group_membership_query = "SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority"
  	simul_count_query = ""
  	simul_verify_query = ""
  	safe_characters = "@àáâãäåæçèéêëìíîïðñòóôõöøùúûüýÿabcdefghijklmnopqrstuvwxyzÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝŸABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /"
   accounting {
   	reference = "%{tolower:type.%{%{Acct-Status-Type}:-none}.query}"
    type {
     accounting-on {
     	query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime))), AcctTerminateCause = '%{%{Acct-Terminate-Cause}:-NAS-Reboot}', WHERE AcctStopTime IS NULL AND NASIPAddress= '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}' AND AcctStartTime <= '%S'::timestamp"
     }
     accounting-off {
     	query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime))), AcctTerminateCause = '%{%{Acct-Terminate-Cause}:-NAS-Reboot}', WHERE AcctStopTime IS NULL AND NASIPAddress= '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}' AND AcctStartTime <= '%S'::timestamp"
     }
     start {
     	query = "INSERT INTO radacct (AcctSessionId,		AcctUniqueId,		UserName, Realm,			NASIPAddress,		NASPortId, NASPortType,		AcctStartTime,		AcctUpdateTime, AcctStopTime,		AcctSessionTime, 	AcctAuthentic, ConnectInfo_start,	ConnectInfo_Stop, 	AcctInputOctets, AcctOutputOctets,	CalledStationId, 	CallingStationId, AcctTerminateCause,	ServiceType,		FramedProtocol, FramedIpAddress) VALUES('%{Acct-Session-Id}', '%{Acct-Unique-Session-Id}', '%{SQL-User-Name}', NULLIF('%{Realm}', ''), '%{%{NAS-IPv6-Address}:-%{NAS-IP-Address}}', %{%{NAS-Port}:-NULL}, '%{NAS-Port-Type}', TO_TIMESTAMP(%{integer:Event-Timestamp}), TO_TIMESTAMP(%{integer:Event-Timestamp}), NULL, 0, '%{Acct-Authentic}', '%{Connect-Info}', NULL, 0, 0, '%{Called-Station-Id}', '%{Calling-Station-Id}', NULL, '%{Service-Type}', '%{Framed-Protocol}', NULLIF('%{Framed-IP-Address}', '')::inet)"
     }
     interim-update {
     	query = "UPDATE radacct SET FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, AcctSessionTime = %{%{Acct-Session-Time}:-NULL}, AcctInterval = (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM (COALESCE(AcctUpdateTime, AcctStartTime)))), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctInputOctets = (('%{%{Acct-Input-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Input-Octets}:-0}'::bigint), AcctOutputOctets = (('%{%{Acct-Output-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Output-Octets}:-0}'::bigint) WHERE (AcctUniqueId = '%{Acct-Unique-Session-Id}') AND AcctStopTime IS NULL"
     }
     stop {
     	query = "UPDATE radacct SET AcctStopTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctUpdateTime = TO_TIMESTAMP(%{integer:Event-Timestamp}), AcctSessionTime = COALESCE(%{%{Acct-Session-Time}:-NULL}, (%{integer:Event-Timestamp} - EXTRACT(EPOCH FROM(AcctStartTime)))), AcctInputOctets = (('%{%{Acct-Input-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Input-Octets}:-0}'::bigint), AcctOutputOctets = (('%{%{Acct-Output-Gigawords}:-0}'::bigint << 32) + '%{%{Acct-Output-Octets}:-0}'::bigint), AcctTerminateCause = '%{Acct-Terminate-Cause}', FramedIPAddress = NULLIF('%{Framed-IP-Address}', '')::inet, ConnectInfo_stop = '%{Connect-Info}' WHERE (AcctUniqueId = '%{Acct-Unique-Session-Id}') AND AcctStopTime IS NULL"
     }
    }
   }
   post-auth {
   	reference = ".query"
   	query = "INSERT INTO radpostauth (username, pass, reply, authdate) VALUES('%{User-Name}', '%{%{User-Password}:-Chap-Password}', '%{reply:Packet-Type}', NOW())"
   }
  }
   postgresql {
   	send_application_name = no
   }
rlm_sql (sql): Driver rlm_sql_postgresql (module rlm_sql_postgresql) loaded and linked
rlm_sql (sql): Attempting to connect to database "jt"
rlm_sql (sql): Initialising connection pool
   pool {
   	start = 5
   	min = 4
   	max = 32
   	spare = 3
   	uses = 0
   	lifetime = 0
   	cleanup_interval = 30
   	idle_timeout = 60
   	retry_delay = 1
   	spread = no
   }
rlm_sql (sql): Opening additional connection (0)
rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4'
Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10252
rlm_sql (sql): Opening additional connection (1)
rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4'
Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10253
rlm_sql (sql): Opening additional connection (2)
rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4'
Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10254
rlm_sql (sql): Opening additional connection (3)
rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4'
Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10255
rlm_sql (sql): Opening additional connection (4)
rlm_sql_postgresql: Connecting using parameters: dbname='jt' host='10.0.10.4' port=5432 user='jt_radius' password='dkjE41ASG25Acfsaa4'
Connected to database 'jt' on '10.0.10.4' server version 90104, protocol version 3, backend PID 10256
  # Loaded module rlm_eap
  # Instantiating module "eap" from file /etc/raddb/mods-enabled/eap
  eap {
  	default_eap_type = "md5"
  	timer_expire = 60
  	ignore_unknown_eap_types = no
  	mod_accounting_username_bug = no
  	max_sessions = 1024
  }
   # Linked to sub-module rlm_eap_md5
   # Linked to sub-module rlm_eap_leap
   # Linked to sub-module rlm_eap_gtc
   gtc {
   	challenge = "Password: "
   	auth_type = "PAP"
   }
   # Linked to sub-module rlm_eap_tls
   tls {
   	tls = "tls-common"
   }
   tls-config tls-common {
   	rsa_key_exchange = no
   	dh_key_exchange = yes
   	rsa_key_length = 512
   	dh_key_length = 512
   	verify_depth = 0
   	ca_path = "/etc/raddb/certs"
   	pem_file_type = yes
   	private_key_file = "/etc/raddb/certs/server.pem"
   	certificate_file = "/etc/raddb/certs/server.pem"
   	ca_file = "/etc/raddb/certs/ca.pem"
   	private_key_password = <<< secret >>>
   	dh_file = "/etc/raddb/certs/dh"
   	fragment_size = 1024
   	include_length = yes
   	check_crl = no
   	cipher_list = "DEFAULT"
   	ecdh_curve = "prime256v1"
    cache {
    	enable = yes
    	lifetime = 24
    	max_entries = 255
    }
    verify {
    }
    ocsp {
    	enable = no
    	override_cert_url = yes
    	url = "http://127.0.0.1/ocsp/"
    	use_nonce = yes
    	timeout = 0
    	softfail = yes
    }
   }
   # Linked to sub-module rlm_eap_ttls
   ttls {
   	tls = "tls-common"
   	default_eap_type = "md5"
   	copy_request_to_tunnel = no
   	use_tunneled_reply = no
   	virtual_server = "inner-tunnel"
   	include_length = yes
   	require_client_cert = no
   }
Using cached TLS configuration from previous invocation
   # Linked to sub-module rlm_eap_peap
   peap {
   	tls = "tls-common"
   	default_method = "mschapv2"
   	copy_request_to_tunnel = no
   	use_tunneled_reply = no
   	proxy_tunneled_request_as_eap = yes
   	virtual_server = "inner-tunnel"
   	soh = no
   	require_client_cert = no
   }
Using cached TLS configuration from previous invocation
   # Linked to sub-module rlm_eap_mschapv2
   mschapv2 {
   	with_ntdomain_hack = no
   	send_error = no
   }
  # Loaded module rlm_utf8
  # Instantiating module "utf8" from file /etc/raddb/mods-enabled/utf8
 } # modules
radiusd: #### Loading Virtual Servers ####
server { # from file /etc/raddb/radiusd.conf
} # server
server default { # from file /etc/raddb/sites-enabled/default
 # Loading authenticate {...}
 # Loading authorize {...}
 # Loading accounting {...}
 # Loading post-proxy {...}
 # Loading post-auth {...}
} # server default
radiusd: #### Opening IP addresses and Ports ####
listen {
  	type = "auth"
  	ipaddr = *
  	port = 0
   limit {
   	max_connections = 16
   	lifetime = 0
   	idle_timeout = 30
   }
}
listen {
  	type = "acct"
  	ipaddr = *
  	port = 0
   limit {
   	max_connections = 16
   	lifetime = 0
   	idle_timeout = 30
   }
}
listen {
  	type = "auth"
  	ipv6addr = ::
  	port = 0
   limit {
   	max_connections = 16
   	lifetime = 0
   	idle_timeout = 30
   }
}
listen {
  	type = "acct"
  	ipv6addr = ::
  	port = 0
   limit {
   	max_connections = 16
   	lifetime = 0
   	idle_timeout = 30
   }
}
Listening on auth address * port 1812 as server default
Listening on acct address * port 1813 as server default
Listening on auth address :: port 1812 as server default
Listening on acct address :: port 1813 as server default
Opening new proxy socket 'proxy address * port 0'
Listening on proxy address * port 60147
Ready to process requests
Received Access-Request Id 179 from 193.35.56.122:36176 to 10.0.10.5:1812 length 140
	NAS-Port-Type = Wireless-802.11
	Calling-Station-Id = '55-44-33-22-11-00'
	Called-Station-Id = '00-11-22-33-44-55'
	User-Name = 'êllô@foo.com'
	User-Password = 'bakingsoda'
	NAS-Port = 2153775108
	Acct-Session-Id = '12345qwert'
	Framed-IP-Address = 192.168.1.1
	Service-Type = Login-User
	NAS-Identifier = 'test'
	NAS-IP-Address = 1.2.3.4
(0) Received Access-Request packet from host 193.35.56.122 port 36176, id=179, length=140
(0) 	NAS-Port-Type = Wireless-802.11
(0) 	Calling-Station-Id = '55-44-33-22-11-00'
(0) 	Called-Station-Id = '00-11-22-33-44-55'
(0) 	User-Name = 'êllô@foo.com'
(0) 	User-Password = 'bakingsoda'
(0) 	NAS-Port = 2153775108
(0) 	Acct-Session-Id = '12345qwert'
(0) 	Framed-IP-Address = 192.168.1.1
(0) 	Service-Type = Login-User
(0) 	NAS-Identifier = 'test'
(0) 	NAS-IP-Address = 1.2.3.4
(0) # Executing section authorize from file /etc/raddb/sites-enabled/default
(0)   authorize {
(0) sql: EXPAND %{%{Stripped-User-Name}:-%{%{User-Name}:-none}}
(0) sql:    --> êllô@foo.com
(0) sql: SQL-User-Name set to 'êllô@foo.com'
rlm_sql (sql): Reserved connection (4)
(0) sql: EXPAND SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '%{SQL-User-Name}' ORDER BY id
(0) sql:    --> SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id
rlm_sql (sql): Executing query: 'SELECT id, UserName, Attribute, Value, Op FROM radcheck WHERE Username = '=C3=AAll=C3=B4@foo.com' ORDER BY id'
rlm_sql_postgresql: Status: PGRES_TUPLES_OK
rlm_sql_postgresql: query affected rows = 0 , fields = 5
(0) sql: EXPAND SELECT GroupName FROM radusergroup WHERE UserName='%{SQL-User-Name}' ORDER BY priority
(0) sql:    --> SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority
rlm_sql (sql): Executing query: 'SELECT GroupName FROM radusergroup WHERE UserName='=C3=AAll=C3=B4@foo.com' ORDER BY priority'
rlm_sql_postgresql: Status: PGRES_TUPLES_OK
rlm_sql_postgresql: query affected rows = 0 , fields = 1
(0) sql: User not found in any groups
rlm_sql (sql): Released connection (4)
(0)     [sql] = notfound
(0) pap: WARNING: No "known good" password found for the user.  Not setting Auth-Type
(0) pap: WARNING: Authentication will fail unless a "known good" password is available
(0)     [pap] = noop
(0)   } # authorize = noop
(0) ERROR: No Auth-Type found: rejecting the user via Post-Auth-Type = Reject
(0) Failed to authenticate the user

3.0.x rlm_sql mime encoding UTF8 characters

Adam Hammond