Any tips on narrowing down this error in the debugging logs? I have run a number of -X sessions, the biggest being 20 minutes generating 517Mb of logs. The error does not appear in debugging mode, grepping for spawn. Even just looking for 'Error' and 'ldap' in the same line returns zero hits. On restarting the server in normal mode, I instantly see the error appearing with regular frequency - around 5 times per minute. The closest I can find is complaining about needing more spares in this section here: rlm_ldap (ldap): Deleting connection (6) rlm_ldap (ldap): 0 of 6 connections in use. Need more spares rlm_ldap (ldap): Opening additional connection (18) rlm_ldap (ldap): Connecting to aaaaa.ad.kent.ac.uk bbbbb.ad.kent.ac.uk cccc.ad.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful (96) [ldap] = ok (96) } # authorize = updated (96) Found Auth-Type = EAP (96) # Executing group from file /etc/raddb/sites-enabled/inner-tunnel (96) authenticate { (96) eap : Expiring EAP session with state 0xbb13282fbb1131a0 (96) eap : Finished EAP session with state 0x05a9503c04a34a83 (96) eap : Previous EAP request found for state 0x05a9503c04a34a83, released from the list (96) eap : Peer sent method MSCHAPv2 (26) The config information is as follows: radiusd: FreeRADIUS Version 3.0.4, for host x86_64-redhat-linux-gnu, built on Dec 28 2014 at 07:00:16 Copyright (C) 1999-2014 The FreeRADIUS server project and contributors There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE You may redistribute copies of FreeRADIUS under the terms of the GNU General Public License For more information about these matters, see the file named COPYRIGHT Starting - reading configuration files ... including dictionary file /usr/share/freeradius/dictionary including dictionary file /usr/share/freeradius/dictionary.dhcp including dictionary file /usr/share/freeradius/dictionary.vqp including dictionary file /etc/raddb/dictionary including configuration file /etc/raddb/radiusd.conf including configuration file /etc/raddb/roaming_proxy.conf including configuration file /etc/raddb/proxy.conf including configuration file /etc/raddb/roaming_clients.conf including configuration file /etc/raddb/clients.conf including files in directory /etc/raddb/mods-enabled/ including configuration file /etc/raddb/mods-enabled/attr_filter including configuration file /etc/raddb/mods-enabled/digest including configuration file /etc/raddb/mods-enabled/ldap including configuration file /etc/raddb/mods-enabled/unpack including configuration file /etc/raddb/mods-enabled/expiration including configuration file /etc/raddb/mods-enabled/unix including configuration file /etc/raddb/mods-enabled/linelog including configuration file /etc/raddb/mods-enabled/ntlm_auth including configuration file /etc/raddb/mods-enabled/passwd including configuration file /etc/raddb/mods-enabled/detail including configuration file /etc/raddb/mods-enabled/files including configuration file /etc/raddb/mods-enabled/dynamic_clients including configuration file /etc/raddb/mods-enabled/eap including configuration file /etc/raddb/mods-enabled/mschap including configuration file /etc/raddb/mods-enabled/chap including configuration file /etc/raddb/mods-enabled/radutmp including configuration file /etc/raddb/mods-enabled/expr including configuration file /etc/raddb/mods-enabled/pap including configuration file /etc/raddb/mods-enabled/detail.log including configuration file /etc/raddb/mods-enabled/exec including configuration file /etc/raddb/mods-enabled/preprocess including configuration file /etc/raddb/mods-enabled/utf8 including configuration file /etc/raddb/mods-enabled/logintime including configuration file /etc/raddb/mods-enabled/cache_eap including configuration file /etc/raddb/mods-enabled/soh including configuration file /etc/raddb/mods-enabled/eduroam-eap including configuration file /etc/raddb/mods-enabled/sradutmp including configuration file /etc/raddb/mods-enabled/always including configuration file /etc/raddb/mods-enabled/realm including configuration file /etc/raddb/mods-enabled/replicate including configuration file /etc/raddb/mods-enabled/echo including configuration file /etc/raddb/mods-enabled/dhcp including configuration file /etc/raddb/mods-enabled/sql including files in directory /etc/raddb/policy.d/ including configuration file /etc/raddb/policy.d/control including configuration file /etc/raddb/policy.d/cui including configuration file /etc/raddb/policy.d/canonicalization including configuration file /etc/raddb/policy.d/eap including configuration file /etc/raddb/policy.d/filter including configuration file /etc/raddb/policy.d/debug including configuration file /etc/raddb/policy.d/kent-local including configuration file /etc/raddb/policy.d/operator-name including configuration file /etc/raddb/policy.d/accounting including configuration file /etc/raddb/policy.d/dhcp including files in directory /etc/raddb/sites-enabled/ including configuration file /etc/raddb/sites-enabled/default including configuration file /etc/raddb/sites-enabled/control-socket including configuration file /etc/raddb/sites-enabled/eduroam-inner-tunnel including configuration file /etc/raddb/sites-enabled/kent-eduroam including configuration file /etc/raddb/sites-enabled/eduroam including configuration file /etc/raddb/sites-enabled/inner-tunnel including configuration file /etc/raddb/sites-enabled/check-eap-tls main { security { user = "radiusd" group = "radiusd" allow_core_dumps = no } } main { name = "radiusd" prefix = "/usr" localstatedir = "/var" sbindir = "/usr/sbin" logdir = "/var/log/radius" run_dir = "/var/run/radiusd" libdir = "/usr/lib64/freeradius" radacctdir = "/var/log/radius/radacct" hostname_lookups = no max_request_time = 30 cleanup_delay = 5 max_requests = 4096 pidfile = "/var/run/radiusd/radiusd.pid" checkrad = "/usr/sbin/checkrad" debug_level = 0 proxy_requests = yes log { stripped_names = no auth = yes auth_badpass = no auth_goodpass = no colourise = yes msg_denied = "You are already logged in - access denied" } security { max_attributes = 200 reject_delay = 1 status_server = yes } } radiusd: #### Loading Realms and Home Servers #### proxy server { retry_delay = 5 retry_count = 3 default_fallback = no dead_time = 120 wake_all_if_all_dead = no } home_server localhost { ipaddr = 127.0.0.1 port = 1812 type = "auth" secret = <<< secret >>> response_window = 20.000000 response_timeouts = 1 max_outstanding = 65536 zombie_period = 40 status_check = "status-server" ping_interval = 30 check_interval = 30 check_timeout = 4 num_answers_to_alive = 3 revive_interval = 120 coa { irt = 2 mrt = 16 mrc = 5 mrd = 30 } limit { max_connections = 16 max_requests = 0 lifetime = 0 idle_timeout = 0 } } home_server eduroam0 { ipaddr = roaming0.ja.net IPv4 address [194.82.174.185] port = 1812 type = "auth" secret = <<< secret >>> response_window = 30.000000 response_timeouts = 1 max_outstanding = 65536 zombie_period = 40 status_check = "status-server" ping_interval = 30 check_interval = 30 check_timeout = 4 num_answers_to_alive = 3 revive_interval = 300 coa { irt = 2 mrt = 16 mrc = 5 mrd = 30 } limit { max_connections = 16 max_requests = 0 lifetime = 0 idle_timeout = 0 } } home_server eduroam1 { ipaddr = roaming1.ja.net IPv4 address [194.83.56.233] port = 1812 type = "auth" secret = <<< secret >>> response_window = 30.000000 response_timeouts = 1 max_outstanding = 65536 zombie_period = 40 status_check = "status-server" ping_interval = 30 check_interval = 30 check_timeout = 4 num_answers_to_alive = 3 revive_interval = 300 coa { irt = 2 mrt = 16 mrc = 5 mrd = 30 } limit { max_connections = 16 max_requests = 0 lifetime = 0 idle_timeout = 0 } } home_server eduroam2 { ipaddr = roaming2.ja.net IPv4 address [194.83.56.249] port = 1812 type = "auth" secret = <<< secret >>> response_window = 30.000000 response_timeouts = 1 max_outstanding = 65536 zombie_period = 40 status_check = "status-server" ping_interval = 30 check_interval = 30 check_timeout = 4 num_answers_to_alive = 3 revive_interval = 300 coa { irt = 2 mrt = 16 mrc = 5 mrd = 30 } limit { max_connections = 16 max_requests = 0 lifetime = 0 idle_timeout = 0 } } home_server_pool my_auth_failover { type = fail-over home_server = localhost } realm example.com { auth_pool = my_auth_failover } realm LOCAL { } home_server_pool eduroam { type = client-port-balance home_server = eduroam0 home_server = eduroam1 home_server = eduroam2 } realm eduroam { auth_pool = eduroam nostrip } realm kent.ac.uk { } realm test.kent.ac.uk { } radiusd: #### Loading Clients #### client roaming0.ja.net { ipaddr = roaming0.ja.net IPv4 address [194.82.174.185] require_message_authenticator = no secret = <<< secret >>> shortname = "roaming0.ja.net" nas_type = "other" virtual_server = "eduroam" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client roaming1.ja.net { ipaddr = roaming1.ja.net IPv4 address [194.83.56.233] require_message_authenticator = no secret = <<< secret >>> shortname = "roaming1.ja.net" nas_type = "other" virtual_server = "eduroam" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client roaming2.ja.net { ipaddr = roaming2.ja.net IPv4 address [194.83.56.249] require_message_authenticator = no secret = <<< secret >>> shortname = "roaming2.ja.net" nas_type = "other" virtual_server = "eduroam" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client localhost { ipaddr = 127.0.0.1 require_message_authenticator = no secret = <<< secret >>> nas_type = "other" virtual_server = "kent-eduroam" proto = "*" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client localhost_ipv6 { ipv6addr = ::1 require_message_authenticator = no secret = <<< secret >>> limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client cwlc-cer { ipaddr = 129.12.6.216 require_message_authenticator = no secret = <<< secret >>> virtual_server = "kent-eduroam" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client cwlc-tlb { ipaddr = 129.12.6.197 require_message_authenticator = no secret = <<< secret >>> virtual_server = "kent-eduroam" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client sentry { ipaddr = 129.12.21.130 require_message_authenticator = no secret = <<< secret >>> virtual_server = "kent-eduroam" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client swdjh { ipaddr = 129.12.87.52 require_message_authenticator = no secret = <<< secret >>> virtual_server = "kent-dot1x" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } radiusd: #### Instantiating modules #### instantiate { } modules { # Loaded module rlm_attr_filter # Instantiating module "attr_filter.post-proxy" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.post-proxy { filename = "/etc/raddb/mods-config/attr_filter/post-proxy" key = "%{Realm}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/post-proxy # Instantiating module "attr_filter.pre-proxy" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.pre-proxy { filename = "/etc/raddb/mods-config/attr_filter/pre-proxy" key = "%{Realm}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/pre-proxy # Instantiating module "attr_filter.access_reject" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.access_reject { filename = "/etc/raddb/mods-config/attr_filter/access_reject" key = "%{User-Name}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/access_reject # Instantiating module "attr_filter.access_challenge" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.access_challenge { filename = "/etc/raddb/mods-config/attr_filter/access_challenge" key = "%{User-Name}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/access_challenge # Instantiating module "attr_filter.accounting_response" from file /etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.accounting_response { filename = "/etc/raddb/mods-config/attr_filter/accounting_response" key = "%{User-Name}" relaxed = no } reading pairlist file /etc/raddb/mods-config/attr_filter/accounting_response # Loaded module rlm_digest # Instantiating module "digest" from file /etc/raddb/mods-enabled/digest # Loaded module rlm_ldap # Instantiating module "ldap" from file /etc/raddb/mods-enabled/ldap ldap { server = "bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk" port = 636 password = <<< secret >>> identity = "cn=freerad,cn=Users,dc=ad,dc=kent,dc=ac,dc=uk" user { filter = "(&(&(sAMAccountName=%{%{Stripped-User-Name}:-%{User-Name}})(memberof:1.2.840.113556.1.4.1941:=CN=eduroam permitted onsite,OU=Wireless Access,OU=Computing Service Maintained Groups,DC=ad,DC=kent,DC=ac,DC=uk))(!(memberof:1.2.840.113556.1.4.1941:=CN=Wifi Blocked Users,OU=Wireless Access,OU=Computing Service Maintained Groups,DC=ad,DC=kent,DC=ac,DC=uk)))" scope = "sub" base_dn = "DC=ad,DC=kent,DC=ac,DC=uk" access_positive = yes } group { filter = "(objectClass=group)" scope = "sub" base_dn = "DC=ad,DC=kent,DC=ac,DC=uk" name_attribute = "cn" membership_attribute = "memberOf" cacheable_name = no cacheable_dn = no } client { filter = "(objectClass=frClient)" scope = "sub" base_dn = "DC=ad,DC=kent,DC=ac,DC=uk" attribute { identifier = "radiusClientIdentifier" shortname = "cn" secret = "radiusClientSecret" } } profile { filter = "(&)" } options { ldap_debug = 40 chase_referrals = yes rebind = yes net_timeout = 1 res_timeout = 20 srv_timelimit = 20 idle = 60 probes = 3 interval = 3 } tls { ca_file = "/etc/raddb/certs/UniversityOfKentRootCA.crt" start_tls = no require_cert = "allow" } } rlm_ldap: Falling back to build time libldap version info. Query for LDAP_OPT_API_INFO returned: -1 rlm_ldap: libldap vendor: OpenLDAP version: 20439 rlm_ldap (ldap): Couldn't find configuration for accounting, will return NOOP for calls from this section post-auth { reference = "." } rlm_ldap (ldap): Initialising connection pool pool { start = 16 min = 8 max = 64 spare = 64 uses = 0 lifetime = 0 cleanup_interval = 30 idle_timeout = 0 retry_delay = 1 spread = no } rlm_ldap (ldap): Opening additional connection (0) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (1) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (2) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (3) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (4) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (5) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (6) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (7) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (8) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (9) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (10) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (11) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (12) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (13) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (14) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful rlm_ldap (ldap): Opening additional connection (15) rlm_ldap (ldap): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (ldap): Waiting for bind result... rlm_ldap (ldap): Bind successful # Instantiating module "eduroam" from file /etc/raddb/mods-enabled/ldap ldap eduroam { server = "bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk" port = 636 password = <<< secret >>> identity = "cn=freerad,cn=Users,dc=ad,dc=kent,dc=ac,dc=uk" user { filter = "(&(&(sAMAccountName=%{%{Stripped-User-Name}:-%{User-Name}})(memberof:1.2.840.113556.1.4.1941:=CN=eduroam permitted roaming,OU=Wireless Access,OU=Computing Service Maintained Groups,DC=ad,DC=kent,DC=ac,DC=uk))(!(memberof:1.2.840.113556.1.4.1941:=CN=Wifi Blocked Users,OU=Wireless Access,OU=Computing Service Maintained Groups,DC=ad,DC=kent,DC=ac,DC=uk)))" scope = "sub" base_dn = "DC=ad,DC=kent,DC=ac,DC=uk" access_positive = yes } group { filter = "(objectClass=group)" scope = "sub" base_dn = "DC=ad,DC=kent,DC=ac,DC=uk" name_attribute = "cn" membership_attribute = "memberOf" cacheable_name = no cacheable_dn = no } client { filter = "(objectClass=frClient)" scope = "sub" base_dn = "DC=ad,DC=kent,DC=ac,DC=uk" attribute { identifier = "radiusClientIdentifier" shortname = "cn" secret = "radiusClientSecret" } } profile { filter = "(&)" } options { ldap_debug = 40 chase_referrals = yes rebind = yes net_timeout = 1 res_timeout = 20 srv_timelimit = 20 idle = 60 probes = 3 interval = 3 } tls { ca_file = "/etc/raddb/certs/UniversityOfKentRootCA.crt" start_tls = no require_cert = "allow" } } rlm_ldap (eduroam): Couldn't find configuration for accounting, will return NOOP for calls from this section post-auth { reference = "." } rlm_ldap (eduroam): Initialising connection pool pool { start = 16 min = 8 max = 64 spare = 8 uses = 0 lifetime = 0 cleanup_interval = 30 idle_timeout = 0 retry_delay = 1 spread = no } rlm_ldap (eduroam): Opening additional connection (0) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (1) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (2) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (3) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (4) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (5) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (6) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (7) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (8) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (9) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (10) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (11) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (12) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (13) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (14) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful rlm_ldap (eduroam): Opening additional connection (15) rlm_ldap (eduroam): Connecting to bbbbb.kent.ac.uk aaaaa.kent.ac.uk ccccc.kent.ac.uk:636 rlm_ldap (eduroam): Waiting for bind result... rlm_ldap (eduroam): Bind successful # Loaded module rlm_unpack # Instantiating module "unpack" from file /etc/raddb/mods-enabled/unpack # Loaded module rlm_expiration # Instantiating module "expiration" from file /etc/raddb/mods-enabled/expiration # Loaded module rlm_unix # Instantiating module "unix" from file /etc/raddb/mods-enabled/unix unix { radwtmp = "/var/log/radius/radwtmp" } # Loaded module rlm_linelog # Instantiating module "linelog" from file /etc/raddb/mods-enabled/linelog linelog { filename = "/var/log/radius/linelog" permissions = 384 format = "This is a log message for %{User-Name}" reference = "messages.%{%{Packet-Type}:-default}" } # Instantiating module "log_accounting" from file /etc/raddb/mods-enabled/linelog linelog log_accounting { filename = "/var/log/radius/linelog-accounting" permissions = 384 format = "" reference = "Accounting-Request.%{%{Acct-Status-Type}:-unknown}" } # Loaded module rlm_exec # Instantiating module "ntlm_auth" from file /etc/raddb/mods-enabled/ntlm_auth exec ntlm_auth { wait = yes program = "/path/to/ntlm_auth --request-nt-key --domain=MYDOMAIN --username=%{mschap:User-Name} --password=%{User-Password}" shell_escape = yes } # Loaded module rlm_passwd # Instantiating module "etc_passwd" from file /etc/raddb/mods-enabled/passwd passwd etc_passwd { filename = "/etc/passwd" format = "*User-Name:Crypt-Password:" delimiter = ":" ignore_nislike = no ignore_empty = yes allow_multiple_keys = no hash_size = 100 } rlm_passwd: nfields: 3 keyfield 0(User-Name) listable: no # Loaded module rlm_detail # Instantiating module "detail" from file /etc/raddb/mods-enabled/detail detail { filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Loaded module rlm_files # Instantiating module "files" from file /etc/raddb/mods-enabled/files files { filename = "/etc/raddb/mods-config/files/authorize" usersfile = "/etc/raddb/mods-config/files/authorize" acctusersfile = "/etc/raddb/mods-config/files/accounting" preproxy_usersfile = "/etc/raddb/mods-config/files/pre-proxy" compat = "cistron" } reading pairlist file /etc/raddb/mods-config/files/authorize [/etc/raddb/mods-config/files/authorize]:181 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:188 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:195 Cistron compatibility checks for entry DEFAULT ... reading pairlist file /etc/raddb/mods-config/files/authorize [/etc/raddb/mods-config/files/authorize]:181 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:188 Cistron compatibility checks for entry DEFAULT ... [/etc/raddb/mods-config/files/authorize]:195 Cistron compatibility checks for entry DEFAULT ... reading pairlist file /etc/raddb/mods-config/files/accounting reading pairlist file /etc/raddb/mods-config/files/pre-proxy [/etc/raddb/mods-config/files/pre-proxy]:33 Cistron compatibility checks for entry DEFAULT ... # Loaded module rlm_dynamic_clients # Instantiating module "dynamic_clients" from file /etc/raddb/mods-enabled/dynamic_clients # Loaded module rlm_eap # Instantiating module "eap" from file /etc/raddb/mods-enabled/eap eap { default_eap_type = "peap" timer_expire = 60 ignore_unknown_eap_types = no mod_accounting_username_bug = no max_sessions = 4096 } # Linked to sub-module rlm_eap_gtc gtc { challenge = "Password: " auth_type = "PAP" } # Linked to sub-module rlm_eap_tls tls { tls = "loanlaptops" virtual_server = "check-eap-tls" } tls-config loanlaptops { rsa_key_exchange = no dh_key_exchange = yes rsa_key_length = 512 dh_key_length = 512 verify_depth = 0 ca_path = "/etc/raddb/certs" pem_file_type = yes private_key_file = "/etc/raddb/certs/loan_laptop_radius2ad.pem" certificate_file = "/etc/raddb/certs/loan_laptop_radius2ad.pem" ca_file = "/etc/raddb/certs/unikentrootCAchain.pem" dh_file = "/etc/raddb/certs/dh" random_file = "/etc/raddb/certs/random" fragment_size = 1024 include_length = yes check_crl = no cipher_list = "DEFAULT" ecdh_curve = "prime256v1" cache { enable = yes lifetime = 24 max_entries = 255 } verify { } ocsp { enable = no override_cert_url = yes url = "http://ocsp.kent.ac.uk/ocsp" use_nonce = yes timeout = 0 softfail = yes } } # Linked to sub-module rlm_eap_ttls ttls { tls = "tls-common" default_eap_type = "mschapv2" copy_request_to_tunnel = no use_tunneled_reply = yes virtual_server = "inner-tunnel" include_length = yes require_client_cert = no } tls-config tls-common { rsa_key_exchange = no dh_key_exchange = yes rsa_key_length = 512 dh_key_length = 512 verify_depth = 0 ca_path = "/etc/raddb/certs" pem_file_type = yes private_key_file = "/etc/pki/tls/private/radius2.ad.kent.ac.uk.key" certificate_file = "/etc/pki/tls/certs/radius2.ad.kent.ac.uk.pem" ca_file = "/etc/pki/tls/certs/globalsign.pem" dh_file = "/etc/raddb/certs/dh" fragment_size = 1024 include_length = yes check_crl = no cipher_list = "DEFAULT" ecdh_curve = "prime256v1" cache { enable = yes lifetime = 24 max_entries = 255 } verify { } ocsp { enable = no override_cert_url = yes url = "http://127.0.0.1/ocsp/" use_nonce = yes timeout = 0 softfail = yes } } # Linked to sub-module rlm_eap_peap peap { tls = "tls-common" default_method = "mschapv2" copy_request_to_tunnel = yes use_tunneled_reply = yes proxy_tunneled_request_as_eap = yes virtual_server = "inner-tunnel" soh = no require_client_cert = no } Using cached TLS configuration from previous invocation # Linked to sub-module rlm_eap_mschapv2 mschapv2 { with_ntdomain_hack = no send_error = yes } # Loaded module rlm_mschap # Instantiating module "mschap" from file /etc/raddb/mods-enabled/mschap mschap { use_mppe = yes require_encryption = yes require_strong = yes with_ntdomain_hack = yes ntlm_auth = "/bin/ntlm_auth --request-nt-key --username=%{%{Stripped-User-Name}:-%{%{User-Name}:-None}} --challenge=%{%{mschap:Challenge}:-00} --nt-response=%{%{mschap:NT-Response}:-00}" passchange { } allow_retry = yes retry_msg = "Re-enter (or reset) the password" } # Loaded module rlm_chap # Instantiating module "chap" from file /etc/raddb/mods-enabled/chap # Loaded module rlm_radutmp # Instantiating module "radutmp" from file /etc/raddb/mods-enabled/radutmp radutmp { filename = "/var/log/radius/radutmp" username = "%{User-Name}" case_sensitive = yes check_with_nas = yes permissions = 384 caller_id = yes } # Loaded module rlm_expr # Instantiating module "expr" from file /etc/raddb/mods-enabled/expr expr { safe_characters = "@abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /" } # Loaded module rlm_pap # Instantiating module "pap" from file /etc/raddb/mods-enabled/pap pap { normalise = yes } # Instantiating module "auth_log" from file /etc/raddb/mods-enabled/detail.log detail auth_log { filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/auth-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } rlm_detail (auth_log): 'User-Password' suppressed, will not appear in detail output # Instantiating module "reply_log" from file /etc/raddb/mods-enabled/detail.log detail reply_log { filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/reply-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Instantiating module "pre_proxy_log" from file /etc/raddb/mods-enabled/detail.log detail pre_proxy_log { filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/pre-proxy-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Instantiating module "post_proxy_log" from file /etc/raddb/mods-enabled/detail.log detail post_proxy_log { filename = "/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/post-proxy-detail-%Y%m%d" header = "%t" permissions = 384 locking = no log_packet_header = no } # Instantiating module "exec" from file /etc/raddb/mods-enabled/exec exec { wait = no input_pairs = "request" shell_escape = yes timeout = 10 } # Loaded module rlm_preprocess # Instantiating module "preprocess" from file /etc/raddb/mods-enabled/preprocess preprocess { huntgroups = "/etc/raddb/mods-config/preprocess/huntgroups" hints = "/etc/raddb/mods-config/preprocess/hints" with_ascend_hack = no ascend_channels_per_line = 23 with_ntdomain_hack = no with_specialix_jetstream_hack = no with_cisco_vsa_hack = no with_alvarion_vsa_hack = no } reading pairlist file /etc/raddb/mods-config/preprocess/huntgroups reading pairlist file /etc/raddb/mods-config/preprocess/hints # Loaded module rlm_utf8 # Instantiating module "utf8" from file /etc/raddb/mods-enabled/utf8 # Loaded module rlm_logintime # Instantiating module "logintime" from file /etc/raddb/mods-enabled/logintime logintime { minimum_timeout = 60 } # Loaded module rlm_cache # Instantiating module "cache_eap" from file /etc/raddb/mods-enabled/cache_eap cache cache_eap { key = "%{%{control:State}:-%{%{reply:State}:-%{State}}}" ttl = 15 max_entries = 16384 epoch = 0 add_stats = no } # Loaded module rlm_soh # Instantiating module "soh" from file /etc/raddb/mods-enabled/soh soh { dhcp = yes } # Instantiating module "eduroam-eap" from file /etc/raddb/mods-enabled/eduroam-eap eap eduroam-eap { default_eap_type = "peap" timer_expire = 60 ignore_unknown_eap_types = no mod_accounting_username_bug = no max_sessions = 4096 } # Linked to sub-module rlm_eap_gtc gtc { challenge = "Password: " auth_type = "PAP" } # Linked to sub-module rlm_eap_tls tls { tls = "tls-common" } tls-config tls-common { rsa_key_exchange = no dh_key_exchange = yes rsa_key_length = 512 dh_key_length = 512 verify_depth = 0 ca_path = "/etc/raddb/certs" pem_file_type = yes private_key_file = "/etc/pki/tls/private/radius2.ad.kent.ac.uk.key" certificate_file = "/etc/pki/tls/certs/radius2.ad.kent.ac.uk.pem" ca_file = "/etc/pki/tls/certs/globalsign.pem" dh_file = "/etc/raddb/certs/dh" fragment_size = 1024 include_length = yes check_crl = no cipher_list = "DEFAULT" ecdh_curve = "prime256v1" cache { enable = yes lifetime = 24 max_entries = 255 } verify { } ocsp { enable = no override_cert_url = yes url = "http://127.0.0.1/ocsp/" use_nonce = yes timeout = 0 softfail = yes } } # Linked to sub-module rlm_eap_ttls ttls { tls = "tls-common" default_eap_type = "mschapv2" copy_request_to_tunnel = no use_tunneled_reply = no virtual_server = "eduroam-inner-tunnel" include_length = yes require_client_cert = no } Using cached TLS configuration from previous invocation # Linked to sub-module rlm_eap_peap peap { tls = "tls-common" default_method = "mschapv2" copy_request_to_tunnel = no use_tunneled_reply = no proxy_tunneled_request_as_eap = yes virtual_server = "eduroam-inner-tunnel" soh = no require_client_cert = no } Using cached TLS configuration from previous invocation # Linked to sub-module rlm_eap_mschapv2 mschapv2 { with_ntdomain_hack = no send_error = yes } # Instantiating module "sradutmp" from file /etc/raddb/mods-enabled/sradutmp radutmp sradutmp { filename = "/var/log/radius/sradutmp" username = "%{User-Name}" case_sensitive = yes check_with_nas = yes permissions = 420 caller_id = no } # Loaded module rlm_always # Instantiating module "reject" from file /etc/raddb/mods-enabled/always always reject { rcode = "reject" simulcount = 0 mpp = no } # Instantiating module "fail" from file /etc/raddb/mods-enabled/always always fail { rcode = "fail" simulcount = 0 mpp = no } # Instantiating module "ok" from file /etc/raddb/mods-enabled/always always ok { rcode = "ok" simulcount = 0 mpp = no } # Instantiating module "handled" from file /etc/raddb/mods-enabled/always always handled { rcode = "handled" simulcount = 0 mpp = no } # Instantiating module "invalid" from file /etc/raddb/mods-enabled/always always invalid { rcode = "invalid" simulcount = 0 mpp = no } # Instantiating module "userlock" from file /etc/raddb/mods-enabled/always always userlock { rcode = "userlock" simulcount = 0 mpp = no } # Instantiating module "notfound" from file /etc/raddb/mods-enabled/always always notfound { rcode = "notfound" simulcount = 0 mpp = no } # Instantiating module "noop" from file /etc/raddb/mods-enabled/always always noop { rcode = "noop" simulcount = 0 mpp = no } # Instantiating module "updated" from file /etc/raddb/mods-enabled/always always updated { rcode = "updated" simulcount = 0 mpp = no } # Loaded module rlm_realm # Instantiating module "IPASS" from file /etc/raddb/mods-enabled/realm realm IPASS { format = "prefix" delimiter = "/" ignore_default = no ignore_null = no } # Instantiating module "suffix" from file /etc/raddb/mods-enabled/realm realm suffix { format = "suffix" delimiter = "@" ignore_default = no ignore_null = no } # Instantiating module "realmpercent" from file /etc/raddb/mods-enabled/realm realm realmpercent { format = "suffix" delimiter = "%" ignore_default = no ignore_null = no } # Instantiating module "ntdomain" from file /etc/raddb/mods-enabled/realm realm ntdomain { format = "prefix" delimiter = "\" ignore_default = no ignore_null = no } # Loaded module rlm_replicate # Instantiating module "replicate" from file /etc/raddb/mods-enabled/replicate # Instantiating module "echo" from file /etc/raddb/mods-enabled/echo exec echo { wait = yes program = "/bin/echo %{User-Name}" input_pairs = "request" output_pairs = "reply" shell_escape = yes } # Loaded module rlm_dhcp # Instantiating module "dhcp" from file /etc/raddb/mods-enabled/dhcp # Loaded module rlm_sql # Instantiating module "sql" from file /etc/raddb/mods-enabled/sql sql { driver = "rlm_sql_mysql" server = "mallard.kent.ac.uk" port = "3306" login = "freeradius" password = <<< secret >>> radius_db = "freeradius" read_groups = yes read_profiles = yes read_clients = no delete_stale_sessions = yes sql_user_name = "" default_user_profile = "" client_query = "SELECT id,nasname,shortname,type,secret FROM nas" authorize_check_query = "" authorize_group_check_query = "" authorize_group_reply_query = "" simul_count_query = "" simul_verify_query = "" safe_characters = "@abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /" } rlm_sql (sql): Couldn't find configuration for accounting, will return NOOP for calls from this section rlm_sql (sql): Couldn't find configuration for post-auth, will return NOOP for calls from this section mysql { tls { } } rlm_sql (sql): Driver rlm_sql_mysql (module rlm_sql_mysql) loaded and linked rlm_sql (sql): Attempting to connect to database "freeradius" rlm_sql (sql): Initialising connection pool pool { start = 5 min = 4 max = 64 spare = 3 uses = 0 lifetime = 0 cleanup_interval = 30 idle_timeout = 60 retry_delay = 1 spread = no } rlm_sql (sql): Opening additional connection (0) rlm_sql_mysql: Starting connect to MySQL server rlm_sql (sql): Opening additional connection (1) rlm_sql_mysql: Starting connect to MySQL server rlm_sql (sql): Opening additional connection (2) rlm_sql_mysql: Starting connect to MySQL server rlm_sql (sql): Opening additional connection (3) rlm_sql_mysql: Starting connect to MySQL server rlm_sql (sql): Opening additional connection (4) rlm_sql_mysql: Starting connect to MySQL server } # modules radiusd: #### Loading Virtual Servers #### server { # from file /etc/raddb/radiusd.conf } # server server default { # from file /etc/raddb/sites-enabled/default # Creating Auth-Type = digest # Loading authenticate {...} # Loading authorize {...} # Loading preacct {...} # Loading accounting {...} # Loading post-proxy {...} # Loading post-auth {...} } # server default server eduroam-inner-tunnel { # from file /etc/raddb/sites-enabled/eduroam-inner-tunnel # Creating Auth-Type = eduroam-eap # Loading authenticate {...} # Loading authorize {...} # Loading post-proxy {...} # Loading post-auth {...} } # server eduroam-inner-tunnel server kent-eduroam { # from file /etc/raddb/sites-enabled/kent-eduroam # Creating Autz-Type = Status-Server # Creating Acct-Type = Status-Server # Loading authenticate {...} # Loading authorize {...} # Loading preacct {...} # Loading accounting {...} # Loading pre-proxy {...} # Loading post-proxy {...} # Loading post-auth {...} } # server kent-eduroam server eduroam { # from file /etc/raddb/sites-enabled/eduroam # Loading authenticate {...} # Loading authorize {...} # Loading preacct {...} # Loading accounting {...} # Loading pre-proxy {...} # Loading post-proxy {...} # Loading post-auth {...} } # server eduroam server inner-tunnel { # from file /etc/raddb/sites-enabled/inner-tunnel # Loading authenticate {...} # Loading authorize {...} # Loading post-proxy {...} # Loading post-auth {...} } # server inner-tunnel server check-eap-tls { # from file /etc/raddb/sites-enabled/check-eap-tls # Loading authorize {...} } # server check-eap-tls radiusd: #### Opening IP addresses and Ports #### listen { type = "control" listen { socket = "/var/run/radiusd/radiusd.sock" uid = "radiusd" gid = "radiusd" mode = "rw" } } listen { type = "auth" ipaddr = * port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "acct" ipaddr = * port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "auth" ipv6addr = :: port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "acct" ipv6addr = :: port = 0 limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } listen { type = "auth" ipaddr = 127.0.0.1 port = 18121 } listen { type = "auth" ipaddr = 127.0.0.1 port = 18120 } Listening on command file /var/run/radiusd/radiusd.sock Listening on auth address * port 1812 as server default Listening on acct address * port 1813 as server default Listening on auth address :: port 1812 as server default Listening on acct address :: port 1813 as server default Listening on auth address 127.0.0.1 port 18121 as server eduroam-inner-tunnel Listening on auth address 127.0.0.1 port 18120 as server inner-tunnel Opening new proxy socket 'proxy address * port 0' Listening on proxy address * port 50659 Ready to process requests On 13/04/16 14:39, David Hartburn wrote:
Hi,
Thanks for this response. The problem appears to be load related, so this will have to be done on the full production. It has never appeared on either our test/dev box or handling just our roaming users. It has been sending campus traffic that has generated the problem. It is not restricted to one client.
It seems to report every few minutes, so I will try doing a -X for a 5 minute spell and see if the error is captured. I will post back what I find.
Dave
On 13/04/16 14:29, Matthew Newton wrote:
On Wed, Apr 13, 2016 at 02:10:14PM +0100, David Hartburn wrote:
Sorry to waste time, but do you mean the full log from doing a 'radius -X'?
That's what contains the required information.
I want to clarify because on a production server that will be a huge log. I am happy to produce it though.
Depending on the problem, you can sometimes be more specific about what you log by using radmin. First run radiusd -X and capture the output to the end of the startup (when it reports about Listening for packets). That grabs all the useful config information.
Then you use radmin to capture auth information for a particular client. There's some information on the FR web page: http://freeradius.org/radiusd/man/radmin.html
I did a post on how you can filter out by attribute a while back: http://notes.asd.me.uk/2014/09/16/debugging-freeradius-packets-with-radmin/
But the best way really is to reproduce it on a test RADIUS server with one or two clients. That doesn't have to be hard - a great way is to add proxy config to your production servers over to a test server. When you want to capture something like this, just proxy particular client(s) over using unlang to select based on some attributes. Could be MAC address, realm, SSID, location etc. Makes testing things much easier and more flexible, and of course the debug logs on the test server will be smaller.
Matthew
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html