RadSec client FR server stalls when offline
Marija Milojkovic
marija.milojkovic at htecgroup.com
Tue Mar 26 14:06:38 UTC 2024
Hello,
We have a setup where we have Local FR server (in WLAN, with caching) used as Radsec Proxy to Central Radius Server (in the cloud). We are obligated to use tcp/tls for Local-Central FR communication.
NAS calls Local FR, local FR checks cache, and if not found propagates request to Central FR. If Central FR replies, we cache result. If it doesn’t reply with some very short timeout, we give users access to WLAN with short lived session (so NAS will retry in short time).
All worked fine except in the case where Local Radius is offline.
If Local Radius doesn’t have access to internet, it stalls for few minutes (instead of few seconds) until it marks Central Radius home server dead, which really makes our setup unusable.
Also, looks like when photo = tcp is used, status_check = "status-server” and connected config params are not used, and it revives home server with fixed revive_interval….it doesn’t do any checks in the mean time, which is bad, because we would like to know when home server is available (we are back online) as soon as possible, and also would’t like to mark it alive if it is not (both of which check_interval with check_timeout would solve)….
Is this bug/known issue/any chance it gets solved in 3.2.x?
What would be the best alternative solution?
We were thinking to split Local FR into two: one that will do the caching and other that will just be used as a Central home server proxy (so we can do home server UDP setup in Local Caching FR that will call Local Proxy FR which will do “real” proxing with tcp/tls, so even if it gets stuck, we don’t block our users).
Thank you for your time and help!
Best regards,
Marija
P.s. Below are our config and reply logs
Here is our current home sever setup in Local FR from logs (I’ve just masked ipaddr):
home_server_pool central_radius_auth_pool {
type = fail-over
home_server = central_radius
home_server = mcl_tmp_session_vs
}
home_server_pool central_radius_acct_pool {
type = fail-over
home_server = central_radius
}
realm central_radius {
auth_pool = central_radius_auth_pool
acct_pool = central_radius_acct_pool
}
Ignoring "response_window = 20.000000", forcing to "response_window = 10.000000"
home_server central_radius {
nonblock = yes
ipaddr = xx.xx.xx.xx
port = 443
type = "auth+acct"
proto = "tcp"
response_window = 3.000000
response_timeouts = 1
max_outstanding = 65536
zombie_period = 5
status_check = "status-server"
ping_interval = 5
check_interval = 5
check_timeout = 3
num_answers_to_alive = 1
revive_interval = 300
limit {
max_connections = 16
max_requests = 0
lifetime = 30
idle_timeout = 1
}
coa {
irt = 2
mrt = 16
mrc = 5
mrd = 30
}
recv_coa {
}
}
tls {
verify_depth = 0
pem_file_type = yes
private_key_file = "/etc/freeradius/certs/client.pem"
certificate_file = "/etc/freeradius/certs/client.pem"
ca_file = "/etc/freeradius/certs/ca.pem"
private_key_password = <<< secret >>>
fragment_size = 1024
include_length = yes
check_crl = no
ca_path_reload_interval = 0
ecdh_curve = "prime256v1"
tls_min_version = "1.2"
}
connect_timeout = 1
Ignoring "ping_interval = 5", forcing to "ping_interval = 6"
Ignoring "num_pings_to_alive = 1", forcing to "num_pings_to_alive = 3"
And here are the logs (I’ve deleted few detail logs and masked ipaddr, but since it wakes up every 5s, you can see that it took def more than connect_timeout + idle_timeout/):
(0) Received Access-Request Id 47 from 127.0.0.1:41175 to 127.0.0.1:1812 length 155
(0) User-Name = "22-99-22-99-22-99"
(0) User-Password = "standardpass"
(0) NAS-IP-Address = 127.0.0.1
(0) NAS-Port = 10
(0) Message-Authenticator = 0x3f3aa23a8ee65e2ce1dc72db1bf2eca6
(0) Calling-Station-Id = "22-99-22-99-22-99"
(0) Called-Station-Id = "11-22-33-44-55-66:Wi-Fi"
(0) Acct-Session-Id = "SomeUUID-1p77777"
(0) # Executing section authorize from file /etc/freeradius/sites-enabled/default
(0) authorize {
(0) policy rewrite_calling_station_id {
(0) if (&Calling-Station-Id && (&Calling-Station-Id =~ /^([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})$/i)) {
(0) if (&Calling-Station-Id && (&Calling-Station-Id =~ /^([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})$/i)) -> TRUE
(0) if (&Calling-Station-Id && (&Calling-Station-Id =~ /^([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})$/i)) {
(0) update request {
(0) EXPAND %{toupper:%{1}-%{2}-%{3}-%{4}-%{5}-%{6}}
(0) --> 22-99-22-99-22-99
(0) &Calling-Station-Id := 22-99-22-99-22-99
(0) } # update request = noop
(0) [updated] = updated
(0) } # if (&Calling-Station-Id && (&Calling-Station-Id =~ /^([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})[^0-9a-f]?([0-9a-f]{2})$/i)) = updated
(0) ... skipping else: Preceding "if" was taken
(0) } # policy rewrite_calling_station_id = updated
(0) policy filter_username {
(0) if (&User-Name) {
(0) if (&User-Name) -> TRUE
(0) if (&User-Name) {
(0) if (&User-Name =~ / /) {
(0) if (&User-Name =~ / /) -> FALSE
(0) if (&User-Name =~ /@[^@]*@/ ) {
(0) if (&User-Name =~ /@[^@]*@/ ) -> FALSE
(0) if (&User-Name =~ /\.\./ ) {
(0) if (&User-Name =~ /\.\./ ) -> FALSE
(0) if ((&User-Name =~ /@/) && (&User-Name !~ /@(.+)\.(.+)$/)) {
(0) if ((&User-Name =~ /@/) && (&User-Name !~ /@(.+)\.(.+)$/)) -> FALSE
(0) if (&User-Name =~ /\.$/) {
(0) if (&User-Name =~ /\.$/) -> FALSE
(0) if (&User-Name =~ /@\./) {
(0) if (&User-Name =~ /@\./) -> FALSE
(0) } # if (&User-Name) = updated
(0) } # policy filter_username = updated
(0) [preprocess] = ok
(0) [chap] = noop
(0) [mschap] = noop
(0) [digest] = noop
(0) suffix: Checking for suffix after "@"
(0) suffix: No '@' in User-Name = "22-99-22-99-22-99", looking up realm NULL
(0) suffix: No such realm "NULL"
(0) [suffix] = noop
(0) eap: No EAP-Message, not doing EAP
(0) [eap] = noop
(0) [files] = noop
(0) [expiration] = noop
(0) [logintime] = noop
(0) if (!request:EAP-Message){
(0) if (!request:EAP-Message) -> TRUE
(0) if (!request:EAP-Message) {
(0) update control {
(0) &Cache-Status-Only := yes
(0) } # update control = noop
(0) cache_auth: EXPAND %{User-Name}
(0) cache_auth: --> 22-99-22-99-22-99
rlm_redis (redis): Reserved connection (0)
(0) cache_auth: No cache entry found for "22-99-22-99-22-99"
rlm_redis (redis): Released connection (0)
Need more connections to reach 10 spares
rlm_redis (redis): Opening additional connection (5), 1 of 27 pending slots used
(0) [cache_auth] = notfound
(0) if (ok) {
(0) if (ok) -> FALSE
(0) else {
(0) policy proxy_to_central_radius {
(0) update control {
(0) &Proxy-To-Realm := central_radius
(0) } # update control = noop
(0) } # policy proxy_to_central_radius = noop
(0) } # else = noop
(0) } # if (!request:EAP-Message) = noop
(0) [pap] = noop
(0) } # authorize = updated
(0) Starting proxy to home server xx.xx.xx.xx port 443
(0) server default {
(0) }
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 5.231368 sec
################### 22 more Detail logs #####################################################
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 5.233863 sec
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 4.945636 sec
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 4.970738 sec
Failed opening new proxy socket 'proxy (0.0.0.0, 0) -> home_server (xx.xx.xx.xx, 443)' : Failed connecting socket: Connection timed out
(0) Failed to insert request into the proxy list
Proxying to virtual server mcl_tmp_session_vs
(0) # Executing section authorize from file /etc/freeradius/sites-enabled/mcl_tmp_session_vs
(0) authorize {
(0) policy filter_username {
(0) if (&User-Name) {
(0) if (&User-Name) -> TRUE
(0) if (&User-Name) {
(0) if (&User-Name =~ / /) {
(0) if (&User-Name =~ / /) -> FALSE
(0) if (&User-Name =~ /@[^@]*@/ ) {
(0) if (&User-Name =~ /@[^@]*@/ ) -> FALSE
(0) if (&User-Name =~ /\.\./ ) {
(0) if (&User-Name =~ /\.\./ ) -> FALSE
(0) if ((&User-Name =~ /@/) && (&User-Name !~ /@(.+)\.(.+)$/)) {
(0) if ((&User-Name =~ /@/) && (&User-Name !~ /@(.+)\.(.+)$/)) -> FALSE
(0) if (&User-Name =~ /\.$/) {
(0) if (&User-Name =~ /\.$/) -> FALSE
(0) if (&User-Name =~ /@\./) {
(0) if (&User-Name =~ /@\./) -> FALSE
(0) } # if (&User-Name) = notfound
(0) } # policy filter_username = notfound
(0) [preprocess] = ok
(0) suffix: Request already has destination realm set. Ignoring
(0) [suffix] = noop
(0) eap: No EAP-Message, not doing EAP
(0) [eap] = noop
(0) [expiration] = noop
(0) [logintime] = noop
(0) pap: WARNING: No "known good" password found for the user. Not setting Auth-Type
(0) pap: WARNING: Authentication will fail unless a "known good" password is available
(0) [pap] = noop
(0) if (!request:EAP-Message){
(0) if (!request:EAP-Message) -> TRUE
(0) if (!request:EAP-Message) {
(0) update control {
(0) &Auth-Type := Accept
(0) } # update control = noop
(0) } # if (!request:EAP-Message) = noop
(0) } # authorize = ok
(0) Found Auth-Type = Accept
(0) Auth-Type = Accept, accepting the user
(0) # Executing section post-auth from file /etc/freeradius/sites-enabled/mcl_tmp_session_vs
(0) post-auth {
(0) if (session-state:User-Name && reply:User-Name && request:User-Name && (reply:User-Name == request:User-Name)) {
(0) if (session-state:User-Name && reply:User-Name && request:User-Name && (reply:User-Name == request:User-Name)) -> FALSE
(0) update {
(0) No attributes updated for RHS &session-state:
(0) } # update = noop
(0) [exec] = noop
(0) policy insert_acct_class {
(0) update reply {
(0) EXPAND ai:%{md5:%t,%{Packet-Src-Port},%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}},%{NAS-IP-Address},%{Calling-Station-ID},%{User-Name},%{session-state:User-Name} }
(0) --> ai:b11c60a3bf271cd382e057a3e96e89b0
(0) &Class = 0x61693a6231316336306133626632373163643338326530353761336539366538396230
(0) } # update reply = noop
(0) } # policy insert_acct_class = noop
(0) policy remove_reply_message_if_eap {
(0) if (&reply:EAP-Message && &reply:Reply-Message) {
(0) if (&reply:EAP-Message && &reply:Reply-Message) -> FALSE
(0) else {
(0) [noop] = noop
(0) } # else = noop
(0) } # policy remove_reply_message_if_eap = noop
(0) policy mcl_tmp_session {
(0) update control {
(0) EXPAND %{expr: 900 * (-1)}
(0) --> -900
(0) &Cache-TTL := -900
(0) &Cache-Merge := no
(0) } # update control = noop
(0) cache_tmp_auth: EXPAND %{User-Name}-%{Acct-Session-Id}-Tmp
(0) cache_tmp_auth: --> 22-99-22-99-22-99-SomeUUID-1p77777-Tmp
rlm_redis (redis): Reserved connection (1)
(0) cache_tmp_auth: No cache entry found for "22-99-22-99-22-99-SomeUUID-1p77777-Tmp"
(0) cache_tmp_auth: Creating new cache entry
(0) cache_tmp_auth: Committed entry, TTL 900 seconds
rlm_redis (redis): Released connection (1)
Need more connections to reach 10 spares
rlm_redis (redis): Opening additional connection (6), 1 of 26 pending slots used
(0) [cache_tmp_auth] = updated
(0) update reply {
(0) EXPAND Central Radius unavailable, temporary session at %t
(0) --> Central Radius unavailable, temporary session at Tue Mar 26 11:11:34 2024
(0) &Reply-Message += Central Radius unavailable, temporary session at Tue Mar 26 11:11:34 2024
(0) &Session-Timeout := 900
(0) &Filter-Id := session: tmp
(0) &Filter-Id += access: captive
(0) } # update reply = noop
(0) } # policy mcl_tmp_session = updated
(0) if (EAP-Key-Name && &reply:EAP-Session-Id) {
(0) if (EAP-Key-Name && &reply:EAP-Session-Id) -> FALSE
(0) } # post-auth = updated
(0) Finished internally proxied request.
(0) Clearing existing &reply: attributes
(0) server mcl_tmp_session_vs {
(0) # Executing section post-proxy from file /etc/freeradius/sites-enabled/mcl_tmp_session_vs
(0) post-proxy {
(0) eap: No pre-existing handler found
(0) [eap] = noop
(0) } # post-proxy = noop
(0) }
(0) Found Auth-Type = Accept
(0) Auth-Type = Accept, accepting the user
(0) # Executing section post-auth from file /etc/freeradius/sites-enabled/default
(0) post-auth {
(0) if (session-state:User-Name && reply:User-Name && request:User-Name && (reply:User-Name == request:User-Name)) {
(0) if (session-state:User-Name && reply:User-Name && request:User-Name && (reply:User-Name == request:User-Name)) -> FALSE
(0) update {
(0) No attributes updated for RHS &session-state:
(0) } # update = noop
(0) [exec] = noop
(0) policy insert_acct_class {
(0) update reply {
(0) EXPAND ai:%{md5:%t,%{Packet-Src-Port},%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}},%{NAS-IP-Address},%{Calling-Station-ID},%{User-Name},%{session-state:User-Name} }
(0) --> ai:b11c60a3bf271cd382e057a3e96e89b0
(0) &Class = 0x61693a6231316336306133626632373163643338326530353761336539366538396230
(0) } # update reply = noop
(0) } # policy insert_acct_class = noop
(0) policy remove_reply_message_if_eap {
(0) if (&reply:EAP-Message && &reply:Reply-Message) {
(0) if (&reply:EAP-Message && &reply:Reply-Message) -> FALSE
(0) else {
(0) [noop] = noop
(0) } # else = noop
(0) } # policy remove_reply_message_if_eap = noop
(0) if (EAP-Key-Name && &reply:EAP-Session-Id) {
(0) if (EAP-Key-Name && &reply:EAP-Session-Id) -> FALSE
(0) } # post-auth = noop
(0) Sent Access-Accept Id 47 from 127.0.0.1:1812 to 127.0.0.1:41175 length 197
(0) Class = 0x61693a6231316336306133626632373163643338326530353761336539366538396230
(0) Reply-Message += "Central Radius unavailable, temporary session at Tue Mar 26 11:11:34 2024"
(0) Session-Timeout := 900
(0) Filter-Id := "session: tmp"
(0) Filter-Id += "class: Unknown"
(0) Filter-Id += "access: captive"
(0) Finished request
Waking up in 4.9 seconds.
(0) Sending duplicate reply to client ap port 41175 - ID: 47
Waking up in 9.9 seconds.
(0) Sending duplicate reply to client ap port 41175 - ID: 47
Waking up in 19.9 seconds.
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 4.763429 sec
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 4.818341 sec
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 5.220757 sec
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Detail listener state unopened waiting 5.031610 sec
Marking home server 54.75.254.205 port 443 as dead.
PING: Reviving home server central_radius in 300 seconds
(0) Cleaning up request packet ID 47 with timestamp +118 due to cleanup_delay was reached
Waking up in 299.9 seconds.
Waking up in 299.9 seconds.
detail (/var/log/freeradius/radacct/proxy_to_central/detail-*:*): Polling for detail file
More information about the Freeradius-Users
mailing list