Control socket directory doesn't exist after systemd stopping service
Hi all, I have a weird bug where the control socket directory `/var/run/radiusd` gets deleted after `systemctl stop radiusd`, and `radiusd -X` doesn't re-create it. I'm using the Rocky repo as there's no AlmaLinux repo available, although I'm expecting this would be compatible. Extra information is below in case it's needed for context. Excerpt from radiusd -X output: radiusd: #### Opening IP addresses and Ports #### listen { type = "control" listen { socket = "/var/run/radiusd/radiusd.sock" mode = "rw" peercred = yes } Failed creating control socket "/var/run/radiusd/radiusd.sock": Failed binding to /var/run/radiusd/radiusd.sock: No such file or directory The directory doesn't exist: # ls -lah /var/run/radiusd ls: cannot access '/var/run/radiusd': No such file or directory # cat /etc/os-release NAME="AlmaLinux" VERSION="8.6 (Sky Tiger)" ID="almalinux" ID_LIKE="rhel centos fedora" VERSION_ID="8.6" PLATFORM_ID="platform:el8" PRETTY_NAME="AlmaLinux 8.6 (Sky Tiger)" ANSI_COLOR="0;34" CPE_NAME="cpe:/o:almalinux:almalinux:8::baseos" HOME_URL=https://almalinux.org/ DOCUMENTATION_URL=https://wiki.almalinux.org/ BUG_REPORT_URL=https://bugs.almalinux.org/ ALMALINUX_MANTISBT_PROJECT="AlmaLinux-8" ALMALINUX_MANTISBT_PROJECT_VERSION="8.6" # yum list freeradius Last metadata expiration check: 2:53:04 ago on Wed 30 Nov 2022 06:40:50 NZDT. Installed Packages freeradius.x86_64 3.2.1-1.el8 @networkradius # cat /etc/yum.repos.d/networkradius.repo [networkradius] name=NetworkRADIUS-$releasever baseurl=http://packages.networkradius.com/freeradius-3.2/rocky/$releasever/ enabled=1 gpgcheck=1 gpgkey=https://packages.networkradius.com/pgp/packages%40networkradius.com I tried radiusd -X with strace attached, showed the same thing, excerpt below: write(1, "listen {\n", 9listen { ) = 9 write(1, " \ttype = \"control\"\n", 19 type = "control" ) = 19 write(1, " listen {\n", 10 listen { ) = 10 write(1, " \tsocket = \"/var/run/radiusd/rad"..., 43 socket = "/var/run/radiusd/radiusd.sock" ) = 43 write(1, " \tmode = \"rw\"\n", 14 mode = "rw" ) = 14 write(1, " \tpeercred = yes\n", 17 peercred = yes ) = 17 write(1, " }\n", 3 } ) = 3 socket(AF_UNIX, SOCK_STREAM, 0) = 22 stat("/var/run/radiusd/radiusd.sock", 0x7fff5e30c580) = -1 ENOENT (No such file or directory) bind(22, {sa_family=AF_UNIX, sun_path="/var/run/radiusd/radiusd.sock"}, 31) = -1 ENOENT (No such file or directory) futex(0x7fe3b3ac0840, FUTEX_WAKE_PRIVATE, 2147483647) = 0 close(22) = 0 write(1, "\33[31mFailed creating control soc"..., 148Failed creating control socket "/var/run/radiusd/radiusd.sock": Failed binding to /var/run/radiusd/radiusd.sock: No such file or directory ) = 148 write(2, "Debugger detected, raising SIGTR"..., 35Debugger detected, raising SIGTRAP ) = 35 rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0 getpid() = 154664 gettid() = 154664 tgkill(154664, 154664, SIGTRAP) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 --- SIGTRAP {si_signo=SIGTRAP, si_code=SI_TKILL, si_pid=154664, si_uid=0} --- +++ killed by SIGTRAP (core dumped) +++ Trace/breakpoint trap (core dumped) If I enable the dnf module for freeradius from the AlmaLinux appstream repo, I get an older version. I list duplicates from the networkradius repo first as a baseline. # dnf --showduplicates list freeradius Last metadata expiration check: 0:04:02 ago on Wed 30 Nov 2022 09:45:47 NZDT. Installed Packages freeradius.x86_64 3.2.1-1.el8 @networkradius Available Packages freeradius.x86_64 3.2.0-1.el8 networkradius freeradius.x86_64 3.2.1-1.el8 networkradius # dnf module list freeradius Last metadata expiration check: 0:00:58 ago on Wed 30 Nov 2022 09:45:47 NZDT. AlmaLinux 8 - AppStream Name Stream Profiles Summary freeradius 3.0 [d][x] server [d] High-performance and highly configurable free RADIUS server Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled # dnf module enable freeradius Last metadata expiration check: 0:01:12 ago on Wed 30 Nov 2022 09:45:47 NZDT. Dependencies resolved. ============================================================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================================================= Enabling module streams: freeradius 3.0 Transaction Summary ============================================================================================================================================================= Is this ok [y/N]: y Complete! # dnf --showduplicates list freeradius Last metadata expiration check: 0:01:19 ago on Wed 30 Nov 2022 09:45:47 NZDT. Installed Packages freeradius.x86_64 3.2.1-1.el8 @networkradius Available Packages freeradius.x86_64 3.0.20-12.module_el8.6.0+2865+fd6928c8 appstream Warm Regards, .[cid:vg-email_e752d47f-182a-48d0-8808-3796b2df8d4a.png]<https://www.vetta.nz> .[cid:058_line_9bae9502-1e3a-48a3-ba5e-34ed9cd44185.png] Matthew McTague Head of Infrastructure [cid:bea-finalist-email_2ddba0ed-b9bb-4ab9-b680-84aa7f54077f.png] [cid:bea-winner-banner-email_590096c3-05a6-4c98-b071-1c4d748c3534.png] . D +64 3 222 6013<tel:+64%203%20222%206013> | P +64 3 684 5770 E matthew.mctague@vetta.nz<mailto:matthew.mctague@vetta.nz> | W www.vetta.nz<https://www.vetta.nz> ? Christmas/New Year Holiday Period ? During the period from 22rd December to 9th January, our offices will be closed, with our teams available via our after-hours service methods, focused on service continuity and restoration only. For more details, click here<https://vetta.link/xmas22email> . Vetta Group Ltd and its subsidiaries accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company, its subsidiaries, partners or contractors. .
On Nov 29, 2022, at 3:52 PM, Matthew McTague via Freeradius-Users <freeradius-users@lists.freeradius.org> wrote:
I have a weird bug where the control socket directory `/var/run/radiusd` gets deleted after `systemctl stop radiusd`, and `radiusd -X` doesn't re-create it.
Depending on the OS and configuration, the server doesn't create that directory. In this case, it's relying on systemd to create it.
I'm using the Rocky repo as there's no AlmaLinux repo available, although I'm expecting this would be compatible.
Extra information is below in case it's needed for context.
Excerpt from radiusd -X output:
radiusd: #### Opening IP addresses and Ports #### listen { type = "control" listen { socket = "/var/run/radiusd/radiusd.sock" mode = "rw" peercred = yes } Failed creating control socket "/var/run/radiusd/radiusd.sock": Failed binding to /var/run/radiusd/radiusd.sock: No such file or directory
Was this run as root? Or as a non-root user?
The directory doesn't exist:
# ls -lah /var/run/radiusd ls: cannot access '/var/run/radiusd': No such file or directory
How about: $ mkdir /var/runradiusd && radiusd -X ?
I tried radiusd -X with strace attached, showed the same thing, excerpt below: ... socket(AF_UNIX, SOCK_STREAM, 0) = 22 stat("/var/run/radiusd/radiusd.sock", 0x7fff5e30c580) = -1 ENOENT (No such file or directory) bind(22, {sa_family=AF_UNIX, sun_path="/var/run/radiusd/radiusd.sock"}, 31) = -1 ENOENT (No such file or directory)
Yup. It's requiring that the directory exist. Because generally the server shouldn't have write permission to /var/run. That directory is owned by "root", and it's very bad to have a non-root program write to it. The directory /var/run/radiusd should be owned by "radiusd", and the server should have write permission to that directory. So... make sure that the directory exists before starting the server in debug mode. Alan DeKok.
Correcting the previous command I sent: mkdir -p /var/run/radiusd && chown radiusd:radiusd /var/run/radiusd && chmod 750 /var/run/radiusd && radiusd -X Warm Regards, .[cid:vg-email_e752d47f-182a-48d0-8808-3796b2df8d4a.png]<https://www.vetta.nz> .[cid:058_line_9bae9502-1e3a-48a3-ba5e-34ed9cd44185.png] Matthew McTague Head of Infrastructure [cid:bea-finalist-email_2ddba0ed-b9bb-4ab9-b680-84aa7f54077f.png] [cid:bea-winner-banner-email_590096c3-05a6-4c98-b071-1c4d748c3534.png] . D +64 3 222 6013<tel:+64%203%20222%206013> | P +64 3 684 5770 E matthew.mctague@vetta.nz<mailto:matthew.mctague@vetta.nz> | W www.vetta.nz<https://www.vetta.nz> 🎄 Christmas/New Year Holiday Period 🎄 During the period from 22rd December to 9th January, our offices will be closed, with our teams available via our after-hours service methods, focused on service continuity and restoration only. For more details, click here<https://vetta.link/xmas22email> . Vetta Group Ltd and its subsidiaries accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company, its subsidiaries, partners or contractors. . From: Matthew McTague <matthew.mctague@vetta.nz> Sent: Wednesday, 30 November 2022 4:27 pm To: Alan DeKok <aland@deployingradius.com>; FreeRadius users mailing list <freeradius-users@lists.freeradius.org> Subject: RE: Control socket directory doesn't exist after systemd stopping service Hi Alan, If I create the directory before running debug, with the below, yes it works. mkdir -p /var/run/radiusd && chown radiusd:radiusd /var/run/radiusd && chmod 750 /var/run/radiusd radiusd -X Is there any better / transparent way to do this, or to stop systemd from removing the directory when stopping the service? Or would it be better to use /var/run/radiusd.sock for the socket and avoid the usage of a subdirectory? Warm Regards, .[cid:image001.png@01D904D8.E91AB930]<https://www.vetta.nz/> .[cid:image002.png@01D904D8.E91AB930] Matthew McTague Head of Infrastructure [cid:image003.png@01D904D8.E91AB930][cid:image004.png@01D904D8.E91AB930]. D +64 3 222 6013<tel:+64%203%20222%206013> | P +64 3 684 5770 E matthew.mctague@vetta.nz<mailto:matthew.mctague@vetta.nz> | W www.vetta.nz<https://www.vetta.nz> 🎄 Christmas/New Year Holiday Period 🎄 During the period from 22rd December to 9th January, our offices will be closed, with our teams available via our after-hours service methods, focused on service continuity and restoration only. For more details, click here<https://vetta.link/xmas22email> . Vetta Group Ltd and its subsidiaries accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company, its subsidiaries, partners or contractors. . -----Original Message----- From: Alan DeKok <aland@deployingradius.com<mailto:aland@deployingradius.com>> Sent: Wednesday, 30 November 2022 10:41 am To: FreeRadius users mailing list <freeradius-users@lists.freeradius.org<mailto:freeradius-users@lists.freeradius.org>> Cc: Matthew McTague <matthew.mctague@vetta.nz<mailto:matthew.mctague@vetta.nz>> Subject: Re: Control socket directory doesn't exist after systemd stopping service On Nov 29, 2022, at 3:52 PM, Matthew McTague via Freeradius-Users <freeradius-users@lists.freeradius.org<mailto:freeradius-users@lists.freeradius.org>> wrote:
I have a weird bug where the control socket directory `/var/run/radiusd` gets deleted after `systemctl stop radiusd`, and `radiusd -X` doesn't re-create it.
Depending on the OS and configuration, the server doesn't create that directory. In this case, it's relying on systemd to create it.
I'm using the Rocky repo as there's no AlmaLinux repo available, although I'm expecting this would be compatible.
Extra information is below in case it's needed for context.
Excerpt from radiusd -X output:
radiusd: #### Opening IP addresses and Ports #### listen { type = "control" listen { socket = "/var/run/radiusd/radiusd.sock" mode = "rw" peercred = yes } Failed creating control socket "/var/run/radiusd/radiusd.sock": Failed binding to /var/run/radiusd/radiusd.sock: No such file or directory
Was this run as root? Or as a non-root user?
The directory doesn't exist:
# ls -lah /var/run/radiusd ls: cannot access '/var/run/radiusd': No such file or directory
How about: $ mkdir /var/runradiusd && radiusd -X ?
I tried radiusd -X with strace attached, showed the same thing, excerpt below: ... socket(AF_UNIX, SOCK_STREAM, 0) = 22 stat("/var/run/radiusd/radiusd.sock", 0x7fff5e30c580) = -1 ENOENT (No such file or directory) bind(22, {sa_family=AF_UNIX, sun_path="/var/run/radiusd/radiusd.sock"}, 31) = -1 ENOENT (No such file or directory)
Yup. It's requiring that the directory exist. Because generally the server shouldn't have write permission to /var/run. That directory is owned by "root", and it's very bad to have a non-root program write to it. The directory /var/run/radiusd should be owned by "radiusd", and the server should have write permission to that directory. So... make sure that the directory exists before starting the server in debug mode. Alan DeKok.
On 30/11/2022 03:29, Matthew McTague via Freeradius-Users wrote:
Correcting the previous command I sent:
mkdir -p /var/run/radiusd && chown radiusd:radiusd /var/run/radiusd && chmod 750 /var/run/radiusd && radiusd -X
Warm Regards,
An alternative (depending on the version of systemd - Rocky is fine for this) is to add a systemd override Create /etc/systemd/system/radiusd.service.d/override.conf with contents like [Service] User=radiusd Group=radiusd RuntimeDirectory=radiusd RuntimeDirectoryPreserve=yes RuntimeDirectoryMode=0770 That will ensure that /var/run/radiusd is created by systemd when the radiusd service is started, but it won't be deleted when the service stops. If you put your control socket in a subdirectory of /var/run/radiusd then you can add that directory to the RuntimeDirectory line e.g. RuntimeDirectory=radiusd radiusd/control will create both /var/run/radiusd and /var/run/radiusd/control. On Debian / Ubuntu machines the names change - the service is freeradius and the user / group are freerad. Older RHEL family systems are more of a pain as they don't have the RuntimeDirectoryPreserve option which is key to this working. Nick
On Nov 29, 2022, at 10:26 PM, Matthew McTague <matthew.mctague@vetta.nz> wrote:
Is there any better / transparent way to do this, or to stop systemd from removing the directory when stopping the service?
See Nick's reply for details.
Or would it be better to use /var/run/radiusd.sock for the socket and avoid the usage of a subdirectory?
Absolutely not. As I said, the radius service should not have write permissions to the /var/run directory. It needs a subdirectory for security reasons. Alan DeKok.
participants (3)
-
Alan DeKok -
Matthew McTague -
Nick Porter