Empty packet, socket not ready message
Hi, We've just upgraded one of our outward facing freeradius servers to Ubuntu 14.04. After the upgrade I rebuilt FR 2.2.9 and while tings are working, I'm now getting a lot of Error: Empty packey: socket not ready messages debug trace shows Thu Jan 14 13:45:54 2016 : Info: [attr_filter.access_reject] expand: %{User-Name} -> as1129@york .ac.uk Thu Jan 14 13:45:54 2016 : Debug: attr_filter: Matched entry DEFAULT at line 11 Thu Jan 14 13:45:54 2016 : Info: ++[attr_filter.access_reject] = updated Thu Jan 14 13:45:54 2016 : Info: +} # group REJECT = updated Thu Jan 14 13:45:54 2016 : Info: Delaying reject of request 67 for 1 seconds Thu Jan 14 13:45:54 2016 : Debug: Going to the next request Thu Jan 14 13:45:54 2016 : Debug: Thread 3 waiting to be assigned a request Thu Jan 14 13:45:55 2016 : Info: Sending delayed reject for request 23 Thu Jan 14 13:45:55 2016 : Info: Sending delayed reject for request 24 Thu Jan 14 13:45:55 2016 : Info: Sending duplicate proxied request to home server 194.83.56.233 por t 1812 - ID: 120 Thu Jan 14 13:45:55 2016 : Error: Empty packet: Socket is not ready. Thu Jan 14 13:45:55 2016 : Info: Sending duplicate proxied request to home server 194.83.56.233 por t 1812 - ID: 7 Thu Jan 14 13:45:55 2016 : Error: Empty packet: Socket is not ready. Thu Jan 14 13:45:55 2016 : Info: Sending delayed reject for request 39 Thu Jan 14 13:45:55 2016 : Info: Sending delayed reject for request 41 Sending Access-Accept of id 237 to 194.82.174.185 port 29981 or Thu Jan 14 13:45:51 2016 : Info: } # server eduroam Thu Jan 14 13:45:51 2016 : Info: Finished request 9. Thu Jan 14 13:45:51 2016 : Debug: Going to the next request Thu Jan 14 13:45:51 2016 : Debug: Thread 1 waiting to be assigned a request Thu Jan 14 13:45:51 2016 : Error: Empty packet: Socket is not ready. Thu Jan 14 13:45:51 2016 : Debug: Waking up in 0.3 seconds. Thu Jan 14 13:45:51 2016 : Debug: Waking up in 0.2 seconds. or Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair Proxy-State = 0x3236 Thu Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair State = 0x382b03450000013700010200c253f60c00 000000000000000000000000000004162412a2 Thu Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair NAS-Identifier = aruba0 Thu Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair NAS-IP-Address = 144.32.64.18 Thu Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair User-Name = hyuai@hyms.ac.uk Thu Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair Message-Authenticator = 0x82e775ee97b58da33b 734bd29026433c Thu Jan 14 13:45:50 2016 : Debug: rlm_perl: Added pair Calling-Station-Id = CC-3A-61-B4-4C-5D Thu Jan 14 13:45:50 2016 : Info: ++[perl] = ok Thu Jan 14 13:45:50 2016 : Info: +} # group pre-proxy = updated Thu Jan 14 13:45:50 2016 : Info: Proxying request 4 to home server 194.83.56.233 port 1812 Thu Jan 14 13:45:50 2016 : Debug: Going to the next request Thu Jan 14 13:45:50 2016 : Debug: Thread 1 waiting to be assigned a request Thu Jan 14 13:45:50 2016 : Error: Empty packet: Socket is not ready. Thu Jan 14 13:45:50 2016 : Debug: Waking up in 0.2 seconds. Any other info reuired to find out why this is now happening? Rgds Alex
On Jan 14, 2016, at 8:50 AM, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
After the upgrade I rebuilt FR 2.2.9 and while tings are working, I'm now getting a lot of Error: Empty packey: socket not ready messages
That message shows up when the server gets told by the OS that there's data ready, but there ends up being no data when it tries to get the next packet. Unfortunately, it's an OS issue. There is very little that FreeRADIUS can do to work around the fact that the OS is lying to it. Try looking for OS upgrades. Maybe there's a kernel bug. Alan DeKok.
hmmm o.k. we've just upgraded 3 machines from ubuntu 12.04 -> 14.04 and I've got 1 machine saying this. i'll check for updates A On 14 January 2016 at 14:40, Alan DeKok <aland@deployingradius.com> wrote:
On Jan 14, 2016, at 8:50 AM, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
After the upgrade I rebuilt FR 2.2.9 and while tings are working, I'm now getting a lot of Error: Empty packey: socket not ready messages
That message shows up when the server gets told by the OS that there's data ready, but there ends up being no data when it tries to get the next packet.
Unfortunately, it's an OS issue. There is very little that FreeRADIUS can do to work around the fact that the OS is lying to it.
Try looking for OS upgrades. Maybe there's a kernel bug.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi, On Thu, Jan 14, 2016 at 9:40 AM, Alan DeKok <aland@deployingradius.com> wrote:
On Jan 14, 2016, at 8:50 AM, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
After the upgrade I rebuilt FR 2.2.9 and while tings are working, I'm now getting a lot of Error: Empty packey: socket not ready messages
That message shows up when the server gets told by the OS that there's data ready, but there ends up being no data when it tries to get the next packet.
Unfortunately, it's an OS issue. There is very little that FreeRADIUS can do to work around the fact that the OS is lying to it.
Try looking for OS upgrades. Maybe there's a kernel bug.
A long time ago, I had a similar problem with a completely different protocol / project. While 'select' was informing that the UDP socket is ready for reading, a subsequent call to 'recvfrom' was returning EAGAIN. It turned out that on Linux you may get EAGAIN if the checksum of the UDP packet was wrong. So if it sounds similar, you might want to take a look on the traffic with wireshark (it gives a warning). HTH
And you're right Isaac, We seem to be seeing fragmented UDP packets from roaming{0,1,2}.ja.net which match up with EAGAIN coming from the recevfrom syscall. Rgds Alex On 14 January 2016 at 15:30, Isaac Boukris <iboukris@gmail.com> wrote:
Hi,
On Thu, Jan 14, 2016 at 9:40 AM, Alan DeKok <aland@deployingradius.com> wrote:
On Jan 14, 2016, at 8:50 AM, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
After the upgrade I rebuilt FR 2.2.9 and while tings are working, I'm now getting a lot of Error: Empty packey: socket not ready messages
That message shows up when the server gets told by the OS that there's data ready, but there ends up being no data when it tries to get the next packet.
Unfortunately, it's an OS issue. There is very little that FreeRADIUS can do to work around the fact that the OS is lying to it.
Try looking for OS upgrades. Maybe there's a kernel bug.
A long time ago, I had a similar problem with a completely different protocol / project. While 'select' was informing that the UDP socket is ready for reading, a subsequent call to 'recvfrom' was returning EAGAIN. It turned out that on Linux you may get EAGAIN if the checksum of the UDP packet was wrong.
So if it sounds similar, you might want to take a look on the traffic with wireshark (it gives a warning).
HTH - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi,
We seem to be seeing fragmented UDP packets from roaming{0,1,2}.ja.net which match up with EAGAIN coming from the recevfrom syscall.
yep - EAP packets can be big - and therefore expect fragmentation - you need to be able to deal with such packets - at all parts of the transit - not just RADIUS server but also firewall (several have drop fragmented UDP by default) alan
hmm only stated seeing this wen we moved to Ubuntu 14.04, were running 12.04 before and not a squeak! On 15 January 2016 at 15:59, <A.L.M.Buxey@lboro.ac.uk> wrote:
Hi,
We seem to be seeing fragmented UDP packets from roaming{0,1,2}.ja.net which match up with EAGAIN coming from the recevfrom syscall.
yep - EAP packets can be big - and therefore expect fragmentation - you need to be able to deal with such packets - at all parts of the transit - not just RADIUS server but also firewall (several have drop fragmented UDP by default)
alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Sent from my iPhone 6 plus
On 15 Jan 2016, at 17:09, A.L.M.Buxey@lboro.ac.uk wrote:
Hi,
hmm only stated seeing this wen we moved to Ubuntu 14.04, were running 12.04 before and not a squeak!
well, updating your server wont affect what is sent to you from remote systems ;-) (but maybe replies based on things YOU send get affected?)
Was wondering if that might be the case Something for Monday
alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Hi, Wondering if we should revert back to 12.04 version of eduroam2 for the time being until I get to the bottom of these errors A On 15 January 2016 at 16:09, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
hmm only stated seeing this wen we moved to Ubuntu 14.04, were running 12.04 before and not a squeak!
On 15 January 2016 at 15:59, <A.L.M.Buxey@lboro.ac.uk> wrote:
Hi,
We seem to be seeing fragmented UDP packets from roaming{0,1,2}.ja.net which match up with EAGAIN coming from the recevfrom syscall.
yep - EAP packets can be big - and therefore expect fragmentation - you need to be able to deal with such packets - at all parts of the transit - not just RADIUS server but also firewall (several have drop fragmented UDP by default)
alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
If you suspect it's a kernel issue, I suggest try other kernel versions first instead of downgrading your OS. For example, linux-generic-lts-wily would give you kernel 4.2 on ubuntu 14.04. -- Fajar On Mon, Jan 18, 2016 at 3:45 PM, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
Hi, Wondering if we should revert back to 12.04 version of eduroam2 for the time being until I get to the bottom of these errors A
On 15 January 2016 at 16:09, Alex Sharaz <alex.sharaz@york.ac.uk> wrote:
hmm only stated seeing this wen we moved to Ubuntu 14.04, were running 12.04 before and not a squeak!
On 15 January 2016 at 15:59, <A.L.M.Buxey@lboro.ac.uk> wrote:
Hi,
We seem to be seeing fragmented UDP packets from roaming{0,1,2}. ja.net which match up with EAGAIN coming from the recevfrom syscall.
yep - EAP packets can be big - and therefore expect fragmentation - you need to be able to deal with such packets - at all parts of the transit
not just RADIUS server but also firewall (several have drop fragmented UDP by default)
alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
participants (5)
-
A.L.M.Buxey＠lboro.ac.uk -
Alan DeKok -
Alex Sharaz -
Fajar A. Nugraha -
Isaac Boukris