Server stopped responding, throwing multiple SSL-related errors
I'm running FreeRADIUS 1.1.2 on Ubuntu. This morning one of the two servers stopped answering requests. The radius log contained thousands of lines like these: Mon Oct 23 12:32:56 2006 : Error: TLS Alert write:fatal:illegal parameter Mon Oct 23 12:32:56 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:32:56 2006 : Error: rlm_eap: SSL error error:1408E098:SSL routines:SSL3_GET_MESSAGE:excessive message size Mon Oct 23 12:32:56 2006 : Error: rlm_eap_tls: SSL_read failed in a system call (-1), TLS session fails. Mon Oct 23 12:33:02 2006 : Error: TLS_accept:error in SSLv3 read client certificate A Mon Oct 23 12:33:02 2006 : Error: rlm_eap: SSL error error:00000000:lib(0):func(0):reason(0) Mon Oct 23 12:33:02 2006 : Error: TLS Alert write:fatal:bad record mac Mon Oct 23 12:33:02 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:33:02 2006 : Error: rlm_eap: SSL error error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac Mon Oct 23 12:33:02 2006 : Error: rlm_eap_tls: SSL_read failed in a system call (-1), TLS session fails. Mon Oct 23 12:33:02 2006 : Error: TLS Alert write:fatal:illegal parameter Mon Oct 23 12:33:02 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:33:02 2006 : Error: rlm_eap: SSL error error:1408E098:SSL routines:SSL3_GET_MESSAGE:excessive message size Mon Oct 23 12:33:02 2006 : Error: rlm_eap_tls: SSL_read failed in a system call (-1), TLS session fails. Mon Oct 23 12:33:16 2006 : Error: TLS_accept:error in SSLv3 read client certificate A Restarting radiusd fixed it. My build of FreeRADIUS was built from source, the SSL library is the Ubuntu system openssl (v. 0.9.7). Any idea what might have gone wrong? -Ben
Sounds very similar to my thread titled SSL_read failed in a system call from last week. -----Original Message----- From: freeradius-users-bounces+mking=bridgew.edu@lists.freeradius.org [mailto:freeradius-users-bounces+mking=bridgew.edu@lists.freeradius.org] On Behalf Of Ben Beuchler Sent: Monday, October 23, 2006 2:06 PM To: FreeRadius users mailing list Subject: Server stopped responding, throwing multiple SSL-related errors I'm running FreeRADIUS 1.1.2 on Ubuntu. This morning one of the two servers stopped answering requests. The radius log contained thousands of lines like these: Mon Oct 23 12:32:56 2006 : Error: TLS Alert write:fatal:illegal parameter Mon Oct 23 12:32:56 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:32:56 2006 : Error: rlm_eap: SSL error error:1408E098:SSL routines:SSL3_GET_MESSAGE:excessive message size Mon Oct 23 12:32:56 2006 : Error: rlm_eap_tls: SSL_read failed in a system call (-1), TLS session fails. Mon Oct 23 12:33:02 2006 : Error: TLS_accept:error in SSLv3 read client certificate A Mon Oct 23 12:33:02 2006 : Error: rlm_eap: SSL error error:00000000:lib(0):func(0):reason(0) Mon Oct 23 12:33:02 2006 : Error: TLS Alert write:fatal:bad record mac Mon Oct 23 12:33:02 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:33:02 2006 : Error: rlm_eap: SSL error error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac Mon Oct 23 12:33:02 2006 : Error: rlm_eap_tls: SSL_read failed in a system call (-1), TLS session fails. Mon Oct 23 12:33:02 2006 : Error: TLS Alert write:fatal:illegal parameter Mon Oct 23 12:33:02 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:33:02 2006 : Error: rlm_eap: SSL error error:1408E098:SSL routines:SSL3_GET_MESSAGE:excessive message size Mon Oct 23 12:33:02 2006 : Error: rlm_eap_tls: SSL_read failed in a system call (-1), TLS session fails. Mon Oct 23 12:33:16 2006 : Error: TLS_accept:error in SSLv3 read client certificate A Restarting radiusd fixed it. My build of FreeRADIUS was built from source, the SSL library is the Ubuntu system openssl (v. 0.9.7). Any idea what might have gone wrong? -Ben - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On 10/23/06, King, Michael <MKing@bridgew.edu> wrote:
Sounds very similar to my thread titled SSL_read failed in a system call from last week.
Yeah... The reason I opened a new thread rather then just appending to yours was the new error that I didn't see in your log snippets: SSL3_GET_MESSAGE:excessive message size -Ben
"Ben Beuchler" <insyte@gmail.com> wrote:
I'm running FreeRADIUS 1.1.2 on Ubuntu. This morning one of the two servers stopped answering requests. The radius log contained thousands of lines like these:
Mon Oct 23 12:32:56 2006 : Error: TLS Alert write:fatal:illegal parameter Mon Oct 23 12:32:56 2006 : Error: TLS_accept:error in SSLv3 read certificate verify A Mon Oct 23 12:32:56 2006 : Error: rlm_eap: SSL error error:1408E098:SSL routines:SSL3_GET_MESSAGE:excessive message size
Hmm... looking at google: http://stunnel.mirt.net/pipermail/stunnel-users/2004-September/000037.html See also https://mail.internet2.edu/wws/arc/shibboleth-users/2004-01/msg00124.html Unfortunately, OpenSSL doesn't really give useful error messages.
Restarting radiusd fixed it.
That says to me there's a memory corruption issue.. maybe back-porting the thread locks from CVS head to 1.1.x would help. Alan DeKok. -- http://deployingradius.com - The web site of the book http://deployingradius.com/blog/ - The blog
participants (3)
-
Alan DeKok -
Ben Beuchler -
King, Michael