OpenSSL Security issues
This is a bad one: http://heartbleed.com/ ... The Heartbleed bug allows anyone on the Internet to read the memory of the systems protected by the vulnerable versions of the OpenSSL software. ... * OpenSSL 1.0.1 through 1.0.1f (inclusive) are vulnerable * OpenSSL 1.0.1g is NOT vulnerable * OpenSSL 0.9.8 branch is NOT vulnerable Everyone using TLS methods with EAP are likely vulnerable. Anyone using RadSec is likely vulnerable. Please check which version of OpenSSL you are using. Alan DeKok.
On 7 Apr 2014, at 22:18, Alan DeKok <aland@deployingradius.com> wrote:
This is a bad one:
... The Heartbleed bug allows anyone on the Internet to read the memory of the systems protected by the vulnerable versions of the OpenSSL software. ...
* OpenSSL 1.0.1 through 1.0.1f (inclusive) are vulnerable * OpenSSL 1.0.1g is NOT vulnerable * OpenSSL 0.9.8 branch is NOT vulnerable
Everyone using TLS methods with EAP are likely vulnerable. Anyone using RadSec is likely vulnerable.
Please check which version of OpenSSL you are using.
That's really bad. Think we should add a configure time check to prevent the server being built against vulnerable versions? Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Arran Cudbard-Bell wrote:
That's really bad. Think we should add a configure time check to prevent the server being built against vulnerable versions?
https://www.openssl.org/news/secadv_20140407.txt ... Users unable to immediately upgrade can alternatively recompile OpenSSL with -DOPENSSL_NO_HEARTBEATS. Wow. The potential side-effects of this problem are enormous. ANY site using TLS for ANYTHING can have ANY memory read by an attacker. i.e. secrets, private keys, etc. Alan DeKok.
On 7 Apr 2014, at 23:00, Alan DeKok <aland@deployingradius.com> wrote:
Arran Cudbard-Bell wrote:
That's really bad. Think we should add a configure time check to prevent the server being built against vulnerable versions?
https://www.openssl.org/news/secadv_20140407.txt
... Users unable to immediately upgrade can alternatively recompile OpenSSL with -DOPENSSL_NO_HEARTBEATS.
Wow. The potential side-effects of this problem are enormous. ANY site using TLS for ANYTHING can have ANY memory read by an attacker.
i.e. secrets, private keys, etc.
Uhuh. That'd be a compile and link time check for FreeRADIUS then. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Yowza!! I'm back in the office tomorrow and will check the CentOS updates. I'll then recompile any of our binary packages immediately. I'll forward this on. Stefan ________________________________________ From: Arran Cudbard-Bell [a.cudbardb@freeradius.org] Sent: Monday, April 07, 2014 11:12 PM To: FreeRadius users mailing list Subject: Re: OpenSSL Security issues On 7 Apr 2014, at 23:00, Alan DeKok <aland@deployingradius.com> wrote:
Arran Cudbard-Bell wrote:
That's really bad. Think we should add a configure time check to prevent the server being built against vulnerable versions?
https://www.openssl.org/news/secadv_20140407.txt
... Users unable to immediately upgrade can alternatively recompile OpenSSL with -DOPENSSL_NO_HEARTBEATS.
Wow. The potential side-effects of this problem are enormous. ANY site using TLS for ANYTHING can have ANY memory read by an attacker.
i.e. secrets, private keys, etc.
Uhuh. That'd be a compile and link time check for FreeRADIUS then. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2 -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
On 08/04/14 01:11, stefan.paetow@diamond.ac.uk wrote:
I'm back in the office tomorrow and will check the CentOS updates
Seems that CentOS 5 is not affected, but CentOS 6 is. An patched update has been released for for RHEL 6 and will presumably make its way into CentOS before too long... https://rhn.redhat.com/errata/RHSA-2014-0376.html Have fun! Jonathan
Hi,
Seems that CentOS 5 is not affected, but CentOS 6 is. An patched update has been released for for RHEL 6 and will presumably make its way into CentOS before too long...
patches are now out for CentOS alan
On 8 Apr 2014, at 10:35, Jonathan Gazeley <Jonathan.Gazeley@bristol.ac.uk> wrote:
On 08/04/14 01:11, stefan.paetow@diamond.ac.uk wrote:
I'm back in the office tomorrow and will check the CentOS updates
Seems that CentOS 5 is not affected, but CentOS 6 is. An patched update has been released for for RHEL 6 and will presumably make its way into CentOS before too long...
Question to representatives of various distributions on the lists. As instead of fix the issues correctly by upgrading to 1.0.1g, you are patching existing versions of libssl, how can we determine whether a version of libssl is vulnerable or not at configure time? As it stands the next versions on all branches will refuse to build against libssl 1.0.1-1.0.1f because of the potential security risk. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
On Tue, Apr 8, 2014 at 5:02 PM, Arran Cudbard-Bell < a.cudbardb@freeradius.org> wrote:
On 8 Apr 2014, at 10:35, Jonathan Gazeley <Jonathan.Gazeley@bristol.ac.uk> wrote:
On 08/04/14 01:11, stefan.paetow@diamond.ac.uk wrote:
I'm back in the office tomorrow and will check the CentOS updates
Seems that CentOS 5 is not affected, but CentOS 6 is. An patched update has been released for for RHEL 6 and will presumably make its way into CentOS before too long...
Question to representatives of various distributions on the lists.
As instead of fix the issues correctly by upgrading to 1.0.1g, you are patching existing versions of libssl, how can we determine whether a version of libssl is vulnerable or not at configure time?
As it stands the next versions on all branches will refuse to build against libssl 1.0.1-1.0.1f because of the potential security risk.
In the past with clamav (the opensource antivirus), when faced with possible security risk in certain zlib versions, they refused to build by default when detecting unsafe versions, but added --disable-zlib-vcheck in configure script to allow manual override. IMHO it's a good workaround. -- Fajar
It's not really up to the FreeRADIUS people to make that call, but rather the OpenSSL guys. But I agree that this is rather annoying. I would have preferred a 1.0.1g upgrade myself. I'll forward this question on to the RHEL OpenSSL guy and find out. S.
-----Original Message----- From: Arran Cudbard-Bell [mailto:a.cudbardb@freeradius.org] Sent: 08 April 2014 11:03 To: FreeRadius users mailing list Subject: Re: OpenSSL Security issues
On 8 Apr 2014, at 10:35, Jonathan Gazeley <Jonathan.Gazeley@bristol.ac.uk> wrote:
On 08/04/14 01:11, stefan.paetow@diamond.ac.uk wrote:
I'm back in the office tomorrow and will check the CentOS updates
Seems that CentOS 5 is not affected, but CentOS 6 is. An patched update has been released for for RHEL 6 and will presumably make its way into CentOS before too long...
Question to representatives of various distributions on the lists.
As instead of fix the issues correctly by upgrading to 1.0.1g, you are patching existing versions of libssl, how can we determine whether a version of libssl is vulnerable or not at configure time?
As it stands the next versions on all branches will refuse to build against libssl 1.0.1-1.0.1f because of the potential security risk.
Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team
FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
Do what ISC do with BIND? it will fail certain version checks. ... and if you know what you are doing you do --disable-openssl-version-check Otherwise you'll be relying on some tool to check/validate the openssl running ... and that'd also only work if there was some listener. .. which I guess could be a companion of the checking tool... but still. .. alan
On 08/04/14 11:02, Arran Cudbard-Bell wrote:
As it stands the next versions on all branches will refuse to build against libssl 1.0.1-1.0.1f because of the potential security risk.
Please don't do that, for the exact reasons you outlined. Hardcoding a version number blacklist into the build environment just means everyone building against an enterprise distro will have to patch your changes out. I realise it's a serious vulnerability, but "configure.in" of a project using the library is not the right place to address this. You'd be better off adding a runtime check and refusing to start without "allow_unsafe_openssl" global set or similar, if you must. At least that way people can configure the server to start once they've patched.
Phil Mayers wrote:
Hardcoding a version number blacklist into the build environment just means everyone building against an enterprise distro will have to patch your changes out.
I understand.
You'd be better off adding a runtime check and refusing to start without "allow_unsafe_openssl" global set or similar, if you must. At least that way people can configure the server to start once they've patched.
Yes. And unfortunately there's no run-time check to say that OpenSSL has been patched to address the vulnerability. <sigh> Alan DeKok.
On 8 Apr 2014, at 14:28, Phil Mayers <p.mayers@IMPERIAL.AC.UK> wrote:
On 08/04/14 13:41, Alan DeKok wrote:
Yes. And unfortunately there's no run-time check to say that OpenSSL has been patched to address the vulnerability. <sigh>
And AFAICT no run-time way to disable the extension. Grumble.
Yep. That was pretty much the first thing I checked this morning. I agree, adding runtime checks is the better option. I actually added a runtime check first, and was going to add a configure time check, but after discussions with Alan offlist we decided it was better to just leave it. As per your suggestion there's now a security.allow_vulnerable_openssl configuration item which enables or disables the security check. # # allow_vulnerable_openssl: Allow the server to start with # versions of OpenSSL known to have critical vulnerabilities. # # This check is based on the version number reported by libssl # and may not reflect patches applied to libssl by # distribution maintainers. # allow_vulnerable_openssl = no If you have a potentially vulnerable version of OpenSSL the server will print: Refusing to start with libssl version %s (in range 1.0.1 - 1.0.1f). Security advisory CVE-2014-0160 (Heartbleed) For more information see http://heartbleed.com end exit. radiusd -xv will now print out the decoded version number to give more information about the local openssl patch level. Though as it's only a 4 bit integer, (and currently 15 on Ubuntu 12.04) it's probably not a whole lot of use. Alan is backporting these features to v2.x.x Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Arran Cudbard-Bell wrote:
As per your suggestion there's now a security.allow_vulnerable_openssl configuration item which enables or disables the security check.
The checks are in v2, v3, and git "master". From talking with Jouni Malinen (wpa_supplicant guy), he says: "I tested with couple RADIUS authentication servers and could not trigger the issue due to reasons that I confirmed to be because of incorrect OpenSSL API use.." I'm not entirely sure what that means, but I'm trying to find out. Alan DeKok.
Surely you'll now have to do this for all the past issues like the recent session renegotiation openssl issue too? alan
Alan Buxey wrote:
Surely you'll now have to do this for all the past issues like the recent session renegotiation openssl issue too?
No, because the cache is disabled by default. Sites using a vulnerable version of OpenSSL can just disable the cache, and not worry about it. Alan DeKok.
But sites (well, admins) who are unaware may enable the cache. .. It which case there should be an interlock which means they must also turn off the openssl version check safety trigger too? As the heartbleed issue isn't as shocking as feared for freeradius is there any need for the current check to be so hard on you if you've got 1.0.1 < g installed now? (However, I'm guessing yes if you run other SSL/TLS services on the same box eg Web server since attacker can scan your memory slowly, so just protecting them from themselves) alan
Alan Buxey wrote:
But sites (well, admins) who are unaware may enable the cache. .. It which case there should be an interlock which means they must also turn off the openssl version check safety trigger too?
The issue is in the TLS protocol, not in a particular version of OpenSSL.
As the heartbleed issue isn't as shocking as feared for freeradius is there any need for the current check to be so hard on you if you've got 1.0.1 < g installed now? (However, I'm guessing yes if you run other SSL/TLS services on the same box eg Web server since attacker can scan your memory slowly, so just protecting them from themselves)
I've been talking with Jouni Malinen offline. After mutating his attack, he can read ~1K of data from the stack of FreeRADIUS. So it's worse than we thought, but not as bad as the attacks on HTTPS servers. So... the OpenSSL version checks will remain. Alan DeKok.
On 8 Apr 2014, at 20:42, Alan Buxey <A.L.M.Buxey@lboro.ac.uk> wrote:
But sites (well, admins) who are unaware may enable the cache. .. It which case there should be an interlock which means they must also turn off the openssl version check safety trigger too?
As the heartbleed issue isn't as shocking as feared for freeradius is there any need for the current check to be so hard on you if you've got 1.0.1 < g installed now? (However, I'm guessing yes if you run other SSL/TLS services on the same box eg Web server since attacker can scan your memory slowly, so just protecting them from themselves)
Memory protection should kick in and trigger a SEGV if they tried to read memory alloced to another process. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
On 8 Apr 2014, at 21:32, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
On 8 Apr 2014, at 20:42, Alan Buxey <A.L.M.Buxey@lboro.ac.uk> wrote:
But sites (well, admins) who are unaware may enable the cache. .. It which case there should be an interlock which means they must also turn off the openssl version check safety trigger too?
As the heartbleed issue isn't as shocking as feared for freeradius is there any need for the current check to be so hard on you if you've got 1.0.1 < g installed now? (However, I'm guessing yes if you run other SSL/TLS services on the same box eg Web server since attacker can scan your memory slowly, so just protecting them from themselves)
Memory protection should kick in and trigger a SEGV if they tried to read memory alloced to another process.
Anyway, tweaked the security.allow_vulnerable_openssl behaviour. In tls.c there's now the start of what i'm sure will become large and depressing array: /* Record critical defects in libssl here (newest first)*/ static libssl_defect_t libssl_defects[] = { { .low = 0x010001000, /* 1.0.1 */ .high = 0x01000106f, /* 1.0.1f */ .id = "CVE-2014-0160", .name = "Heartbleed", .comment = "For more information see http://heartbleed.com" } }; If security.allow_vulnerable_openssl == libssl_defects[0].id then the check is disabled. If security.allow_vulnerable_openssl == 'yes' then the check is disabled. Server will output something like: Refusing to start with libssl version OpenSSL 1.0.1 14 Mar 2012 0x01000100f (1.0.1 15) (in range 1.0.1-0 - 1.0.1f-15) Security advisory Heartbleed (CVE-2014-0160) For more information see http://heartbleed.com For each applicable vulnerability, then: Once you have verified libssl has been correctly patched, set security.allow_vulnerable_openssl = 'CVE-2014-0160' This way it's not a one time thing the admin disables and forgets about. Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Please don't do that, for the exact reasons you outlined.
Hardcoding a version number blacklist into the build environment just means everyone building against an enterprise distro will have to patch your changes out.
I realise it's a serious vulnerability, but "configure.in" of a project using the library is not the right place to address this.
You'd be better off adding a runtime check and refusing to start without "allow_unsafe_openssl" global set or similar, if you must. At least that way people can configure the server to start once they've patched.
Response from Fedora project was: "I took the approach of least resistance, which was to patch the bug. The OpenSSL maintainers have whatever reason they have to keeping OpenSSL at 1.0.1e and it wasn't my place to change that. It also happens to be the approach that RHEL took." There we have it. Path of least bleating and most expediency. :-) Stefan -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
On 8 Apr 2014, at 16:52, <stefan.paetow@diamond.ac.uk> <stefan.paetow@diamond.ac.uk> wrote:
Please don't do that, for the exact reasons you outlined.
Hardcoding a version number blacklist into the build environment just means everyone building against an enterprise distro will have to patch your changes out.
I realise it's a serious vulnerability, but "configure.in" of a project using the library is not the right place to address this.
You'd be better off adding a runtime check and refusing to start without "allow_unsafe_openssl" global set or similar, if you must. At least that way people can configure the server to start once they've patched.
Response from Fedora project was:
"I took the approach of least resistance, which was to patch the bug. The OpenSSL maintainers have whatever reason they have to keeping OpenSSL at 1.0.1e and it wasn't my place to change that. It also happens to be the approach that RHEL took."
There we have it. Path of least bleating and most expediency.
*sigh*. Well thanks for checking anyway. To be fair OpenSSL don't seem to be taking security seriously, this should have been caught by static analysis... except that code only gets submitted to Coverity sporadically, and they don't like using it because of the high rate of false positives. You know what causes high rates of false positives? Weird fucked up code... Arran Cudbard-Bell <a.cudbardb@freeradius.org> FreeRADIUS Development Team FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2
Arran Cudbard-Bell wrote:
To be fair OpenSSL don't seem to be taking security seriously, this should have been caught by static analysis... except that code only gets submitted to Coverity sporadically, and they don't like using it because of the high rate of false positives.
Valgrind is almost useless with FreeRADIUS && OpenSSL, because of the massive amounts of complaints about OpenSSL issues.
You know what causes high rates of false positives? Weird fucked up code...
Yes. Valgrind isn't perfect, but I'd be surprised if all of it's complaints about OpenSSL are wrong. Alan DeKok.
*sigh*. Well thanks for checking anyway.
To be fair OpenSSL don't seem to be taking security seriously, this should have been caught by static analysis… except that code only gets submitted to Coverity sporadically, and they don't like using it because of the high rate of false positives.
Well, to be fair, the bloke who responded is a release engineer for Fedora, not a package maintainer… I'd guess the OpenSSL package maintainers have to make the call. Despite that, I hear your pain. I had the… *ahem* misfortune of working with OpenSSL on Windows. *sigh* Stefan -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
The patched version of OpenSSL 1.0.1e (which is current on CentOS 6.5) was updated here this morning. The dev libs have also been updated. Stefan
-----Original Message----- From: Jonathan Gazeley [mailto:Jonathan.Gazeley@bristol.ac.uk] Sent: 08 April 2014 10:35 To: freeradius-users@lists.freeradius.org Subject: Re: OpenSSL Security issues
On 08/04/14 01:11, stefan.paetow@diamond.ac.uk wrote:
I'm back in the office tomorrow and will check the CentOS updates
Seems that CentOS 5 is not affected, but CentOS 6 is. An patched update has been released for for RHEL 6 and will presumably make its way into CentOS before too long...
https://rhn.redhat.com/errata/RHSA-2014-0376.html
Have fun! Jonathan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
There appear to have been 2 patches. The first one did the basic disable heartbeat method. ... so tools available at the time then declared you 'safe'. The second patch appears to have brought back the heartbeat function (well. . it was there for a reason! ;) ) but fixed the actual memory access bug. This second patch is the 'correct' way to go but means basic testing tools now fail and you need to use exploit-like code to see if you can get big replies All fun. Alan -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
participants (9)
-
A.L.M.Buxey＠lboro.ac.uk -
Alan Buxey
-
Alan DeKok -
Alan DeKok -
Arran Cudbard-Bell -
Fajar A. Nugraha -
Jonathan Gazeley -
Phil Mayers -
stefan.paetow＠diamond.ac.uk