EAP-TLS Signature Check Failure

[Peter Bance]

2 months later, a quick update on this topic (apologies, I may have 
broken the threading as I don't have a copy of the original e-mails any 
more!), just so there's an online reference here for anyone encountering 
the same error.

Turns out I was wrong about it being Windows and OpenSSL getting muddled 
over TLS 1.2 vs. 1.3...

To recap: Windows 10 clients, corporate WiFi network using EAP-TLS to 
RADIUS with machine certificate (SCEP) authentication only.

Error in Freeradius logs:

(6) eap_tls: ERROR: TLS Alert write:fatal:decrypt error
tls: TLS_accept: Error in error
(6) eap_tls: ERROR: Failed in __FUNCTION__ (SSL_read)
(6) eap_tls: ERROR: error:0407E086:rsa 
routines:RSA_verify_PKCS1_PSS_mgf1:last octet invalid
(6) eap_tls: ERROR: error:1417B07B:SSL 
routines:tls_process_cert_verify:bad signature
(6) eap_tls: ERROR: System call (I/O) error (-1)
(6) eap_tls: ERROR: TLS receive handshake failed during operation
(6) eap_tls: ERROR: [eaptls process] = fail

After encountering the error on a few machines again, even with 
"tls_max_version" set, I delved into the guts of Windows again, and 
found the error (which isn't helpful or descriptive) was actually caused 
because:

* Some machine certificates had were stored/managed by the "Microsoft 
Platform Crypto Provider" (TPM-backed)
* Built-in security measures control access to keys in that CSP, and it 
cannot be used non-interactively
* Windows WiFi profile was set to auto-join (non-interactive)
* Windows couldn't get the key, so $deity knows what signature it was 
sending
* OpenSSL rightly got confused

The solution:

Ensure machine certificate keys are stored in the "Microsoft Software 
Key Storage Provider" (or another CSP/KSP that permits non-interactive 
use).

All is well again, and I'm close to shutting down NPS :-)

-- 
Peter Bance
Information Security Adviser

Alan DeKok wrote:
> A final update on this, in case anyone here's interested (or to "wrap 
> up" for anyone stumbling across this thread online) - I fixed it, and 
> Windows clients are now happily joining WiFi. It's a beautiful thing to 
> behold :-)
> 
> In the end, I had to force OpenSSL on FreeRADIUS to stop offering 
> TLS1.3 ciphers using the mods/eap config:
> 
> tls_max_version = "1.2"

   Good to hear.

> It seems there may be a bug in OpenSSL 1.1.1 such that even though the 
> negotiation resulted in a TLS 1.2 session, the weird back-port of TLS 
> 1.3 ciphers into TLS 1.2 confused things (a lot), and it tried checking 
> for TLS 1.3 style signatures inappropriately.

   Weird, but OK.  It's OpenSSL :(

   Alan DeKok.