Logging EAP-TLS certificate details

Mon Mar 22 22:04:52 CET 2021

Let me clarify - we issue certificate to actual people and applications. Every certificate is accounted for, and the Common Name will either have the persons's name or the serial/MAC address of the IoT it was issued to. That is not the issue. We *know* who the certificates are issued to, and if we have the Cert Serial number or the CN we can immediately identify them.

For legitimate applications and users, this will work fine, as a "real" user will enter their username in the EAP-TLS request, or the operating system will provide the MAC address. Those are non-issues. We know everything there is to know.

The problem is with an attacker, or in my original post with the incompetent vendor who is specifying "User" as the EAP-TLS username for each one of their hundreds of devices.

If I (the attacker) can get my hands on a certificate with the private key, when logging in with EAP-TLS I can provide whatever username I want. Freeradius will only be logging/auditing that fake username, and whatever fame MAC address I assign to the virtual machine I'm using to perform the attack. I will be completely anonymous because freeradius is not logging the Cert Serial or the Cert Common Name. If freeradius logged the Serial or the CN we would not be having this conversation as we'd have all the info necessary to track down the compromised certificate and the person/device it was originally issued to.

>  FreeRADIUS just logs whatever the NAS sends it.  The NAS can (and will) lie about what it's doing.  Saying this is a security issue in FreeRADIUS is to severely misunderstand the problem.
No that's incorrect in EAP-TLS - the NAS will send the *correct* information about the certificate. That cannot be faked. freeradius is configured to accept only certs issued by our Certification Authority.

>  This User-Name is taken from the Access-Request packet.  Which in turn is derived from the User-Name in the EAP packet.  Which is under the control of the malicious user.

Exactly - the User-Name is useless when it comes to EAP-TLS. That's not what should be audited. It's the certificate information (which IS available on the access-request) that needs to be logged.

>  So... why did you let the user in?
> 
>  A good chunk of this problem is that you're not validating the user names.  You've previously said:
How do you expect us to validate the username? We can't force all applications to follow one single standard. Windows and Mac sometimes connect by putting the MAC address. Some NIC cards take the current logged-in-user's username. In our case we have a vendor who has hardware with a hard-coded "User" as a username. Other ioT have their serial number in the CN. You cannot enforce a standard for a "Username" EAP-TLS field that is free-form with no RFC that requires it to be something specific (I've been looking for one to try telling that one vendor to fix their stuff and follow the standards... but there isn't one)

At the end, who cares about the EAP-TLS "User-name" free-form field when the actual identity used to login is the certificate, but freeradius is not logging anything about it? This is a 100% a freeradius issue, not a RADIUS protocol issue.

>  This is a problem.  Most people don't do this.  Most people put a real name into the certificate.  For devices, they put the device MAC address.  So this comes down to:
> 
>  "We provisioned entirely similar user credentials to 40K devices, and now we don't know which device is which".

As I said, we're doing it correctly already. All certs either have the persons's name or the device's serial/MAC address in the CN. Each cert it unique.

>  And why is it not possible to look at the NAS IP address / port, etc. as I suggested before?  Why isn't that information available?  If it isn't, I would hope that you would reply "No, it isn't available" instead of ignoring my comments and coming up with yet another scenario for why this doesn't work.
It's useless to look at the NAS IP / port information, as that will do nothing as far as letting us know what certificate was used to login. Why...? Because the TLS Client Cert details are only received by freeradius, which is not logging it.

>  Sure.  So... configure your certificates to contain useful information, like names and/or MAC addresses.
> 
>  Use the information in the packets to cross-correlate Access-Request and Accounting-Request packets.
> 
>  This is all what people do.
We do too. Our certs DO have that info. But if freeradius doesn't log which certificate is used to login, how can we retrieve said information if freeradius doesn't tell us which certificate was used?

>  If only the configuration files were editable, and you could log the certificate serial number along with the fake User-Name.
> 
>  See?  It's easy.
...Actually in multiple posts you have said not to change the "User-Name" field. We did however change the config files to retrieve the info required, but we also had to change the schema of the database to log what freeradius wasn't logging.
We LOVE freeradius for the flexibility it provides. We canned the super-expensive Cisco Radius appliances we were using several years ago in favor of freeradius not because of the cost, but because of the immensely superior capabilities freeradius has.
All I'm trying to point out is that when it comes to EAP-TLS, it's imperative for freeradius to log the client certificate information. I'd wager there is a large number of admins using freeradius for EAP-TLS authentication that have no idea that the current implementation they are using will be useless to track down a compromised certificate should the need ever arise. We ourselves were oblivious to this until we had to deal with the vendor that was deploying their IoTs with the generic "User" username, and realized what a gaping security hole we had.

>  You've configured 40K certificates with "user" as the name.  Then, you're saying that FreeRADIUS is broken because it doesn't anticipate this situation, and log *extra* information that *you* need, and *only* you need for your particular deployment.
> 
>  Please don't blame us for a broken system.
No, once more. We did not configure 40K certs with "user" as the name. That is not the problem. The problem is that if you steal one of those 40K certs, you will be able to come onsite and login via WiFi to our network by specifying a fake username, and it will be *impossible* for me to find out which certificate you stole since freeradius is not logging that information. If freeradius logged the serial or the CN, I *would* immediately know which of the 40K certs was compromised as each one has a unique serial/CN.
Please do not deviate blame. Our system is fine. The issue is using a product (freeradius) that is not logging the identity (meaning the certificate) that is used to login when using EAP-TLS.

>  Normal configurations work.  If you use unique user names for each certificate, then the default configuration works.  If you put "user" into all of the certs, then well...

We don't. The attacker is the one that would specify "User" in the "User-Name".
I don't know how to make this clearer...

>  If the TLS-Client-Cert-Common name is always "user", then you can also check for bad user-names:
It's not. As I've repeatedly said in my post, the CN will either have the name of the person the original certificate was issued to, or the serial/MAC of the IoT device. 

> 	if (TLS-Client-Cert-Common-Name && (TLS-Client-Cert-Common-Name != User-Name)) {
> 		reject
> 	}
> 	...
> }
> 
>  2 lines of code will prevent users from lying about their User-Name.

Not doable. As I explained above you/we cannot require multiple vendors/applications to use a specific User-Name. We have different vendors specifying different data in different formats. There is no standard - there's no RFC for that. It's a simple label. Vendor A will use their serial. Vendor B their MAC address with ":" separators. Vendor C will use the MAC with "-". Vendor D will require the CN to be the person's name by specifying "CN=" in front of it. Vendor E will require the name without the "CN=" in front. And so not. This is not an issue with legitimate vendors and applications. That User-Name field is not to be used as an identifier in the EAP-TLS process period - it's a label. The authenticator is the certificate - that is what matters and needs to be tracked/logged.

>  Or, in 3.0.18 and later we have a new policy "sql_session_start".  This policy is run in the post-auth section, and creates an entry in the radacct table:
We're using a very complex config in our v2 to handle a dozen different types of authentications/applications/configurations. I got a headache when I first looked at the migration guide. I'm hoping to retire before I'm forced to upgrade.

> And there's no need to complain that the default configuration is "insecure", because you have 40K users, all with the same name.
We don't. All 40K certs are unique and immediately identifiable. We're worried about the single certificate that can be stolen, with the hacker being able to specify a useless random user-name, and with freeradius not logging the serial/issuer of the certificate, without which we cannot identify the compromised certificate.

>  So... don't blame us for failing to anticipate *and* work around site-local practices which create security issues.
This is not a "local" practice. Anyone who is using the default freeradius configuration to authenticate EAP-TLS users has a gaping security hole. Any attacker who compromises a client certificate will be able to login without the security staff being able to identify which certificate was compromised, and that is going to be a problem. All because freeradius is not logging the cert's serial number and issuer...

>> For anyone who thinks this is an actual concern,
> 
>  Making snide insults to the developers is quite rude.  Don't do that.
I'm sorry - it wasn't intended that way. My two co-workers were in a "oh wow" shock moment when I showed them how I was able to authenticate with a certificate and a fake name, rendering them incapable of finding our which cert I had used to login. While I am at loss as to why you do not see the fact of the certificates not being logged as being a big security issue, I've posted the previous information for anyone who like me may have been searching for TLS info, looking for a solution to retrieve the certificate info, and spent the time to document it properly to try (1) to better clarify the root issue and (2) to assist anyone who like me isn't too versed in the ability to customize freeradius.

I again apologize for the tone - it's just frustrating being told that our environment is not configured correctly by saying we issued 40K certificates all being the same, when that is incorrect. All our certs are unique, and can uniquely identify the person/device they were issued to either with their serial number or CN. The problem is another - an attacker who is able to steal one and use it without us being able to tell which of the 40K cert was stolen as freeradius does not log that critical piece of information (cert serial/issuer/CN).

We have a solution. I'm merely pointing out to the radius community that whoever is using EAP-TLS is likely at risk as they are unable to track down the specifics of certificates used for authentication in case an attacker steals one of them, and they haven't modified their system to track them.

Roberto

> On Mar 22, 2021, at 3:25 PM, Alan DeKok <aland at deployingradius.com> wrote:
> 
> On Mar 22, 2021, at 2:46 PM, <Roberto.Franceschetti at ocfl.net> <Roberto.Franceschetti at ocfl.net> wrote:
>> 
>> I thank you both for letting me know that it's not possible to obtain certificate information when logging the accounting information. I had been looking for a way for the past week and would still be looking for one if it wasn't for your reply.
> 
>  That's good to hear.
> 
>> However I believe my point of this being a big freeradius security issue is incorrectly overlooked.
> 
>  Please be aware that FreeRADIUS is *strongly* limited by the RADIUS protocol.  We just can't do what we'd like to do, as you saw with trying to get TLS information into the Accounting process.
> 
>> Consider a scenario where freeradius is configured to authenticate against an LDAP/Active Directory.
>> 
>> * We detect IP 10.255.1.100 on our internal network currently attacking one of our Exchange servers.
>> * We see 10.255.1.100 as belonging to our WiFi DHCP pool.
>> * We thus go to the RADIUS radacct table to see the FramedIP address issued at the time of the attack.
>> * We see the UserName that was logged in as being "JohnS". We thus know exactly which LDAP account is compromised, are able to block the attack and quickly investigate how the account was compromised.
>> 
>> The key there is that access was granted with a username/password credential. The username in this case unequivocally identifies the account used to login WiFi, and that "identity" information was logged by freeradius.
> 
>  Mostly.  There are additional caveats, of course.  One is that the credential logged in radacct is the one sent by the NAS.  It is *usually* the same as the one in the Access-Request, but it doesn't *have* to be.
> 
>  FreeRADIUS just logs whatever the NAS sends it.  The NAS can (and will) lie about what it's doing.  Saying this is a security issue in FreeRADIUS is to severely misunderstand the problem.
> 
>> Now consider a different scenario (ours...) were, in addition to LDAP/Active Directory, we also allow EAP-TLS authentication via certificates issues by our certificate authority (we have issued over 40,000 certificates).
>> 
>> * We detect IP 10.255.1.100 on our internal network currently attacking one of our Exchange servers.
>> * We see 10.255.1.100 as belonging to our WiFi DHCP pool.
>> * We thus go to the RADIUS radacct table to see the FramedIP address issued at the time of the attack.
>> * We see the UserName that was logged in as being "HaHaHaTryToCatchMeIfYouCan".
> 
>  This User-Name is taken from the Access-Request packet.  Which in turn is derived from the User-Name in the EAP packet.  Which is under the control of the malicious user.
> 
>> We discover that the account "HaHaHaTryToCatchMeIfYouCan" does not exist in our LDAP/Active Directory.
> 
>  So... why did you let the user in?
> 
>  A good chunk of this problem is that you're not validating the user names.  You've previously said:
> 
>> While each device has its own unique client certificate having the device's serial number as the CN in the cert, the hardware vendors is using a generic username of "user" when specifying the username during the authentication process
> 
>  This is a problem.  Most people don't do this.  Most people put a real name into the certificate.  For devices, they put the device MAC address.  So this comes down to:
> 
>  "We provisioned entirely similar user credentials to 40K devices, and now we don't know which device is which".
> 
>  To which the only possible answer is "well, don't do that!"
> 
>> We then think the user probably authenticated with EAP-TLS and a certificate, but we're guessing since neither the radacct nor the radpostauth tables provide any clues as to the authentication used.
> 
>  This is why the configuration files are editable.  So you can edit them.
> 
>> * We check the "CallingStationID" in radacct to see if the MAC address of the hacker appears elsewhere, but it's a fake MAC.
> 
>  But the same MAC will be sent in both the Access-Request and Accounting-Request packets.
> 
>  Or, you can check the Framed-IP-Address as you said before.  Or is this no longer possible?  What changed?
> 
>  And why is it not possible to look at the NAS IP address / port, etc. as I suggested before?  Why isn't that information available?  If it isn't, I would hope that you would reply "No, it isn't available" instead of ignoring my comments and coming up with yet another scenario for why this doesn't work.
> 
>> We are thus in a nightmare scenario where the hacker was able to obtain the private key for one of our 40,000 certificates to authenticate with our WiFi, but we have absolutely no idea which of these 40K certificates was compromised. We will need to revoke them all and that would be a disaster.
> 
>  Sure.  So... configure your certificates to contain useful information, like names and/or MAC addresses.
> 
>  Use the information in the packets to cross-correlate Access-Request and Accounting-Request packets.
> 
>  This is all what people do.
> 
>> Saying "this is how radius works" does not apply here. There's nothing wrong with RADIUS. It's authenticating correctly. EAP-TLS allows the user to specify a random username as it's not taking part of the authentication process. There's nothing wrong with that either. WHY? Because they are authenticating with a certificate that we issued, and given any specific certificate's serial number, we know who it was issued to. If there's a problem with the specific certificate being compromised, we revoke it. Freeradius checks the CRL so it will not allow access to revoked certificates.
>> 
>> BUT WAIT. Freeradius is logging usernames for users who authenticating with a password, BUT it's NOT logging absolutely anything at for the certificate used by users who authenticate with a certificate. NOTHING.
> 
>  If only the configuration files were editable, and you could log the certificate serial number along with the fake User-Name.
> 
>  See?  It's easy.
> 
>> The issue is thus that while freeradius is be default logging the critical "Username" when regular authentication is occurring, freeradius is completely oblivious when it comes to logging the equivalent of the username when dealing with EAP-TLS. Any properly written application will have adequate auditing for who is logging into it. Freeradius is instead allowing users to login via EAP-TLS with absolutely no auditing whats-over.
> 
>  Nonsense.
> 
>  You've configured 40K certificates with "user" as the name.  Then, you're saying that FreeRADIUS is broken because it doesn't anticipate this situation, and log *extra* information that *you* need, and *only* you need for your particular deployment.
> 
>  Please don't blame us for a broken system.
> 
>> Auditing means auditing the identity used to login. Auditing an EAP-TLS username that is not actually being used for login purposes is useless - Freeradius MUST log whatever method is used to authenticate a user, and with EAP-TLS that means logging the TLS-Client-Certificate information.
> 
>  If only the configuration files were editable...
> 
>> As you said there's nothing from with RADIUS protocol. But there everything wrong with the default installation of freeradius when it comes to auditing EAP-TLS authentications.
> 
>  Nonsense.
> 
>  NORMAL installations doing NORMAL things work fine.  Your situation where you're doing very non-normal things doesn't work.  But FreeRADIUS gives you the power to fix it.
> 
>  So... fix it.
> 
>> This is a huge issue security hole as the logging of EAP-TLS certificate information that should be included in the default freeradius configuration, not left to admins to figure out that (1) they actually have an auditing problem of an application that's not logging who's authenticating, and (2) for them to figure out how to log it.
> 
>  Normal configurations work.  If you use unique user names for each certificate, then the default configuration works.  If you put "user" into all of the certs, then well...
> 
>  Yes, if you do unusual things, then you have to understand how the server works, and then configure it log extra information.
> 
>> We were able to implement what was missing after much pain and Googling.
> 
>  Or perhaps asking on this list?  You got an answer in much less than a day.  An answer which was specific, clear, and helpful.
> 
>> For anyone who thinks this is an actual concern,
> 
>  Making snide insults to the developers is quite rude.  Don't do that.
> 
>  I understand that YOUR system is insecure and broken.  Because YOU did unusual things, and the server magically doesn't know how to fix that.
> 
>  The solution is NOT to complain that the server is broken.  The solution is to ask for help, and to listen to the answers.
> 
>  The default configuration of the server works in as many situations as possible.  However, it does NOT automatically do everything that everyone wants, everywhere, always.  That's impossible.
> 
>> this is how we addressed it.
>> 
>> 
>> 1. Our NAS logs the MAC address of our wireless clients on authentication requests. So we modified the radpostauth table to include the "CallingStationID" (which has the MAC address of the client) and the client cert information. This allows us to had an audit trail of usernames, MAC and Client Cert for every Access-Request:
>> 
>> ALTER TABLE `radius`.`radpostauth` ADD COLUMN `CallingStationId ` VARCHAR(64) AFTER `nasipaddress`,
>> ADD COLUMN `TLS_Client_Cert_Common_Name` VARCHAR(256) AFTER `CallingStationId`,
> 
>  If the TLS-Client-Cert-Common name is always "user", then you can also check for bad user-names:
> 
> post-auth {
> 	...
> 	if (TLS-Client-Cert-Common-Name && (TLS-Client-Cert-Common-Name != User-Name)) {
> 		reject
> 	}
> 	...
> }
> 
>  2 lines of code will prevent users from lying about their User-Name.
> 
>  This doesn't solve the problem of having 40K certs with the same identity, but whatever.
> 
>> 4. Change the postauth_query in the dialup.conf module to log the CallingStationid and the Client Cert data in the database on the Access-Request:
>> postauth_query = "INSERT INTO ${postauth_table} \
>>                         (username, message, nasipaddress, reply, authdate, CallingStationid, TLS_Client_Cert_Common_Name, TLS_Client_Cert_Serial, TLS_Client_Cert_Issuer) \
>>                         VALUES ( \
>>                         '%{User-Name}', \
>>                         '%{Module-Failure-Message}', \
>>                         '%{NAS-IP-Address}', \
>>                         '%{reply:Packet-Type}', '%S', '%{Calling-Station-Id}', \
>>                         '%{TLS-Client-Cert-Common-Name}', '%{TLS-Client-Cert-Serial}', '%{TLS-Client-Cert-Issuer}')"
>> 
>> 
>> 5. Create and call every minute or so the following stored procedure which will update the certificate details for the records in the radacct table by matching logins via their MAC address (CallingStationId) in the radpostauth table (which does contain the certificate information previously retrieved/stored by the access-request).
> 
>  That's very heavy-weight.
> 
>  Or, in 3.0.18 and later we have a new policy "sql_session_start".  This policy is run in the post-auth section, and creates an entry in the radacct table:
> 
> https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/policy.d/accounting#L80
> 
>  it can then log TLS information there, too.
> 
>> The nightmare scenario above where EAP-TLS clients would be untraceable
> 
>  Again, because *your* deployment is broken.  Having 40K certificates with identity "user" is just terrible security practice.
> 
>> now becomes the following:
>> 
>> * We detect IP 10.255.1.100 on our internal network currently attacking one of our Exchange servers.
>> * We see 10.255.1.100 as belonging to our WiFi DHCP pool.
>> * We thus go to the RADIUS radacct table to see the FramedIP address issued at the time of the attack.
>> * We see the certificate's serial number and common name in the radacct table for that record
>> * We revoke the certificate, force a re-download of the CRL in the cron job, stop the attack, and can now focus on investigating how the user the certificate was issued to mishandled it.
> 
>  Or:
> 
> * create certificates with unique contents (device name, MAC address, etc.)
> * verify that the User-Name matches TLS-Client-Cert-Common-Name
> * use the default config to log default values
> 
>  At that point, it is *trivial* to figure out which certificate has been compromised.  There's no need to go edit the default configuration files.  And there's no need to complain that the default configuration is "insecure", because you have 40K users, all with the same name.
> 
>  Yes we didn't anticipate that someone would do that.  Largely because such practices are insecure.
> 
>  So... don't blame us for failing to anticipate *and* work around site-local practices which create security issues.
> 
>  Alan DeKok.
> 
> 
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

PLEASE NOTE: Florida has a very broad public records law (F. S. 119).
All e-mails to and from County Officials are kept as a public record.
Your e-mail communications, including your e-mail address may be
disclosed to the public and media at any time.