How *not* to do research: MS NPS vs FreeRADIUS

[Alan DeKok]

  I found a fun paper today.  Their conclusion?  MS NPS is faster than FreeRADIUS!

http://iajit.org/PDF/November%202019,%20No.%206/13255.pdf

  Some quotes, and my comments follow.

The research revealed that FreeRADIUS by default
allows the support for a variety of authentication
methods including the ones using clear text passwords.
This is okay if the implementer thinks of a broader
compatibility but at the same time allows the network
intruders may manipulate the user passwords.
Additionally, FreeRADIUS was trying to initiate full
authentication sessions when only reconnections were
required. This ended up in increased overhead of the
authentication system

  Comment: They didn't enable fast session resumption, so FreeRADIUS didn't use it.  They then blamed FreeRADIUS for not using it.

  This is proven via the following statement:

Overhead in reconnections were large in FreeRADIUS
as it initiated a new EAP session altogether, resulting
in an overhead nearly equal to that of a complete
authentication phase.

  Comment: Yes, if you don't enable fast session resumption, FreeRADIUS does a complete authentication.  It's not "nearly equal" to a complete authentication, it *is* a complete authentication.

In addition
to the RADIUS service, Microsoft NPS was capable of
checking the health state of the connecting client and
any non-compliant clients can be blocked or applied
for automatic remediation of network health
procedures.

  Comment: Not true.  FreeRADIUS has had SoH support for a long time now.  And further, it's not a surprise that MS products work together with proprietary MS protocols.

Overhead in failed authentications were observed
lower in Microsoft NPS

  Comment: Because FreeRADIUS has "reject_delay = 1" by default.  For security reason.  However, MS NPS doesn't do this.  So it's less secure, but it's faster!

Microsoft NPS has better performance for successfully
authenticated sessions.

  Comment: Because they didn't change the default configuration.  Which does all kinds of things that are useful to many people, but which aren't required for PEAP.

  The take-away is:

* it's easy to generate fake statistics if you're willing to do a hack job of analysis
* the default configuration of MS NPS is optimized for Windows and PEAP
* the default configuration of FreeRADIUS is optimized to work everywhere

  Don't be fooled by nice graphs and polite lies.  This only worth of this paper is to show people how *not* to write a paper.

  Alan DeKok.