smbencrypt calculates false hash for German umlauts and other non-ASCII letters

[Matthias Nagel]

Hi Phil,
> Probably a fairly trivial patch if you feel like it ;o)
I had a quick glace at the source code and I found two files named "smbencrypt.c". If you give me a hint, which is the correct file to start with, I will brosw the source code from that point and see what I can do. But probably not before next month.
Matthias


Am Sonntag 18 August 2013, 17:44:46 schrieb Phil Mayers:
> Matthias Nagel <matthias.h.nagel at gmail.com> wrote:
> >Hello,
> >
> >if a do a "smbencrypt ä" then the output for the NT hash is
> >"B5CF5E386433C7CB69E43ED774717792" but the correct hash would be
> >"3104EAB484D59EFABCEA2C44B07F41D3". (If you do not see the letter: It
> >is a small "a" with two dots, unicode code point 00E4.) Similar results
> >hold for other umlauts, too.
> >
> >My Freeradius version is 2.2.0 running on Linux 3.8.13 with system
> >locale set to en_US.utf8.
> >
> >I wrote an own utitly to calculate NT hashes to fill the Radius
> >database. While I compared the results of my own utility with those
> >from "smbencrypt", I found these discrepancies. In order to check which
> >result was the correct one, I took a Windows computer, added a dummy
> >user to it and set the passwords in concern. Then I extracted the NT
> >hashes from the SAM database.
> >
> >One note of caution: If you take a web site like
> >http://www.onlinehashcrack.com/hash-calculator.php, do not trust it. If
> >it comes to non-ASCII letters the output is false, too.
> >
> >Matthias
> >
> >----------------------------------------------------------------------
> >Matthias Nagel
> >Parkstraße 27
> >76131 Karlsruhe
> >
> >Mobil: +49-151-15998774
> >e-Mail: matthias.h.nagel at gmail.com
> >ICQ: 499797758
> >Skype: nagmat84
> >
> >-
> >List info/subscribe/unsubscribe? See
> >http://www.freeradius.org/list/users.html
> 
> Almost certainly. Nt hashes are the 16-bit encoding, and smbencrypt likely treats each byte in the utf8 encoding as on ASCII char and pads it to 16 bits.
> 
> I made some effort to handle this in the mschap password change code, but really the server should probably pull in libiconv for the few places this is needed (such as calculating correct nt hashes). Probably a fairly trivial patch if you feel like it ;o)
> 
----------------------------------------------------------------------
Matthias Nagel
Parkstraße 27
76131 Karlsruhe

Mobil: +49-151-15998774
e-Mail: matthias.h.nagel at gmail.com
ICQ: 499797758
Skype: nagmat84