smbencrypt calculates false hash for German umlauts and other non-ASCII letters

[Alan DeKok]

Matthias Nagel wrote:
> Hi Phil,
>> Probably a fairly trivial patch if you feel like it ;o)
> I had a quick glace at the source code and I found two files named "smbencrypt.c". If you give me a hint, which is the correct file to start with, I will brosw the source code from that point and see what I can do. But probably not before next month.

  Please check src/modules/rlm_mschap/smbencrypt.c

  The main issue is that there is *no* character set information in the
MS-CHAP calculations.  The character set could be UTF-8, or any
non-standard 16-bit encoding.  So the calculation of the NT hash will
depend on the character set... which is largely secret.

  This makes it very difficult to create the *correct* NT hash.

  Alan DeKok.