Following up from this, I think I've discovered what the real problem here is. I think there's a problem with the MS-CHAP module....

The module looks in the username to find "host/" at the beginning, and if it does then handles it differently. Whilst it sets the "username" section correctly, it doesn't set the "domain" section properly.

ntlm_auth can handle both netbios and FQDN versions of a domain. For machine Auth, the mschap module works on the assumption that the first "DN=" bit of the FQDN is always the same as the netbios name - which in many situations it is, but not all the time. It should work on the logic of: "OK, I found a host/ at the beginning, so everything after the /host but before the first '.' + a '$' is the username of the machine, and *everything* after the first '.' is the domain name, not everything between the first and second periods is the domain name.

My C programming isn't too hot, and so I'm not sure how to correct this logic - even though I think I've found it in source for rlm_mschap.