Possible issue with UTF8 misconversion (username with umlauts)

[Alan DeKok]

On Jul 27, 2022, at 7:29 AM, Herda, Martin <Martin.Herda at fritz-edv.de> wrote:
> We've got an issue with umlauts in usernames:
> 
> We are using privacyIDEA to enroll 2FA TOTP token to users for securing login process to VMware Horizon View infrastructure. (Homeoffice use)
> In backend, privacyIDEA is using freeRadius (in conjunction with rlm_perl) to authenticate users against active directory.
> 
> When a user with umlaut is trying to log in, it seems that there is a (double) misconversion happening:

  The server core doesn't care about UTF-8 or random binary data.  It's all just data.

> Radius.log:
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Config File /etc/privacyidea/rlm_perl.ini found!
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Debugging config: 
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Default URL https://localhost/validate/check 
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Looking for config for auth-type Perl
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Password encoding guessed: ascii
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Setting client IP to 192.168.0.12.
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: Auth-Type: Perl
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: url: https://localhost/validate/check
> 	Wed Jul 27 08:31:14 2022 : Info: rlm_perl: user sent to privacyidea: G%C3%83%C2%B6the

  So... what happened before that?

  Please READ the documentation, and FOLLOW IT.  See http://wiki.freeradius.org/list-help

> Username should be 'Göthe' but somwhere on the way it gets messed up to 'G%C3%83%C2%B6the'

  If only there was some kind of debugging mode which could tell you what's going on.

> Users without umlauts are working fine. I don't know, in which part exactly the things gets messed up. But I think this issue should lie anywhere between freeRadius and rlm_perl, because the first instance right after the Horizon Server is freeRadius and rlm_perl.

  FreeRADIUS does many things when it receives a request.  But people also change the local configuration.  So until we get some debug output, we have no idea.

  What I can say for absolute certainty is that the default configuration does not change umlauts for %-encoded data.

  I'd also check the Perl module you're using.  Any non-ASCII data sent to a web URL is usually URL encoded... with % and hex characters.  So it might not be anything in FreeRADIUS.

> Any ideas how to solve this issue? But removing/replacing umlauts is not an option.

  http://wiki.freeradius.org/list-help

  Alan DeKok.