I think parts of our conversation move towards a non-productive direction; probably because I am using different terminology (e.g. "machine authentication") due to my unfamiliarity with the topic.
Before going ahead let me once again describe the setup I want:
1.) Client presents a certificate signed by the CA -> authentication should succeed ("machine authentication"). (I thought this would best be done via EAP-TLS but not sure)
If the client is configured with a cert and to use TTLS then freeradius will use that. The radius server does not tell the client which method to use.
2.) If the client does NOT have a client certificate signed by the CA installed it should query for username/password ("user authentication") exactly as in my current setup. Authentication should succeed via PEAP-MSCHAPv2 if correct credentials are presented.
If you have a certificate the client will be need to be configured for TTLS. If not the client/supplicant will be configured for PEAP. I'll repeat, the radius server responds to the method configured in the client/supplicant. It is all a client configuration issue. The client configuration determines the authentication method, not the radius server.