FWIW, I've had very little trouble getting YubiKeys to work (in OATH-HOTP mode) with the Google Authenticator PAM through FreeRADIUS. The benefit to that was that it allowed me to support YubiKeys and the Google Authenticator app with the same configuration so that users could have one or the other (YubiKeys are definitely more secure and more universal than smartphone apps, but there is a cost attached to purchasing and (re-)programming them).
In my case the FreeRADIUS side of this was pretty easy to configure since my NAS allowed for a secondary authentication server, so all I really had to do in FR was deal with the OTP, although later on I also set up a two-step process using FreeRADIUS for both stages with an Access-Challenge to drive the OTP authentication.