Hi, I setup a cluster (Pacemaker) of two freeradius servers on CentOS 5.8 (freeradius 1.1.3). We use it for 802.1X: our switchs (HP Procurve) send it EAP challenges and it authenticates our users/hosts against an ActiveDirectory domain. It's been running smoothly for a month and stopped working this morning. We switched back to our old server but I want to know what happened before putting the cluster back in production. I'm currently trying to find out what happened but information is sparse: - From the cluster perspective, everything is fine. The daemon is running and no failover event occurred. - From samba, same thing. - In radius.log: nothing. There are client SSL certs errors but they have been there the whole time. - In auth-detail logfile: I can see all received requests. Before _and_ after the problem. However, it doesn't contain the reply :-/ - In detail logfile: Nothing after 7:56, time of the outage. So, I need pointers on two different issues: - I don't have enough information in logs. I realize that the recommended solution is to run the server in debug mode but I'm not sure I can store a month - or more - of such verbose logging. Any way to have more information, except the debug mode ? - I failed to check the whole authentication process both at the cluster level and at the monitoring level (nagios). Tools like NTradPing and check_radius_adv are of little help here because they can only check static accounts declared in radius configuration. They do not exercise the whole chain (Windows client > switch > radius server > ntlm_auth > AD). I tried replaying a successful authentication but it failed (Access-Denied). Any way to perform a 'real' check ? Sorry for the long post and thanks for any idea/pointer you can give me. Regards,
Hi,
I setup a cluster (Pacemaker) of two freeradius servers on CentOS 5.8 (freeradius 1.1.3).
with a version as hideously old as that (really! check the date it was released!) there could be one of many countless bugs you have hit. If you need (why?) the 1.1.x version then I would advise 1.1.8 (which is also no longer maintained). I would strongly recommend you migrate to FR 2.1.12 - not only is it maintained, with latest features, bug fixes etc but it is also faster. in short, even if ran in debug mode, you would either find a bug that was already fixed in post 1.1.3 releases - or find a bug that wont get fixed because 1.1.x is no longer maintained. alan
On 05/15/2012 06:58 AM, alan buxey wrote:
Hi,
I setup a cluster (Pacemaker) of two freeradius servers on CentOS 5.8 (freeradius 1.1.3).
with a version as hideously old as that (really! check the date it was released!) there could be one of many countless bugs you have hit.
RHEL 5.8 contains FreeRADIUS 2.1.12 under the package name freeradius2. -- John Dennis <jdennis@redhat.com> Looking to carve out IT costs? www.redhat.com/carveoutcosts/
Thanks to both of you for taking the time to answer.
I would strongly recommend you migrate to FR 2.1.12 - not only is it maintained, with latest features, bug fixes etc but it is also faster.
Unfortunately, this is not an option. HPIDM3 (a radius plugin provided by HP) doesn't work with Radius 2, and there is no way we can buy the upgrade to HPIDM4 any time soon. So I'm stuck with 1.1.x Anyway, I can live with a server that crashes once in a while, as long as I can detect it. The default timeout on the switches is 60 seconds, which is plenty enough to start a daemon and move a virtual IP... This brings me back to my second question: how do you monitor your Radius servers ? I haven't been able to find anything except very basic Nagios checks... Regards,
On 15/05/12 13:21, Julien Cornuwel wrote:
Thanks to both of you for taking the time to answer.
I would strongly recommend you migrate to FR 2.1.12 - not only is it maintained, with latest features, bug fixes etc but it is also faster.
Unfortunately, this is not an option. HPIDM3 (a radius plugin provided by HP) doesn't work with Radius 2, and there is no way we can buy the upgrade to HPIDM4 any time soon. So I'm stuck with 1.1.x
At the very least, upgrade to 1.1.8
Anyway, I can live with a server that crashes once in a while, as long as I can detect it. The default timeout on the switches is 60 seconds, which is plenty enough to start a daemon and move a virtual IP...
This brings me back to my second question: how do you monitor your Radius servers ? I haven't been able to find anything except very basic Nagios checks...
You implied you wanted to test a "real" authentication and tried to replay one, which didn't (and shouldn't) work. Instead, I recommend you download the wpa_supplicant sources, and compile eapol_test. Write a wrapper script to run this, and perform a full 802.1x auth against the server. If it fails, it's down and you need to restart it. We do something like this: #!/bin/bash EAPOL_TEST=/usr/local/bin/eapol_test CFG=`mktemp` BUF=`mktemp` trap "rm -f $CFG $BUF" EXIT # write out the eapol_test config file cat <<EOF >$CFG network={ ssid="example 802.1x network" key_mgmt=IEEE8021X eap=PEAP phase2="auth=MSCHAPV2" identity="user@domain" password="XXX" } EOF # run eapol_test $EAPOL_TEST -c $CFG -a 127.0.0.1 -p 1812 -s testing123 >$BUF 2>&1 if [ $? -ne 0 ] then echo eapol_test failed exit 1 fi # it's fine exit 0
Thank you. I'll try that as soon as I'm done upgrading to 1.1.8 2012/5/15 Phil Mayers <p.mayers@imperial.ac.uk>:
On 15/05/12 13:21, Julien Cornuwel wrote:
Thanks to both of you for taking the time to answer.
I would strongly recommend you migrate to FR 2.1.12 - not only is it maintained, with latest features, bug fixes etc but it is also faster.
Unfortunately, this is not an option. HPIDM3 (a radius plugin provided by HP) doesn't work with Radius 2, and there is no way we can buy the upgrade to HPIDM4 any time soon. So I'm stuck with 1.1.x
At the very least, upgrade to 1.1.8
Anyway, I can live with a server that crashes once in a while, as long as I can detect it. The default timeout on the switches is 60 seconds, which is plenty enough to start a daemon and move a virtual IP...
This brings me back to my second question: how do you monitor your Radius servers ? I haven't been able to find anything except very basic Nagios checks...
You implied you wanted to test a "real" authentication and tried to replay one, which didn't (and shouldn't) work.
Instead, I recommend you download the wpa_supplicant sources, and compile eapol_test. Write a wrapper script to run this, and perform a full 802.1x auth against the server. If it fails, it's down and you need to restart it.
We do something like this:
#!/bin/bash
EAPOL_TEST=/usr/local/bin/eapol_test
CFG=`mktemp` BUF=`mktemp`
trap "rm -f $CFG $BUF" EXIT
# write out the eapol_test config file cat <<EOF >$CFG network={ ssid="example 802.1x network" key_mgmt=IEEE8021X eap=PEAP phase2="auth=MSCHAPV2" identity="user@domain" password="XXX" } EOF
# run eapol_test $EAPOL_TEST -c $CFG -a 127.0.0.1 -p 1812 -s testing123 >$BUF 2>&1 if [ $? -ne 0 ] then echo eapol_test failed exit 1 fi
# it's fine exit 0
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Thank you guys! Phil, your check script works perfectly. Maybe 1.1.8 will behave better. But even if it doesn't, as long as the cluster layer detects it and reacts, it's fine by me. Regards,
Will do. Thanks. 2012/5/15 Alan DeKok <aland@deployingradius.com>:
Julien Cornuwel wrote:
I setup a cluster (Pacemaker) of two freeradius servers on CentOS 5.8 (freeradius 1.1.3).
If you're stuck on 1.1.x, use 1.1.8. Really.
Don't do ANYTHING else until you upgrade. It could be a solved bug.
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
participants (5)
-
alan buxey -
Alan DeKok -
John Dennis -
Julien Cornuwel -
Phil Mayers