radius timeout

Hi all I have freeradius-1.0.1-1 (with Petr Nixon's patch) running on FC-3 and using a postgresql backend to collect cisco AAA (stop only) records. My cisco IOS is 12.3(2)T2. I see a lot of radius timeout on my cisco router while the ping times is < 10ms and my radius timeout is set to 50. I even went further to connect the radius server and the router back to back with a cross cable and still have some timeouts. What could be wrong with my configs? Any help will be appreciated. Thank you.