free radius won't use libtallsc

I'm trying to setup freeradius3.0.9 on cents 7 and I'm having an issue with it. When I first use ./configure it says that I can't find libtalloc so I had to use the --with-talloc-lib-dir flag. Before I used the flag I tried using yum to install the library but it said it was already there. I found the library in /usr/lib64 so I used the flag to point to that. But it still says that it can't see the library. How can I get talloc to work?