Compiling Python & FreeRadius 3.x

[Andrew Parisio]

If we use the apt/yum installed versions of python it works, but if we
compile python (tried 2.7.3, .6, and .10, installing in /usr/local) and
then configure freeradius to look at the compiled versions we are able to
get it to work, but as soon as you import a module such as json or boto
(the aws sdk) we hit the PyThreadState_Get error.  When looking at the
strace we have verified that all references to python are to the
correct/compiled version, there are no accidental references to the local
system version of python.

We've tried this in several different environments (internal build tools,
clean copies of ubuntu & amazon linux), and have been able to reliably
reproduce the same PyThreadState_Get error message, but have been unable to
figure out why it's happening.

I think it's something to do with the way freeradius is compiled, however
it's also possible it's related to the way python itself is compiled but I
think that's less likely because we used the same flags used to compile the
yum version of python.

FYI we can compile freeradus 2.2.x against our compiled python and
everything works just fine (but freeradius 2.2.x doesn't support threaded
python).


This is the script used to build/install python & freeradius on Amazon
Linux 2015.03 on EC2:

sudo su

cd

yum update -y

yum groupinstall "Development Tools" -y

yum install libtalloc-devel openssl-devel -y

yum remove python27-devel -y # so there's no way it accidnetally compiles
against the system python

wget https://www.python.org/ftp/python/2.7.6/Python-2.7.6.tgz

wget ftp://ftp.freeradius.org/pub/freeradius/freeradius-server-3.0.8.tar.gz

tar -xzvf Python-2.7.6.tgz

tar -zxvf freeradius-server-3.0.8.tar.gz

PATH=/usr/local/bin:$PATH

export PATH

PYTHONHOME=/usr/local

export PYTHONHOME



cd /root/Python-2.7.6 && ./configure  --enable-shared
LDFLAGS="-Wl,-rpath,/usr/local/lib/python2.7" LINKFORSHARED="-Xlinker
--export-dynamic" CPPLAGS="-fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g
-fwrapv -O3 -Wall -Wstrict-prototypes"  --prefix=/usr/local &&  make -j16
&& make altinstall



cd /root/freeradius-server-3.0.8 && ./configure
LDFLAGS="-Wl,-rpath,/usr/local/lib/python2.7" BASECFLAGS="-fPIC -Xlinker
--export-dynamic"  --with-rlm-python-lib-dir=/usr/local/lib/python2.7
--with-rlm-python-include-dir=/usr/local/include/python2.7 && make -j16 &&
make install



cp /usr/local/etc/raddb/mods-available/python
/usr/local/etc/raddb/mods-enabled/python

echo "import json # comment out this line and radius will run

def instantiate(p):

    print json.dumps({})

    print 'YAY!'

" > /usr/local/lib/python2.7/site-packages/example.py

 vi /usr/local/etc/raddb/mods-enabled/python # uncomment the insantiate
line (too lazy to sed)



then /usr/local/sbin/radiusd -X -xxx

stat("/usr/local/lib/python2.7/_locale", 0x7ffd1f05b170) = -1 ENOENT (No
such file or directory)
open("/usr/local/lib/python2.7/_locale.so", O_RDONLY) = -1 ENOENT (No such
file or directory)
open("/usr/local/lib/python2.7/_localemodule.so", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/usr/local/lib/python2.7/_locale.py", O_RDONLY) = -1 ENOENT (No such
file or directory)
open("/usr/local/lib/python2.7/_locale.pyc", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat("/usr/local/lib/python27.zip", 0x7ffd1f059f10) = -1 ENOENT (No such
file or directory)
stat("/usr/local/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/usr/local/lib/python27.zip", 0x7ffd1f05aec0) = -1 ENOENT (No such
file or directory)
stat("/usr/local/lib/python2.7/plat-linux2", {st_mode=S_IFDIR|0755,
st_size=4096, ...}) = 0
stat("/usr/local/lib/python2.7/plat-linux2", {st_mode=S_IFDIR|0755,
st_size=4096, ...}) = 0
stat("/usr/local/lib/python2.7/plat-linux2/_locale", 0x7ffd1f05b170) = -1
ENOENT (No such file or directory)
open("/usr/local/lib/python2.7/plat-linux2/_locale.so", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/local/lib/python2.7/plat-linux2/_localemodule.so", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/usr/local/lib/python2.7/plat-linux2/_locale.py", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/local/lib/python2.7/plat-linux2/_locale.pyc", O_RDONLY) = -1
ENOENT (No such file or directory)
stat("/usr/local/lib/python2.7/lib-tk", {st_mode=S_IFDIR|0755,
st_size=4096, ...}) = 0
stat("/usr/local/lib/python2.7/lib-tk", {st_mode=S_IFDIR|0755,
st_size=4096, ...}) = 0
stat("/usr/local/lib/python2.7/lib-tk/_locale", 0x7ffd1f05b170) = -1 ENOENT
(No such file or directory)
open("/usr/local/lib/python2.7/lib-tk/_locale.so", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/local/lib/python2.7/lib-tk/_localemodule.so", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/local/lib/python2.7/lib-tk/_locale.py", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/local/lib/python2.7/lib-tk/_locale.pyc", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat("/usr/local/lib/python2.7/lib-old", 0x7ffd1f059f10) = -1 ENOENT (No
such file or directory)
stat("/usr/local/lib/python2.7", {st_mode=S_IFDIR|0755, st_size=20480,
...}) = 0
stat("/usr/local/lib/python2.7/lib-old", 0x7ffd1f05aec0) = -1 ENOENT (No
such file or directory)
stat("/usr/local/lib/python2.7/lib-dynload", {st_mode=S_IFDIR|0755,
st_size=4096, ...}) = 0
stat("/usr/local/lib/python2.7/lib-dynload", {st_mode=S_IFDIR|0755,
st_size=4096, ...}) = 0
stat("/usr/local/lib/python2.7/lib-dynload/_locale", 0x7ffd1f05b170) = -1
ENOENT (No such file or directory)
open("/usr/local/lib/python2.7/lib-dynload/_locale.so", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0755, st_size=60757, ...}) = 0
open("/usr/local/lib/python2.7/lib-dynload/_locale.so", O_RDONLY|O_CLOEXEC)
= 6
read(6,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\32\0\0\0\0\0\0"...,
832) = 832
fstat(6, {st_mode=S_IFREG|0755, st_size=60757, ...}) = 0
mmap(NULL, 2116856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 6, 0) =
0x7f2398e93000
mprotect(0x7f2398e97000, 2097152, PROT_NONE) = 0
mmap(0x7f2399097000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 6, 0x4000) = 0x7f2399097000
close(6)                                = 0
write(2, "Fatal Python error: PyThreadStat"..., 57Fatal Python error:
PyThreadState_Get: no current thread
) = 57
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
tgkill(1420, 1420, SIGABRT)             = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=1420, si_uid=0} ---
+++ killed by SIGABRT +++
Aborted

Does anybody have any thoughts / suggestions?

Thanks!