After hunting around for a couple of hours today for a decent build system we decided to try out Travis CI. Travis is a hosted continuous integration system which is integrated into GitHub. Whenever someone opens a pull request, or pushes to the master branch it triggers a Travis build. Travis checks out the commit onto a basic Ubuntu 12.04 VM, installs any dependencies you have set, and builds the server with clang and gcc. Currently building involves ./configure && make && make test. Travis supports a number of fixtures including MySQL and PostgreSQL so we should be able to get fairly good coverage on modules that traditionally haven't been tested that much because it was annoying to setup test environments. If you want to check on the current build state, we've added the build status button to the readme file, so it'll show up if you go on the root freeradius-server repo url. https://github.com/FreeRADIUS/freeradius-server or go here: https://travis-ci.org/FreeRADIUS/freeradius-server To checkout the status of the latest builds. -Arran
Hi,
Travis supports a number of fixtures including MySQL and PostgreSQL so we should be able to get fairly good coverage on modules that traditionally haven't been tested that much because it was annoying to setup test environments.
building is one thing....having a working environment is another - eg will there be a client making a RADIUS request that USES the DB etc etc - will there be a 'test config; that runs all the FR bells and whistles? alan
On 2013-02-26, at 4:01 AM, A.L.M.Buxey@lboro.ac.uk wrote:
building is one thing....having a working environment is another - eg will there be a client making a RADIUS request that USES the DB etc etc - will there be a 'test config; that runs all the FR bells and whistles?
"make test" already does a basic test. It creates a test config, and sends a bunch of packets using radclient. More tests would be nice. As always, patches are welcome. Alan DeKok.
On 26 Feb 2013, at 04:01, A.L.M.Buxey@lboro.ac.uk wrote:
Hi,
Travis supports a number of fixtures including MySQL and PostgreSQL so we should be able to get fairly good coverage on modules that traditionally haven't been tested that much because it was annoying to setup test environments.
building is one thing....having a working environment is another - eg will there be a client making a RADIUS request that USES the DB etc etc -
Travis has support for database/directory fixtures, so yes this will be possible.
will there be a 'test config; that runs all the FR bells and whistles?
Well sure if someone has the time to write test cases for the hundreds of the configuration options. -Arran
On 02/26/2013 03:51 AM, Arran Cudbard-Bell wrote:
After hunting around for a couple of hours today for a decent build system we decided to try out Travis CI.
Slightly curious why you didn't go with buildbot; FWIW on other projects I've seen community members contribute buildbot nodes on many and varied platforms, which can help with coverage. As we've seen this week, ubuntu is not the entirety of Linux ;o) Still, any CI is better than none. I wonder if it might be worth considering a small dev process change at this point to, which is to always commit/pull to a dedicated branch, and only pull that to master once CI/testsuite has passed (probably automated)? This might help others who base/build off master, and will leave master in a state where it always builds & runs. Just a thought!
On 26 Feb 2013, at 04:29, Phil Mayers <p.mayers@IMPERIAL.AC.UK> wrote:
On 02/26/2013 03:51 AM, Arran Cudbard-Bell wrote:
After hunting around for a couple of hours today for a decent build system we decided to try out Travis CI.
Slightly curious why you didn't go with buildbot;
Time constraints. It took all of 10 minutes to skip the documentation and get a working (though not passing) build system with Travis, and it takes absolutely no time to maintain seeing as it's all centrally hosted.
FWIW on other projects I've seen community members contribute buildbot nodes on many and varied platforms, which can help with coverage. As we've seen this week, ubuntu is not the entirety of Linux ;o)
True, though the problems with rlm_perl were actually down to the version of perl, and not the fact it was running on Centos. The segfault OTOH was a bit weird, so yes it'd probably help to have examples available of different Linux flavours. I think the Travis team are working on adding OSX at least. I'm sure the list of supported OS will grow as interest in the project increases.
Still, any CI is better than none. I wonder if it might be worth considering a small dev process change at this point to, which is to always commit/pull to a dedicated branch, and only pull that to master once CI/testsuite has passed (probably automated)? This might help others who base/build off master, and will leave master in a state where it always builds & runs. Just a thought!
Yes. That was the idea behind creating the vp_fixup and talloc branches. Unfortunately issues still sometimes appear when merging. -Arran
https://travis-ci.org/FreeRADIUS/freeradius-server/branches Shows nicely how much more efficient the new build system is, even when building from scratch. -Arran
Arran Cudbard-Bell wrote:
https://travis-ci.org/FreeRADIUS/freeradius-server/branches
Shows nicely how much more efficient the new build system is, even when building from scratch.
Weird. It's using libtool, not boilermake. Arg. That should be fixed. And there seems to be leftovers of libltdl. I'll take a look at nuking those. Alan DeKok.
On 26 Feb 2013, at 11:53, Alan DeKok <aland@DEPLOYINGRADIUS.COM> wrote:
Arran Cudbard-Bell wrote:
https://travis-ci.org/FreeRADIUS/freeradius-server/branches
Shows nicely how much more efficient the new build system is, even when building from scratch.
Weird. It's using libtool, not boilermake. Arg. That should be fixed.
And there seems to be leftovers of libltdl. I'll take a look at nuking those.
Only in 2.x.x branch? Or did you see something in master... Anyway Travis just caught its first pull request build failure :D https://github.com/FreeRADIUS/freeradius-server/pull/193 Which we wouldn't have seen from buildbot etal until after the merge... -Arran
On Tue, Feb 26, 2013 at 12:20:17PM -0500, Arran Cudbard-Bell wrote:
Anyway Travis just caught its first pull request build failure :D
Oops, that was me :-) I pushed before a successful local build. I was having more fundamental problems building master: ... checking type of OS... Darwin checking for developer gcc flags... yes. Using -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -W -Wredundant-decls -Wundef -Wformat-y2k -Wno-format-extra-args -Wno-format-zero-length -Wformat-nonliteral -Wformat-security -Wformat=2 checking for TLS... no checking for lt_dladvise_init... no checking for talloc.h... no configure: WARNING: talloc headers not found. Use --with-talloc-include-dir=<path>. configure: error: FreeRADIUS requires libtalloc I thought I'd read that FreeRADIUS was going to be bundled with its own talloc, but it appears I was wrong. Anyway, I've just installed talloc from homebrew now. Regards, Brian. P.S. Does the continuous-integration stuff include experimental modules, like rlm_redis{,who} which are not in src/modules/stable?
On 26 Feb 2013, at 12:49, Brian Candler <B.Candler@pobox.com> wrote:
On Tue, Feb 26, 2013 at 12:20:17PM -0500, Arran Cudbard-Bell wrote:
Anyway Travis just caught its first pull request build failure :D
Oops, that was me :-) I pushed before a successful local build.
I was having more fundamental problems building master:
... checking type of OS... Darwin checking for developer gcc flags... yes. Using -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -W -Wredundant-decls -Wundef -Wformat-y2k -Wno-format-extra-args -Wno-format-zero-length -Wformat-nonliteral -Wformat-security -Wformat=2 checking for TLS... no checking for lt_dladvise_init... no checking for talloc.h... no configure: WARNING: talloc headers not found. Use --with-talloc-include-dir=<path>. configure: error: FreeRADIUS requires libtalloc
I thought I'd read that FreeRADIUS was going to be bundled with its own talloc, but it appears I was wrong.
Anyway, I've just installed talloc from homebrew now.
Regards,
Brian.
P.S. Does the continuous-integration stuff include experimental modules, like rlm_redis{,who} which are not in src/modules/stable?
It does. All git checkouts build in developer mode, all developer mode builds enable the experimental modules by default. As for talloc, yes it probably will still be bundled, but just to get things working we added it as an external dependency. Packaged talloc seems to work fine on the systems we've tested, so for now just: sudo apt-get install libtalloc-dev sudo yum install libtalloc-devel brew install talloc The plan is for CI builds to build as much as possible, hence the request for the SecurID libraries. I guess we'll have to get oracle instaclient from somewhere as well, and I shudder to think what the mssql client requires, probably sacrificial chickens and puppie dogs. -Arran
On 26 Feb 2013, at 13:35, Arran Cudbard-Bell <a.cudbardb@freeradius.org> wrote:
On 26 Feb 2013, at 12:49, Brian Candler <B.Candler@pobox.com> wrote:
On Tue, Feb 26, 2013 at 12:20:17PM -0500, Arran Cudbard-Bell wrote:
Anyway Travis just caught its first pull request build failure :D
Oops, that was me :-) I pushed before a successful local build.
I was having more fundamental problems building master:
... checking type of OS... Darwin checking for developer gcc flags... yes. Using -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -W -Wredundant-decls -Wundef -Wformat-y2k -Wno-format-extra-args -Wno-format-zero-length -Wformat-nonliteral -Wformat-security -Wformat=2 checking for TLS... no checking for lt_dladvise_init... no checking for talloc.h... no configure: WARNING: talloc headers not found. Use --with-talloc-include-dir=<path>. configure: error: FreeRADIUS requires libtalloc
I thought I'd read that FreeRADIUS was going to be bundled with its own talloc, but it appears I was wrong.
Anyway, I've just installed talloc from homebrew now.
Regards,
Brian.
P.S. Does the continuous-integration stuff include experimental modules, like rlm_redis{,who} which are not in src/modules/stable?
It does. All git checkouts build in developer mode, all developer mode builds enable the experimental modules by default.
As for talloc, yes it probably will still be bundled, but just to get things working we added it as an external dependency.
Packaged talloc seems to work fine on the systems we've tested, so for now just:
sudo apt-get install libtalloc-dev sudo yum install libtalloc-devel brew install talloc
The plan is for CI builds to build as much as possible, hence the request for the SecurID libraries. I guess we'll have to get oracle instaclient from somewhere as well, and I shudder to think what the mssql client requires, probably sacrificial chickens and puppie dogs.
Or just freetds :) Guess we should probably add autoconf stuff and makefiles for the rest of the database drivers. -Arran
participants (5)
-
A.L.M.Buxey＠lboro.ac.uk -
Alan DeKok -
Arran Cudbard-Bell -
Brian Candler -
Phil Mayers