bootstrap cert generation issues

[John Dennis]

We've had some bug reports with not generating the bootstrap certs
(correctly) with some of our packages. I investigated and found two
issues which we've fixed. I would send a patch but our fix is not
generic because it depends on knowing the uid and gid of the server
when either the bootstrap script or the Makefile executes. Here are
the two issues in summary, then I'll follow with detail:

1) Failure to set openssl's $RANDFILE environment variable.
    Symptom is "unable to write 'random state'" error.

2) 10 second sub-process timeout when executing bootstrap.
    Symptom is incomplete cert generation, failure to start.

Details
-------

1) /etc/raddb/certs/Makefile creates the random file
    /etc/raddb/certs/random, it has 0640 permissions.  If bootstrap is
    run directly by root it's ownership is root:root, if bootstrap is
    run by radiusd -X it's ownership is root:radiusd (because that's
    the effective uid:gid of radiusd during it's configuration phase)

2) openssl's command line tools use a random file defined by the
    environment variable $RANDFILE. if $RANDFILE is not defined it
    defaults to $HOME/.rnd. $RANDFILE is often re-written by the
    openssl command line tools, when it is re-written it's permissions
    are forced to 0600.

3) /etc/raddb/certs/Makefile does not set $RANDFILE, thus openssl does
    not use /etc/raddb/certs/random, rather it uses $HOME/.rnd which
    because the uid is root is /root/.rnd (on Linux). Also this means
    the file /etc/raddb/certs/random is never used nor modified during
    cert generation.

    This is why we sometimes get "unable to write 'random state'"
    because if root's homedir already has a .rnd file radiusd does not
    have permission to write it.

    See http://www.openssl.org/support/faq.html#USER2
    for a complete explanation.

4) When radiusd starts up it needs to read /etc/raddb/certs/random
    because it configures openssl to use that file as it's random
    file. Because that file is not modified in item #3 and retains it's
    0640 permissions there are no initialization problems.

5) However if the Makefile is modfied to fix the problem defined in
    item #2 by exporting the environment variable
    $RANDFILE=/etc/raddb/certs/random then the openssl command line
    tools will rewrite the file with permissions 0600. This causes
    radiusd to fail it's initialization because it's running as
    root:radiusd and cannot read the random file.

6) We DO NOT want to write any file in $HOME, therefore we must set
    $RANDFILE. Also since the openssl command line tools will generate
    $RANDFILE there is no point in having the Makefile generate it,
    that should be removed. After the Makefile is run it must reset the
    permissions of $RANDFILE to 0640 so radiusd can read it.

7) The cert boostrap program is run by the function
    radius_exec_program() (in the file src/main/exec.c:74). The
    function radius_exec_program() has a hardcoded time limit of 10
    seconds for child processes. If 10 seconds of elapsed time is
    exceeded it attempts to kill the child process.

    Testing the bootstap progam on an i386 virtual machine shows it's
    execution duration varies widely, between 5 seconds and 30 seconds.

    In fact the openssl dh parameter generation explictily warns:
    "This is going to take a long time". Thus it should be expected the
    bootstrap cert generation could easily exceed 10 seconds of elapsed
    time.

    Thus even when all the issues mentioned above are corrected the
    bootstrap cert generation will sometimes fail and sometimes succeed
    depending on whether it exceeded the 10 second timeout. This
    clearly is not acceptable. However it does make sense to limit
    child processes to 10 seconds during normal server execution.

In Fedora and RHEL6 we moved the bootstrap cert generation to the RPM
install, thus the bootstrap certs were generated the first time the
RPM was installed, running radius -X did not run the cert
bootstrap. This is preferable for a variety of reasons:

a) You shouldn't need to run the server in debug mode (e.g. radiusd
    -X) to get it configured to run the first time, this is clumsy and
    confusing to users. The only reason it might not be confusing to
    users is the "bootstrap during first debug" behaviour is discussed
    widely on the FreeRADIUS mailing lists (and probaby appears in
    documentation someplace).

b) It doesn't suffer the permission problems with the random file. But
    that's just a fortunate artifact because a package install runs as
    root. We should fix those problems irrespective of where the first
    bootstrap is run from, in part because sometimes an admin might run
    the cert generation manually.

c) The duration of the bootstrap command is irrelevant when run by the
    RPM install, it will not be terminated if it exceeds the hardcoded
    10 second timeout.

Although there is a good argument for generating the temporary
bootstrap certs during initial package install this may not an option
for the generic FreeRADIUS distribution because it is not package
based, rather it's a configure,make,install sequence. However there is
no reason the bootstrap cert generation couldn't be moved to the
install phase rather than being invoked from a running server in debug
mode.

The RHEL5 version of freeradius2 was packaged prior to moving the cert
bootstrap to RPM install as is currently done in Fedora and
RHEL6. Therefore we only were seeing those problems in RHEL5 because
the problems did not manifest themselves if cert bootstrap was done
during package install.

Necessary Fixes:
----------------

Both the boostrap script and the Makefile need to add this export:

export RANDFILE=random

You might ask why it's necessary to also add it to the Makefile
because the Makefile inherits the environment from the bootstrap
script. But it should be perfectly O.K. to just run make directly,
thus it needs to be defined in both places.

After the certs are generated the newly created files need their
ownship and permissions fixed. We use a uid:gid of radiusd:radiusd for
the server and depend on group permission (gid=radiusd) for files
owned by root. Thus after cert generation the following must be done
(but this is specific to our installation)

chmod 0640 random
chown root:radiusd *

FWIW, In our case the file ownership and permissions only needed to be
fixed if the cert genration was performed manually because when it was
done automatically during RPM install it was done like this:

   if [ ! -e /etc/raddb/certs/server.pem ]; then
     /sbin/runuser -g radiusd -c 'umask 007; /etc/raddb/certs/bootstrap' 
 > /dev/null 2>&1
   fi

Which resulted in the right ownship and permissions (modulo the newly
introduced $RANDFILE which now needs it's permission adjusted)

HTH,

John

-- 
John Dennis <jdennis at redhat.com>

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/