I have an application where I would like to read and process rlm_detail records in near-real-time (specifically to turn them into JSON and stuff them into a local database). I'm familiar with rlm_sql_log + radsqlrelay, and I know this is robust. So I was hoping to modify radsqlrelay to work with files being appended to by rlm_detail, but I turned up a wrinkle when checking the file locking in 2.2.x code. rlm_sql_log uses fcntl() locking only; but rlm_detail is different. It calls rad_lockfd_nonblock(), which uses lockf if available, else flock, and only uses fcntl if the other two are not available. It looks like fcntl and flock locks are independent of each other: http://www.mjmwired.net/kernel/Documentation/filesystems/locks.txt Question: what's the safe way to process detail files? Is there a Perl implementation of radrelay I can hack from? Looks like the original C implementation of radrelay was removed in 2005 (acd40e2e) I am aware of raddb/sites-available/buffered-sql, but that only shows how to get freeradius to read a detail file, not a standalone program. Thanks in advance, Brian.
On 07/02/13 15:10, Brian Candler wrote:
I have an application where I would like to read and process rlm_detail records in near-real-time (specifically to turn them into JSON and stuff them into a local database).
I'm familiar with rlm_sql_log + radsqlrelay, and I know this is robust.
So I was hoping to modify radsqlrelay to work with files being appended to by rlm_detail, but I turned up a wrinkle when checking the file locking in 2.2.x code.
rlm_sql_log uses fcntl() locking only; but rlm_detail is different. It calls rad_lockfd_nonblock(), which uses lockf if available, else flock, and only uses fcntl if the other two are not available.
Yes. This was changed in "master".
It looks like fcntl and flock locks are independent of each other: http://www.mjmwired.net/kernel/Documentation/filesystems/locks.txt
Yes. But also be aware that lockf and fcntl are not guaranteed to be equivalent either - POSIX in general specifies no relationship. They happen to be the same under Linux, right now.
Question: what's the safe way to process detail files? Is there a Perl implementation of radrelay I can hack from? Looks like the original C implementation of radrelay was removed in 2005 (acd40e2e)
Personally, I would just backport the "fcntl-only" commit - that way you know you're getting fcntl locking: https://github.com/FreeRADIUS/freeradius-server/commit/582852042b4aa6810a683...
Personally, I would just backport the "fcntl-only" commit - that way you know you're getting fcntl locking:
That looks perfect, thank you. I had another thought: would it be sane to get rlm_detail to write to a named pipe? I guess the problem then is it could block if the listener goes away. You could use a second freeradius instance as file reader / pipe writer. But reading the file directly and using fcntl locking sounds a much better solution anyway. Cheers, Brian.
participants (2)
-
Brian Candler -
Phil Mayers