Hello, I've been looking at free radius wiki <http://wiki.freeradius.org/Rlm_perl>, and read all about interpreter pools. the prerequisites are that perl is complied with USE_ITHREADS and a bit lower also with MULTIPLICITY. I have both. what I can't seem to get working is the thread management as: /#max_clones = 32/ /#start_clones = 32/ /#min_spare_clones = 0/ /#max_spare_clones = 32/ /#cleanup_delay = 5/ /#max_request_per_clone = 0/ Next stage was to look at the source, but I can't find any reference to any of the above! Is this still to come? Kind regards Simon Eseye Ltd ESEYE Ltd. <http://www.eseye.com> Simon Earthrowl <mailto:searthrowl@eseye.com> Surrey Technology Centre, 40 Occam Way, Guildford, Surrey, GU2 7YG Company No: 06397669 Telephone: +44 (0)1483 685200, Cellular: +44 (0)7775 777672 Skype: simonearthrowl <skype:simonearthrowl?chat> *Winner: * */2009 DTI Digital Communications Knowledge Network Competition/* *Winner: * */2009 SetSquared Investment Showcase/* *Have you seen Eseye's Cool-Tel <http://www.youtube.com/watch?v=haqWJNNkj2c> product on * <http://www.youtube.com/watch?v=haqWJNNkj2c> This message (including any attachments) is confidential and may be legally privileged. If you are not the intended recipient, you should not disclose, copy or use any part of it - please delete all copies immediately and notify Eseye on +44 (0)1483 685200. Any statements, opinions or information in this message are provided by the author, not on behalf of Eseye, unless subsequently confirmed by an individual who is authorised to represent Eseye. This email has been scanned for viruses; however, it is recommended that the addressee also check attachments. Eseye does not accept responsibility for any virus that is contracted via this email Follow us on Twitter at EseyeM2M Surface mount embedded SIMs in stock - adapter kit available for testing in a SIM socket "Smart Metering Technology of the Year Award 2012- Shortlisted' Eseye Ltd , Company No: 06397669 - Surrey Technology Centre, Guildford, UK +44 1483 685200
Hi,
I've been looking at [1]free radius wiki, and read all about interpreter pools. the prerequisites are that perl is complied with USE_ITHREADS and a bit lower also with MULTIPLICITY. I have both. what I can't seem to get working is the thread management as:
what version of FR ? in older versions you had to care about the PERL clones entry - to make sure it matched the parent process counts.. (learnt that the hard way) - but with newer releases all of this is dealt with for you alan
FR 2.1.12-1 and 2.1.12-7, but also looked at 3.0.0. I manage to get 2048 perl threads (assumed from /netstat -ap/ - as I'm interfacing into Apache's ActiveMQ), then when I hit 2049 threads FR crashes. Simon On 02/13/2012 11:25 PM, Alan Buxey wrote:
Hi,
I've been looking at [1]free radius wiki, and read all about interpreter pools. the prerequisites are that perl is complied with USE_ITHREADS and a bit lower also with MULTIPLICITY. I have both. what I can't seem to get working is the thread management as:
what version of FR ? in older versions you had to care about the PERL clones entry - to make sure it matched the parent process counts.. (learnt that the hard way) - but with newer releases all of this is dealt with for you
alan - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Follow us on Twitter at EseyeM2M Surface mount embedded SIMs in stock - adapter kit available for testing in a SIM socket "Smart Metering Technology of the Year Award 2012- Shortlisted' Eseye Ltd , Company No: 06397669 - Surrey Technology Centre, Guildford, UK +44 1483 685200
On Tue, Feb 14, 2012 at 6:32 AM, Simon Earthrowl <searthrowl@eseye.com> wrote:
FR 2.1.12-1 and 2.1.12-7, but also looked at 3.0.0. I manage to get 2048 perl threads (assumed from netstat -ap - as I'm interfacing into Apache's ActiveMQ), then when I hit 2049 threads FR crashes.
Why would you want that many threads? Can your activemq backend REALLY process > 2048 concurrent active connections without slowing down? It might be simply that nobody has tested that many perl threads in FR, so if a lower number works then I suggest you stick with that for now.
From my experience with sql, the optimum number of radius threads is equal to the number of backend connections (in my case, sql). And the optimum number of sql connections depends on several things, including processing delay (e.g. network roundtrip between nodes) and available cpu cores. But even then my optimum number is around 200. If a radius request arrive when all the 200-threads are busy, it will simply be queued for later processing.
-- Fajar
Simon Earthrowl wrote:
FR 2.1.12-1 and 2.1.12-7, but also looked at 3.0.0. I manage to get 2048 perl threads (assumed from /netstat -ap/ - as I'm interfacing into Apache's ActiveMQ), then when I hit 2049 threads FR crashes.
See doc/bugs for how to deal with crashes. And 2K threads? There's something wrong with your architecture if you need that. Your backend is VERY slow, or your load is too high, or you've under-provisioned your machines. The biggest mistake is that you're talking about solutions, not problems. Using 2K threads is a solution. Since you haven't specified what the problem is, there may be OTHER solutions which work better. e.g. If you're doing something crazy like using 2K threads, the usual response is "don't do that" Alan DeKok.
Hardware: based on ESX host: 4 core 2.1GHz processor (have 24 cores to play with) 8GB Memory (have more as needed) CentOS 5.7 x84_64 (latest patches) MySQL 5.5.20 - Same VM and supplied by Oracle FreeRadius 2.1.12-7 - complied here, with MySQL libraries for version 5.5.20 Limitations so far: 4 million dial-in potential users (16 million gets a bit slow - so looking for other improvements) In bound transaction rate (sustained mix of 1:1.5 of radius authentication:Radius accounting) 2048 Response time (so far and improving) < 500mS (current gains are from reworking MySQL data tables, structures, and indexes) CPU utilisation is still low (as reported by VSphere) ~15% ie MySQL is running well, and so is FR. Packet loss increases to 10% 2.5K transactions/sec. I am looking to use activeMQ to relieve some of the 3ggp (3ggp-Location etc.), add enrichment; and then ultimately manage quotas with overuse getting PoD. The activeMQ is on another host (utilisation of the primary network interface isn't that high; but will be looking to use a second interface, or even using the radius VM to host the activeMQ queue, with an additional VM running quota management. NB Quota is measured in credits, and other systems can charge credits - so there isn't a 'fixed' byte usage for radius accounting to count down. Also there is another system based on netflow managing nearer real time (well 5 minutes) actual data usage. I've wanted to limit the number of threads, as if the activeMQ server fails, I don't want radius to fail (users shouldn't be penalised because of poor systems management/setup). It's all a bit too open ended for me to feel comfortable with this as a solution as it stands. I'll be testing a local activeMQ server later today, with a view to either give it up as a bad idea, or to find some other way (postath database processing?). My feeling is that I've yet to unleash the real power of FR; but it's far from obvious to me, as to how to improve MySQL performance with FR. Reading others: dumping MySQL (albeit in a MySQL configuration -> local file + reload) way seems the next step. Simon On 02/14/2012 07:20 AM, Alan DeKok wrote:
Simon Earthrowl wrote:
FR 2.1.12-1 and 2.1.12-7, but also looked at 3.0.0. I manage to get 2048 perl threads (assumed from /netstat -ap/ - as I'm interfacing into Apache's ActiveMQ), then when I hit 2049 threads FR crashes. See doc/bugs for how to deal with crashes.
And 2K threads? There's something wrong with your architecture if you need that. Your backend is VERY slow, or your load is too high, or you've under-provisioned your machines.
The biggest mistake is that you're talking about solutions, not problems. Using 2K threads is a solution. Since you haven't specified what the problem is, there may be OTHER solutions which work better.
e.g. If you're doing something crazy like using 2K threads, the usual response is "don't do that"
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Follow us on Twitter at EseyeM2M Surface mount embedded SIMs in stock - adapter kit available for testing in a SIM socket "Smart Metering Technology of the Year Award 2012- Shortlisted' Eseye Ltd , Company No: 06397669 - Surrey Technology Centre, Guildford, UK +44 1483 685200
On Tue, Feb 14, 2012 at 7:57 PM, Simon Earthrowl <searthrowl@eseye.com> wrote:
Hardware: based on ESX host: 4 core 2.1GHz processor (have 24 cores to play with) 8GB Memory (have more as needed)
err ... that's not really much these days.
Limitations so far: 4 million dial-in potential users (16 million gets a bit slow - so looking for other improvements)
It's way overkill for FR with files backend. But once you include any kind of db or external backend (e.g. mysql, perl, whatever) then those system can quickly becomes the bottleneck. In the case of mysql, the bottleneck is usually disk IOPS.
In bound transaction rate (sustained mix of 1:1.5 of radius authentication:Radius accounting) 2048
2048 per what? seconds?
Response time (so far and improving) < 500mS (current gains are from reworking MySQL data tables, structures, and indexes)
That'd still mean you have a bottleneck somewhere. IIRC on a simple FR-mysql setup, I got several thousand auth+acct/sec, and that's with a pretty low max thread count (the 200-something I mentioned earlier. It's low compared to yours).
CPU utilisation is still low (as reported by VSphere) ~15% ie MySQL is running well, and so is FR. Packet loss increases to 10% 2.5K transactions/sec.
is this during your load test? If yes, then there's no reason to use 2048 threads. Really. Just lower them.
I've wanted to limit the number of threads, as if the activeMQ server fails, I don't want radius to fail (users shouldn't be penalised because of poor systems management/setup). It's all a bit too open ended for me to feel comfortable with this as a solution as it stands.
It's kinda complicated. Short version is if you use your external system only for acct, then using something similar to sites-available/buffered-sql should do the trick (i.e. log to detail file first, process later). But if you also need it for auth, then it gets compicated. Possible (especially if you only consider the case when the external system is dead), but complicated (especially if you consider the case when the external system simply becomes too slow)
My feeling is that I've yet to unleash the real power of FR; but it's far
It's pretty flexible. The hardest part is defining your policies. For example: how would you want FR to detect if your external system fails? How long must it wait?
from obvious to me, as to how to improve MySQL performance with FR. Reading others: dumping MySQL (albeit in a MySQL configuration -> local file + reload) way seems the next step.
mysql can perform just fine on super-busy implementation, IF you have the skills of a competent dba (or have someone with that skill helping you). -- Fajar
Simon Earthrowl wrote:
Limitations so far: 4 million dial-in potential users (16 million gets a bit slow - so looking for other improvements)
You need to design your solution VERY carefully. Scaling to 4M users isn't the same as 1K or 100K users.
In bound transaction rate (sustained mix of 1:1.5 of radius authentication:Radius accounting) 2048 Response time (so far and improving) < 500mS (current gains are from reworking MySQL data tables, structures, and indexes)
Response times of 500ms is RIDICULOUSLY bad. You have a catastrophic design decision somewhere. Find it, fix it.
CPU utilisation is still low (as reported by VSphere) ~15% ie MySQL is running well, and so is FR. Packet loss increases to 10% 2.5K transactions/sec.
My smart phone can handle 2.5K RADIUS packets/s. It's the REST of the system that needs fixing.
I am looking to use activeMQ to relieve some of the 3ggp (3ggp-Location etc.), add enrichment; and then ultimately manage quotas with overuse getting PoD.
You're again talking about a solution. Is using activeMQ a *requirement*? Or is it a potential solution?
I'll be testing a local activeMQ server later today, with a view to either give it up as a bad idea, or to find some other way (postath database processing?).
To do... what? Find out what you need to do per request. Make it as cheap as possible. Do as little work as possible per request. Post-process as much as possible.
My feeling is that I've yet to unleash the real power of FR; but it's far from obvious to me, as to how to improve MySQL performance with FR. Reading others: dumping MySQL (albeit in a MySQL configuration -> local file + reload) way seems the next step.
For 4M users? Not a good idea. I've built multiple systems with many-millions of users. It requires a bit of care, but it's possible. Alan DeKok.
Simon Earthrowl wrote:
Hello, I've been looking at free radius wiki <http://wiki.freeradius.org/Rlm_perl>, and read all about interpreter pools. the prerequisites are that perl is complied with USE_ITHREADS and a bit lower also with MULTIPLICITY. I have both. what I can't seem to get working is the thread management as:
That hasn't worked for MANY years. I've deleted that nonsense from the Wiki. Alan DeKok.
participants (4)
-
Alan Buxey -
Alan DeKok -
Fajar A. Nugraha -
Simon Earthrowl