freeradius high cpu load
Hello, iam using freeradius version 3.0.15 with rlm_redis and rlm_perl. on this server i have 4 VCPU, when i start a becnh test with 6000 accounting start / sec i see that all my 4 VCPU are used 97% for every vcpu any advise please to minimize the use of vcpu on freeradius? Best Regards Mettichi Bassem
On 18 Jan 2018, at 14:08, Bassem Mettichi <mettichi@gmail.com> wrote:
on this server i have 4 VCPU, when i start a becnh test with 6000 accounting start / sec i see that all my 4 VCPU are used 97% for every vcpu any advise please to minimize the use of vcpu on freeradius?
What have you configured FreeRADIUS to do with those 6000 packets/second? Regards, Adam Bishop gpg: E75B 1F92 6407 DFDF 9F1C BF10 C993 2504 6609 D460 jisc.ac.uk Jisc is a registered charity (number 1149740) and a company limited by guarantee which is registered in England under Company No. 5747339, VAT No. GB 197 0632 86. Jisc’s registered office is: One Castlepark, Tower Hill, Bristol, BS2 0JA. T 0203 697 5800. Jisc Services Limited is a wholly owned Jisc subsidiary and a company limited by guarantee which is registered in England under company number 2881024, VAT number GB 197 0632 86. The registered office is: One Castle Park, Tower Hill, Bristol BS2 0JA. T 0203 697 5800.
On Jan 18, 2018, at 9:08 AM, Bassem Mettichi <mettichi@gmail.com> wrote:
iam using freeradius version 3.0.15 with rlm_redis and rlm_perl.
Perl is slow. Very, very, slow.
on this server i have 4 VCPU, when i start a becnh test with 6000 accounting start / sec i see that all my 4 VCPU are used 97% for every vcpu any advise please to minimize the use of vcpu on freeradius?
a) send it fewer packets per second b) make it do less work per-packet Those are your two choices. So I'll echo Adam's comment.. what are you doing with those packets? As an explanation, if you put user entries into the "users" file, the server will happily authentication 20K packets/s on a single CPU. So any *higher* CPU usage is something you did to your local configuration. Alan DeKok.
Hello, thanks a lot for your response, we need these number of accounting start because our Customer is a telecom operator so weed more than 6000 accounting start / second but just ia clarification: for accounting start i don't use rlm_perl iam suing rediswho module so why i have this higher CPU usage? could you give me an explanantion for this behavior with freeradius? Best Regards Mettichi Bassem 2018-01-18 15:48 GMT+01:00 Alan DeKok <aland@deployingradius.com>:
On Jan 18, 2018, at 9:08 AM, Bassem Mettichi <mettichi@gmail.com> wrote:
iam using freeradius version 3.0.15 with rlm_redis and rlm_perl.
Perl is slow. Very, very, slow.
on this server i have 4 VCPU, when i start a becnh test with 6000 accounting start / sec i see that all my 4 VCPU are used 97% for every vcpu any advise please to minimize the use of vcpu on freeradius?
a) send it fewer packets per second
b) make it do less work per-packet
Those are your two choices.
So I'll echo Adam's comment.. what are you doing with those packets?
As an explanation, if you put user entries into the "users" file, the server will happily authentication 20K packets/s on a single CPU.
So any *higher* CPU usage is something you did to your local configuration.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/ list/users.html
On Jan 18, 2018, at 10:00 AM, Bassem Mettichi <mettichi@gmail.com> wrote:
thanks a lot for your response, we need these number of accounting start because our Customer is a telecom operator so weed more than 6000 accounting start / second
Which means you should know how to build systems for telecom operators. Your messages so far don't give me confidence that this is the case.
but just ia clarification: for accounting start i don't use rlm_perl iam suing rediswho module so why i have this higher CPU usage? could you give me an explanantion for this behavior with freeradius?
No. You have refused to answer any questions we have about the system. Since we don't know what you're doing, we don't know what's going wrong, and we can't help you. Alan DeKok.
Hello Alan, i don't understand why your answers are always like this, if you don't like to give a hand you don't need to write me any thing you didn't answered any question i have posted on the this list always you blame me i don't like why, you have to trusted me, iam using Redhat 7.1, on this machine i have 4VCPU + 8GB RAM, iam using freeradius with rediswho for accounting and freeradius with rlm_perl for authentication, iam doing a performnce tests because the Customer has a lot of trafic that is why i have stopped using MYSQL and iam using now redis database, the Customer need 10000 accounting start / seconde but i have now reached only 6000 accounting start / second because of the high cpu use of radiusd process on the system when i execute top command i see only radiusd running and used the totality of the 4 VCPU please if you can't help me don't tell any thing, i don't need for your blame. Best Regards Mettichi Bassem 2018-01-18 16:11 GMT+01:00 Alan DeKok <aland@deployingradius.com>:
On Jan 18, 2018, at 10:00 AM, Bassem Mettichi <mettichi@gmail.com> wrote:
thanks a lot for your response, we need these number of accounting start because our Customer is a telecom operator so weed more than 6000 accounting start / second
Which means you should know how to build systems for telecom operators.
Your messages so far don't give me confidence that this is the case.
but just ia clarification: for accounting start i don't use rlm_perl iam suing rediswho module so why i have this higher CPU usage? could you give me an explanantion for this behavior with freeradius?
No.
You have refused to answer any questions we have about the system. Since we don't know what you're doing, we don't know what's going wrong, and we can't help you.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/ list/users.html
Bassem, Thanks for writing into the mailing list (ML) for help. While I mostly lurk on this list, the notion that Alan isn't helpful is wildly wrong. Just because his answers are succinct does not mean they are rude. And to be honest, this is *your* system and *your* design isn't working. The quality of your questions to the ML and the quality of the answers you provide to the ML will be directly proportional to the type/amount/quality of help you receive. For instance, "top" posting to his reply while he (and the rest of the FR-users ML) uses "inline". This is slightly disrespectful to those whose help you are requesting. I'll continue replying inline... On Thu, Jan 18, 2018 at 10:34 AM, Bassem Mettichi <mettichi@gmail.com> wrote:
Hello Alan,
i don't understand why your answers are always like this, if you don't like to give a hand you don't need to write me any thing you didn't answered any question i have posted on the this list always you blame me i don't like why, you have to trusted me, iam using Redhat 7.1, on this machine i have 4VCPU + 8GB RAM, iam using freeradius with rediswho for accounting and freeradius with rlm_perl for authentication,
Alan already stated that perl is slow. If you are going to insist on using perl, then you need to do some analysis on if you can be speed up what perl is doing. There was already advice on this: "b) make it do less work per-packet"
iam doing a performnce tests because the Customer has a lot of trafic that is why i have stopped using MYSQL and iam using now redis database, the Customer need 10000 accounting start / seconde but i have now reached only 6000 accounting start / second because of the high cpu
Is the CPU usage due to perl above? Is it starving the CPU for redis?
use of radiusd process on the system when i execute top
top isn't going to be the best way to measure where the CPU is spending its time.
command i see only radiusd running and used the totality of the 4 VCPU please if you can't help me don't tell any thing, i don't need for your blame.
For the sake of my own curiosity, does anyone know the throughput for FR dumping accounting packets to disk? Respectfully, -m
On Jan 18, 2018, at 11:55 AM, Matt Zagrabelny <mzagrabe@d.umn.edu> wrote:
For the sake of my own curiosity, does anyone know the throughput for FR dumping accounting packets to disk?
To the detail file? 1000's of packets per second. It doesn't do fsync, so it should as fast as memory / disk IO allows. But, it all depends on the local configuration. OS, disk type, etc. It can vary widely. We've done 10K+ writes per second to Redis. I don't imagine that plain disk IO is faster than that. So slower. but perhaps not by a factor of 10. When you have millions of users and 10K packets/s, you need high availability, fail-over, clusters of servers, load balancers. A "naive" install of "install FR and DB" is fine for 10K users, and 100 packets. It is entirely the wrong thing to do for 10K packets/s. My final $0.02 is that it's not really nice to (a) charge a customer for services you're incapable of providing, and (b) ask *us* to support you for free, and then (c) complain about the free support. Alan DeKok.
so...what are these VCPU - what sort of horsepower are you dealing with? have you measured the actual ability of your REDIS sub-system - how many requests can it deal with directly? what CPU load do you get when doing that? have you profiled the radiusd process and seen what exactly is taking the CPU time when packets are going through your system? are things stuck in WAIT etc? alan On 18 January 2018 at 17:13, Alan DeKok <aland@deployingradius.com> wrote:
On Jan 18, 2018, at 11:55 AM, Matt Zagrabelny <mzagrabe@d.umn.edu> wrote:
For the sake of my own curiosity, does anyone know the throughput for FR dumping accounting packets to disk?
To the detail file? 1000's of packets per second. It doesn't do fsync, so it should as fast as memory / disk IO allows.
But, it all depends on the local configuration. OS, disk type, etc. It can vary widely.
We've done 10K+ writes per second to Redis. I don't imagine that plain disk IO is faster than that. So slower. but perhaps not by a factor of 10.
When you have millions of users and 10K packets/s, you need high availability, fail-over, clusters of servers, load balancers.
A "naive" install of "install FR and DB" is fine for 10K users, and 100 packets. It is entirely the wrong thing to do for 10K packets/s.
My final $0.02 is that it's not really nice to (a) charge a customer for services you're incapable of providing, and (b) ask *us* to support you for free, and then (c) complain about the free support.
Alan DeKok.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On Jan 18, 2018, at 11:34 AM, Bassem Mettichi <mettichi@gmail.com> wrote:
i don't understand why your answers are always like this,
Clearly. You ask vague questions. We ask for more information, and you refuse. How, exactly, do you expect us to help you?
if you don't like to give a hand you don't need to write me any thing you didn't answered any question i have posted on the this list always you blame me
Yes. You don't answer questions we ask. You don't follow instructions. So it's simple. Do you want to fix the problem? Or do you want to ask questions, and then ignore everything we tell you?
i don't like why, you have to trusted me, iam using Redhat 7.1, on this machine i have 4VCPU + 8GB RAM, iam using freeradius with rediswho for accounting and freeradius with rlm_perl for authentication,
You keep saying that. I keep asking you for more information. You *always* refuse to give more information. Do you understand that I'm asking questions for a *reason*? That I really, truly, do need more information in order to help you? That I'm *trying* to help you? And that you insist on making it as hard as possible for me to help you?
iam doing a performnce tests because the Customer has a lot of trafic that is why i have stopped using MYSQL and iam using now redis database, the Customer need 10000 accounting start / seconde
It's not trivial to create such a system. If you don't know how to build a system which does 10K packets per second, then you shouldn't promise people that you can build one.
but i have now reached only 6000 accounting start / second because of the high cpu use of radiusd process on the system when i execute top command i see only radiusd running and used the totality of the 4 VCPU please
Which means you're looking at *very* high level information. And (like on this list) refusing to look at any more *detailed* information which can help you solve the problem.
if you can't help me don't tell any thing, i don't need for your blame.
Yes, you do. It's *your* fault that you refuse to answer any questions I ask. It's simple: either answer the questions we ask, or stop posting on the list. We ask questions because we need more information. If you refuse to give us information, you are making it impossible for us to help you. And, if you complain that I'm being mean and unhelpful, you will be unsubscribed from the list, and permanently banned from it. This list is for people who want to solve problems. People who post the same question over and over and over, and then ignore all attempts to help them are NOT welcome here. Alan DeKok.
participants (5)
-
Adam Bishop -
Alan Buxey -
Alan DeKok -
Bassem Mettichi -
Matt Zagrabelny