Hello, I just subscribed so I won't be able to quote properly but I hope at least the message is associated with the right thread (I found it on the web archive of the mailing list). I've been using FreeRADIUS for about 4 year now and it is a wonderful product - there's no question why you're No 1! For the past years of using and developing rlm_perl modules for FreeRADIUS, I've only been astonished and happy because of the new functionalities etc that the development team have been adding to the new releases. This all ended with this crippling
Were you also depending on the rlm_perl threads to make connections to multiple DBs? I know that I can make an array of db handlers within one perl thread and use them interchangeably, but the functionality I had in the 2.0.x release where every perl thread had its own connection to the DBs and the queries were executed simultaneously was much better. I am getting the feeling that I am loosing in parallelism. I understand that there may some benefits in the current implementation (2.1.x) such as less threads, smaller memory footprint etc. but why change something that has been tested and working in the first place? Borislav Dimitrov wrote:
Hello,
I just subscribed so I won't be able to quote properly but I hope at least the message is associated with the right thread (I found it on the web archive of the mailing list). I've been using FreeRADIUS for about 4 year now and it is a wonderful product - there's no question why you're No 1! For the past years of using and developing rlm_perl modules for FreeRADIUS, I've only been astonished and happy because of the new functionalities etc that the development team have been adding to the new releases. This all ended with this crippling- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr -------------------------------------------
Yes this was the main reason but there are others too. С поздрави Борислав Димитров e-mail: b.dimitrov@ngsystems.net GSM: 0889 28 54 57 NG Systems Лавеле 32, ет: 4, София, България On 15.04.2009, at 11:06, Apostolos Pantsiopoulos wrote:
Were you also depending on the rlm_perl threads to make connections to multiple DBs? I know that I can make an array of db handlers within one perl thread and use them interchangeably, but the functionality I had in the 2.0.x release where every perl thread had its own connection to the DBs and the queries were executed simultaneously was much better. I am getting the feeling that I am loosing in parallelism.
I understand that there may some benefits in the current implementation (2.1.x) such as less threads, smaller memory footprint etc. but why change something that has been tested and working in the first place?
Borislav Dimitrov wrote:
Hello, I just subscribed so I won't be able to quote properly but I hope at least the message is associated with the right thread (I found it on the web archive of the mailing list). I've been using FreeRADIUS for about 4 year now and it is a wonderful product - there's no question why you're No 1! For the past years of using and developing rlm_perl modules for FreeRADIUS, I've only been astonished and happy because of the new functionalities etc that the development team have been adding to the new releases. This all ended with this crippling- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr ------------------------------------------- - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Apostolos Pantsiopoulos wrote:
I understand that there may some benefits in the current implementation (2.1.x) such as less threads, smaller memory footprint etc. but why change something that has been tested and working in the first place?
A quest to make it better. If we were satisfied with the functionality of the server in 1.0, we would have had no improvements since then. In any case, it looks like it may be easy to change it so that there is one Perl thread per server thread. Would you be prepared to test patches? Alan DeKok.
Yes, that would be great. One perl interpreter per freeradius thread, that is. And I suppose the CLONE function would work again as expected (i.e. each freeradius thread would have its own perl interpreter and each script relaying on this interpreter would have its own connection to the DB). And the perl clones will not be controlled by the perl.conf (as in 2.0.x) but from the max_servers directive in radiusd.conf, right? I am ready for testing whenever you have a patch ready. Alan DeKok wrote:
Apostolos Pantsiopoulos wrote:
I understand that there may some benefits in the current implementation (2.1.x) such as less threads, smaller memory footprint etc. but why change something that has been tested and working in the first place?
A quest to make it better. If we were satisfied with the functionality of the server in 1.0, we would have had no improvements since then.
In any case, it looks like it may be easy to change it so that there is one Perl thread per server thread. Would you be prepared to test patches?
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr -------------------------------------------
From my point of view we should have pool of perl clones per each module instance. This way we could have multiple perl instances (each with its own perl script to run). Limiting on perl clone or interp per server thread will limit the multiple instances feature of rlm_perl. Again playing with min and max spare can give us some possibility's to force not unload perl interpreter during the lifetime of server and this way we can keep some DB handlers not to reconnect each time. Alan what is your point ? Best Regards, Boian Jordanov SNE Orbitel - Next Generation Telecom tel. +359 2 4004 723 tel. +359 2 4004 002 On Apr 16, 2009, at 2:38 PM, Apostolos Pantsiopoulos wrote:
Yes, that would be great. One perl interpreter per freeradius thread, that is. And I suppose the CLONE function would work again as expected (i.e. each freeradius thread would have its own perl interpreter and each script relaying on this interpreter would have its own connection to the DB). And the perl clones will not be controlled by the perl.conf (as in 2.0.x) but from the max_servers directive in radiusd.conf, right?
I am ready for testing whenever you have a patch ready.
Alan DeKok wrote:
I understand that there may some benefits in the current implementation (2.1.x) such as less threads, smaller memory footprint etc. but why change something that has been tested and working in the first place? A quest to make it better. If we were satisfied with the functionality of the server in 1.0, we would have had no improvements since then. In any case, it looks like it may be easy to change it so that there is one Perl thread per server thread. Would you be prepared to test patches? Alan DeKok.
Apostolos Pantsiopoulos wrote: - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr ------------------------------------------- - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Boian Jordanov wrote:
From my point of view we should have pool of perl clones per each module instance.
Yes.
This way we could have multiple perl instances (each with its own perl script to run).
Yes.
Limiting on perl clone or interp per server thread will limit the multiple instances feature of rlm_perl.
We don't need that limit. The server should not be running more Perl threads than system threads. It also should not be running less Perl threads than system threads. It should be running one Perl thread per system thread. The server core already manages min/max spare threads, idle threads, etc.
Again playing with min and max spare can give us some possibility's to force not unload perl interpreter during the lifetime of server and this way we can keep some DB handlers not to reconnect each time.
Alan what is your point ?
The pthread keys in the current rlm_perl should be moved to the "perl_inst" struct. The keys should be allocated per thread, and not via pthread_once. Alan DeKok.
Alan DeKok wrote:
Boian Jordanov wrote:
From my point of view we should have pool of perl clones per each module instance.
Yes.
This way we could have multiple perl instances (each with its own perl script to run).
Yes.
Limiting on perl clone or interp per server thread will limit the multiple instances feature of rlm_perl.
We don't need that limit. The server should not be running more Perl threads than system threads. It also should not be running less Perl threads than system threads.
My point exactly.
It should be running one Perl thread per system thread. The server core already manages min/max spare threads, idle threads, etc.
I totally agree. In the old config I used to have the same clone= and max_servers= directives to achieve that.
Again playing with min and max spare can give us some possibility's to force not unload perl interpreter during the lifetime of server and this way we can keep some DB handlers not to reconnect each time.
Alan what is your point ?
The pthread keys in the current rlm_perl should be moved to the "perl_inst" struct. The keys should be allocated per thread, and not via pthread_once.
I hope this implementation will satisfy Borislav too. Will he be able to instantiate different perl scripts for different needs? So, when do I start testing :)
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr -------------------------------------------
I hope this implementation will satisfy Borislav too. Will he be able to instantiate different perl scripts for different needs?
So, when do I start testing :)
Hi, Yes, being able to instantiate and use several rlm_perl instances with different scripts to take care of different circumstances is what will make me and many others (I think) happy. Sacrificing the *_clones flexibility for lower memory footprint, better performance and more importantly code is certainly worth doing it, if people are still able to have multiple rlm_perl instances. I imagine that probably the best way will be to have X (the number of rlm_perl instances) per system thread - this is the way it'd be if they were different modules (like sql, preprocess etc) which custom Perl scripts executing under rlm_perl a kind of are... For now I downgraded to 2.0.5 which works perfect for me but will be happy to help with testing (on some client's production system... don't tell anyone ;-) ). OFFTOPIC: Btw, do you know of some existing effort to develop rlm_ruby? What's its state etc? I had the ambition to develop something like that myself but don't have the time anymore :-(. On 16.04.2009, at 20:17, Apostolos Pantsiopoulos wrote:
Alan DeKok wrote:
Boian Jordanov wrote:
From my point of view we should have pool of perl clones per each module instance. Yes. This way we could have multiple perl instances (each with its own perl script to run). Yes. Limiting on perl clone or interp per server thread will limit the multiple instances feature of rlm_perl. We don't need that limit. The server should not be running more Perl threads than system threads. It also should not be running less Perl threads than system threads.
My point exactly.
It should be running one Perl thread per system thread. The server core already manages min/max spare threads, idle threads, etc.
I totally agree. In the old config I used to have the same clone= and max_servers= directives to achieve that.
Again playing with min and max spare can give us some possibility's to force not unload perl interpreter during the lifetime of server and this way we can keep some DB handlers not to reconnect each time.
Alan what is your point ? The pthread keys in the current rlm_perl should be moved to the "perl_inst" struct. The keys should be allocated per thread, and not via pthread_once.
I hope this implementation will satisfy Borislav too. Will he be able to instantiate different perl scripts for different needs?
So, when do I start testing :)
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr ------------------------------------------- - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Borislav Dimitrov wrote:
Sacrificing the *_clones flexibility for lower memory footprint, better performance and more importantly code is certainly worth doing it, if people are still able to have multiple rlm_perl instances.
If we update the module to have one Perl thread per system thread, it will still have the clone functionality.
I imagine that probably the best way will be to have X (the number of rlm_perl instances) per system thread - this is the way it'd be if they were different modules (like sql, preprocess etc) which custom Perl scripts executing under rlm_perl a kind of are... For now I downgraded to 2.0.5 which works perfect for me but will be happy to help with testing (on some client's production system... don't tell anyone ;-) ).
Or, grab the rlm_perl source from 2.1.x, and use it in the latest version. I don't think that there are any incompatibilities.
OFFTOPIC: Btw, do you know of some existing effort to develop rlm_ruby? What's its state etc? I had the ambition to develop something like that myself but don't have the time anymore :-(.
http://github.com/Antti/rlm_ruby/tree/master If he can fork the git tree, and add the rlm_ruby module to it, I can pull the changes into the main server. Alan DeKok.
It should be running one Perl thread per system thread. The server core already manages min/max spare threads, idle threads, etc. <snip> I hope this implementation will satisfy Borislav too. Will he be able to instantiate different perl scripts for different needs?
So, when do I start testing :)
Just to say, we're currently using FreeRadius 2.1.3 with rlm_perl in a project currently in active development, and having read this i'm holding off upgrading to 2.1.4 as we also use the multiple perl threads functionality of the module to parallel process and increase throughput. We have a specific development server/environment, and would be more than happy to test any patches designed to fix this 2.1.4 issue on it. -- Dan Meyers Network Specialist, Lancaster University E-Mail: d.meyers@lancaster.ac.uk
If any changes are to be made to the current implementation to support multiple interpreters (one per thread) would they show up in a 2.1.x release or a future one (2.2.x or something)? Meyers, Dan wrote:
It should be running one Perl thread per system thread. The server core already manages min/max spare threads, idle threads, etc. <snip> I hope this implementation will satisfy Borislav too. Will he be able to instantiate different perl scripts for different needs?
So, when do I start testing :)
Just to say, we're currently using FreeRadius 2.1.3 with rlm_perl in a project currently in active development, and having read this i'm holding off upgrading to 2.1.4 as we also use the multiple perl threads functionality of the module to parallel process and increase throughput. We have a specific development server/environment, and would be more than happy to test any patches designed to fix this 2.1.4 issue on it.
-- Dan Meyers Network Specialist, Lancaster University E-Mail: d.meyers@lancaster.ac.uk
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr -------------------------------------------
Apostolos Pantsiopoulos wrote:
If any changes are to be made to the current implementation to support multiple interpreters (one per thread) would they show up in a 2.1.x release or a future one (2.2.x or something)?
They will show up in the next release, whatever that is. i.e. "next after the changes have been made". Alan DeKok.
On 22.04.2009, at 13:23, Alan DeKok wrote:
Apostolos Pantsiopoulos wrote:
If any changes are to be made to the current implementation to support multiple interpreters (one per thread) would they show up in a 2.1.x release or a future one (2.2.x or something)?
They will show up in the next release, whatever that is.
i.e. "next after the changes have been made".
I suppose Apostolos Pantsiopoulos ment the multiple rlm_perl instances ( modules { perl inst1 { } perl inst2 { } } capability inside each thread, that we discussed a lot some time ago. Just to clarify... Anyways that's really cool ;-) and I'm looking forward to seeing it implemented. I'll do my best to help with testing.
On Apr 22, 2009, at 7:25 PM, Borislav Dimitrov wrote:
On 22.04.2009, at 13:23, Alan DeKok wrote:
Apostolos Pantsiopoulos wrote:
If any changes are to be made to the current implementation to support multiple interpreters (one per thread) would they show up in a 2.1.x release or a future one (2.2.x or something)?
They will show up in the next release, whatever that is.
i.e. "next after the changes have been made".
I suppose Apostolos Pantsiopoulos ment the multiple rlm_perl instances ( modules { perl inst1 { } perl inst2 { } } capability inside each thread, that we discussed a lot some time ago. Just to clarify... Anyways that's really cool ;-) and I'm looking forward to seeing it implemented. I'll do my best to help with testing.
If you look at my mail you can find that this is exactly what I said. Until new release you can just use an old module as Alan says.
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
On Wed, Apr 22, 2009 at 12:23 PM, Alan DeKok <aland@deployingradius.com> wrote:
Apostolos Pantsiopoulos wrote:
If any changes are to be made to the current implementation to support multiple interpreters (one per thread) would they show up in a 2.1.x release or a future one (2.2.x or something)?
They will show up in the next release, whatever that is.
i.e. "next after the changes have been made".
Have any changes been made already? :) -- Alexandr Kovalenko http://uafug.org.ua/
I think yes. In my current config (2.1.8) it works fine. -- ------------------------------------------- Apostolos Pantsiopoulos Kinetix Tele.com R & D email: regs@kinetix.gr ------------------------------------------- On 3/3/2010 4:26 μμ, Alexandr Kovalenko wrote:
On Wed, Apr 22, 2009 at 12:23 PM, Alan DeKok<aland@deployingradius.com> wrote:
Apostolos Pantsiopoulos wrote:
If any changes are to be made to the current implementation to support multiple interpreters (one per thread) would they show up in a 2.1.x release or a future one (2.2.x or something)?
They will show up in the next release, whatever that is.
i.e. "next after the changes have been made".
Have any changes been made already? :)
Borislav Dimitrov wrote:
I just subscribed so I won't be able to quote properly but I hope at least the message is associated with the right thread (I found it on the web archive of the mailing list). I've been using FreeRADIUS for about 4 year now and it is a wonderful product - there's no question why you're No 1! For the past years of using and developing rlm_perl modules for FreeRADIUS, I've only been astonished and happy because of the new functionalities etc that the development team have been adding to the new releases. This all ended with this crippling-
... of the rlm_perl module? The changes were made to simplify the code. Having *two* thread pool managers is inefficient, and unnecessary. Can you say *exactly* what your issue is with these changes? Alan DeKok.
Hi, The lack of the _clones options is not my (primary) problem but I think it was a good functionality. I understand that simplifying, refactoring and optimizing the code is important and that the changes done are probably for the better but being unable to instantiate several rlm_perl instances to take care of different circumstances based on the NAS-IP or something else is a huge problem for me. Otherwise, if we have a working radiusd thread pool and there's one perl interpreter per radiusd thread/process it's OK. The requests won't be waiting each other to be processed. But if there can be only one perl interpreter shared between the whole thread pool, there will be no performance benefits at all - just the opposite - resources won't be used effectively and there will be requests which will wait for minutes... I've experienced it. Anyways my main trouble is being unable to use multiple rlm_perl instances like this (I've put the comments to illustrate the flexibility of using *_clones which is now gone): perl instance1 { # This is a large branch => I put higher values for max_clones, start_clones etc but keeping the pool static to avoid reloading the settings from the DB on module instantiation } perl instance2 { # This is another (slightly modified) script which takes care of special circumstances and is connected to a different DB } # ... and now using the wonderful ulang functionality I do the following in ... say accounting switch "%{NAS-IP-Address}" { case '10.250.15.1' { instance1 } case '10.250.17.192' { instance2 } } This doesn't work with 2.1.4 although my perl is compiled with ithreads and multiplicity etc On 15.04.2009, at 11:11, Alan DeKok wrote:
Borislav Dimitrov wrote:
I just subscribed so I won't be able to quote properly but I hope at least the message is associated with the right thread (I found it on the web archive of the mailing list). I've been using FreeRADIUS for about 4 year now and it is a wonderful product - there's no question why you're No 1! For the past years of using and developing rlm_perl modules for FreeRADIUS, I've only been astonished and happy because of the new functionalities etc that the development team have been adding to the new releases. This all ended with this crippling-
... of the rlm_perl module?
The changes were made to simplify the code. Having *two* thread pool managers is inefficient, and unnecessary.
Can you say *exactly* what your issue is with these changes?
Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Borislav Dimitrov wrote:
Anyways my main trouble is being unable to use multiple rlm_perl instances like this (I've put the comments to illustrate the flexibility of using *_clones which is now gone):
Ah... OK. That was *not* the intent of the change. I'll take a look at fixing it for the next release. Alan DeKok.
Hi, On 15.04.2009, at 20:31, Alan DeKok wrote:
Borislav Dimitrov wrote:
Anyways my main trouble is being unable to use multiple rlm_perl instances like this (I've put the comments to illustrate the flexibility of using *_clones which is now gone):
Ah... OK. That was *not* the intent of the change. I'll take a look at fixing it for the next release.
Cool! This is very comforting. If removing the separate "thread" pool for the perl interpreters is done with the intend to sacrifice some flexibility in order to simplify, refactor and make the code more efficient (mainly the memory footprint) it's fine. And after all with the new behavior it will be possible to achieve parallelism with the radiusd's thread pool, instead of using *_clones. This is fine and I'm sure that it'll become one of the wonderful enhancements I've seen through the years (I've been a happy user for about 4 years now. Fantastic product btw!). But my problems is being unable to instantiate two or more instances with the well known syntax: module_name instance_name { } ... that is (in my /rlm_perl's/ case): perl inst1 { } perl inst2 { } ... and then - in say accounting - I fork the processing with a switch on the NAS-IP routing it to a given instance: accounting { # if cond inst1 else inst2 ...} # for example This is what I'm unable to do with the new 2.1.4 and it is a huge concern for me and our company, leaving us with the only option to downgrade since right now we don't have the time to modify our or rlm_perl's code. I tried several different ways to achieve the above like: using the instantiate {} section, putting files named inst1 and inst2 in the modules directory with the rlm_perl configurations in each one, putting it all together in radiusd.conf etc... and it didn't work. Please let me know if I'm doing something wrong but if it is a "bug", I'm starting to think that many users will be happy (people do different setups using this technique like using different DB handles in the instances to achieve load-balancing etc). Let the source be with you all and happy hacking ;-)
participants (6)
-
Alan DeKok -
Alexandr Kovalenko -
Apostolos Pantsiopoulos -
Boian Jordanov -
Borislav Dimitrov -
Meyers, Dan