<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Loius,<br>
<br>
Be aware, there are some major design flaws within the Cisco WLC
controller software (Georgia Tech is working with Cisco to work
through them) regarding the number of requests a controller can
field.<br>
<br>
<a class="moz-txt-link-freetext" href="https://tools.cisco.com/bugsearch/bug/CSCuj88508">https://tools.cisco.com/bugsearch/bug/CSCuj88508</a><br>
<br>
The flaws in the controller software cause an "overrun" of radiusIDs
if you have too many authentications/second which will manifest as
"duplicate" and "discards" in the logs. No amount of tweaking on the
radius side will fix this. You can however, improve performance to
try and improve the client experience.<br>
<br>
When we are talking about AD, Phil Mayers had some great suggestions
on improving ntlm_auth performance. Here were his recommendations:<br>
<br>
1. Upgraded the radius servers.
<br>
Old spec: 3Gb RAM, 2x P4-based Xeon 1 core @ 3.2GHz, RHEL5
<br>
New spec: 16Gb RAM, 1x Xeon E5-2620 6 core @ 2GHz, RHEL6
<br>
<br>
2. Upgraded Samba - went from RHEL5 samba3x-3.5.4 to RHEL6
samba-3.6.9
<br>
<br>
3. Set "winbind max domain connections = 12" in smb.conf
(restart winbind) (we at GT actually have so many authentications,
we set to 128 as we reached our limit during peak times)<br>
<br>
4. Forced our smb.conf to talk to specific AD controllers which are
physical, not VMWare (most our DCs are VMWare)
<br>
<br>
5. Spent a <b class="moz-txt-star"><span class="moz-txt-tag">*</span>lot<span
class="moz-txt-tag">*</span></b> of time debugging and tracking
the Samba->DC RPC round-trip times and hassling our AD people to
keep these stable; not sure what they did, if anything.
<br>
<br>
6. Increased radiusd.conf setting to "max_requests = 16384"
<br>
<br>
7. Worked really, really hard on getting the Cisco APs, AP radios
and controllers to STOP CRASHING; their software quality has been
abysmal, and this was a contributing factor - APs or controllers
would crash under load, and this would trigger a burst of auths,
which would trigger the problem.
<br>
<br>
As Alan said before, there are lots of moving parts where issues can
happen. If you improve server performance within the pieces
(AD/database/winbind/etc), that's a start. <br>
<br>
If you are in a large scale Cisco deployment, depending on how many
APs and users, you may find yourself having issues regardless. It's
a hard problem to advise on, but adding additional radius servers
and optimizing ours for performance has helped us immensely.<br>
<br>
- JohnD<br>
<br>
<div class="moz-cite-prefix">On 09/19/2014 02:58 PM, Louis Munro
wrote:<br>
</div>
<blockquote
cite="mid:1DC33912-2080-4DBE-9165-2F5AEC30957C@inverse.ca"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
Hello,
<div><br>
</div>
<div>While troubleshooting a system I came upon a case of 'hung
children' and duplicate requests. </div>
<div>I would usually ascribe this to a database issue but in this
case the database is mostly unused and properly indexed. </div>
<div>Accounting is not used, so that's one less thing to consider.</div>
<div><br>
</div>
<div>On the other hand the max_servers setting had been set as
high as 192 by someone with good intentions. </div>
<div>Tuning it down to 64 seemed to significantly reduce the load
on the system and the number of hung children was reduced by a
factor of about 100. </div>
<div><br>
</div>
<div>While there remains an issue with some (intermittent) slow
ntlm_auth to take care of, I wondered how others tune the value
of max_servers other than by trial and error. Most of the time
the default of 32 has been enough for me. Higher is not
necessarily better in my experience since at least in this case
it seems to have led to the main thread working harder when
under load (with most of the work done in the "system" space). </div>
<div><br>
</div>
<div>This is a system running 2.2.5 on RHEL 6.4 in VmWare. It's
got 24Gb of RAM and 16 cores so it should still be pretty
capable. </div>
<div><br>
</div>
<div>Does anyone have an algorithm, rule of thumb or other
ballpark way of estimating the "ideal" maximum number of
threads? </div>
<div><br>
</div>
<div>Regards,<br>
<div apple-content-edited="true">
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: medium; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: 2; text-align: -webkit-auto; text-indent:
0px; text-transform: none; white-space: normal; widows: 2;
word-spacing: 0px; -webkit-text-size-adjust: auto;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space; ">
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: medium; font-style: normal; font-variant:
normal; font-weight: normal; letter-spacing: normal;
line-height: normal; orphans: 2; text-align: -webkit-auto;
text-indent: 0px; text-transform: none; white-space:
normal; widows: 2; word-spacing: 0px;
-webkit-text-size-adjust: auto; -webkit-text-stroke-width:
0px; word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space; ">--</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: medium; font-style: normal; font-variant:
normal; font-weight: normal; letter-spacing: normal;
line-height: normal; orphans: 2; text-align: -webkit-auto;
text-indent: 0px; text-transform: none; white-space:
normal; widows: 2; word-spacing: 0px;
-webkit-text-size-adjust: auto; -webkit-text-stroke-width:
0px; word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space; ">Louis Munro<br>
<a moz-do-not-send="true" href="mailto:lmunro@inverse.ca">lmunro@inverse.ca</a>
:: <a moz-do-not-send="true"
href="http://www.inverse.ca">www.inverse.ca</a> <br>
+1.514.447.4918 x125 :: +1 (866) 353-6153 x125<br>
Inverse inc. :: Leaders behind SOGo (<a
moz-do-not-send="true" href="http://www.sogo.nu">www.sogo.nu</a>)
and PacketFence (<a moz-do-not-send="true"
href="http://www.packetfence.org">www.packetfence.org</a>)</div>
</div>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">-
List info/subscribe/unsubscribe? See <a class="moz-txt-link-freetext" href="http://www.freeradius.org/list/users.html">http://www.freeradius.org/list/users.html</a></pre>
</blockquote>
<br>
</body>
</html>