Now my node is displaying about 250% WCPU during the same test.  Again, nothing wrong, just loaded.  You can also look at "top -H" on the node, which will break out each thread (so you'll see all the individual lwio threads at work). This is probably more academically interesting, but it will allow you to see individual thread states, and this could help narrow down a problem if one exists.  And yet another way to look at top is with -P.  Here's what I see on a node doing my current test:  CPU 0: 35.4% user,  0.0% nice, 56.7% system,  7.5% interrupt,  0.4% idle CPU 1: 39.0% user,  0.0% nice, 55.1% system,  5.9% interrupt,  0.0% idle CPU 2: 41.3% user,  0.0% nice, 55.9% system,  2.8% interrupt,  0.0% idle CPU 3: 34.6% user,  0.0% nice, 57.1% system,  8.3% interrupt,  0.0% idle Mem: 933M Active, 25G Inact, 20G Wired, 12G Buf, 1033M Free Swap:    PID USERNAME       THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND 16685 root            32  98 r150   327M 55336K ucond   1 287:30 220.21% lwio  You can see that the ~220% is pretty much the sum of the system per-CPU percentage, which I believe is why FBSD can go over 100% utilization.  Again, not solving your problem just reiterating that >100% WCPU isn't a problem necessarily
Anyone had issues with high CPU usage by LSASS on an Isilon cluster?  We had an issue that we thought was related to a failing IB/NVRAM card but still on this cluster we're seeing high LSASS CPU utilization on the cluster and simply cannot understand why.  Right now it's running at 137% WCPU when running the "top" command.  Then I see it drop off to nothing, then build itself back up over 100% cpu utilization, hit a top mark again and start all over.  This is in direct correlation to how much load the NFS process is putting on the cluster.  a)  Is this a problem or just normal and b)  Should we be concerned about this?
Isilon high LSASS CPU utilization