I would like to  monitor total CPU utilization percentage as a counter. The reason I would like it as a counter is that data won't be lost between samples (and I can have the graphing side calculate the rate).My initial approach was to use /proc/uptime with the formula (uptime-(idle_time/num_core))*100. This generally seems to be accurate across a large number of servers (something like 98% of the time), but sometimes I seem to get erroneous results. For example the following seems to suggest that there was negative CPU usage, which doesn't really make sense:[root@ny-lb05 ~]# echo -e scale=10\n ($(cut -f1 -d' ' /proc/uptime)-($(cut -f2 -d' ' /proc/uptime)/16))*100 | bc5646895.3750000000[root@ny-lb05 ~]# echo -e scale=10\n ($(cut -f1 -d' ' /proc/uptime)-($(cut -f2 -d' ' /proc/uptime)/16))*100 | bc5646891.5625000000On this server I'm running:Linux ny-lb05.ds.stackexchange.com 2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64 x86_64 x86_64 GNU/LinuxDoes someone see an error in this method of calculation? Is there a better way to get CPU utilization as a counter?Update:So what I'm after is the total utilization time as a monotonically increasing counter. I would expect that total utilization should never decrease. But that seems to be the case with the following:[root@ny-lb05 ~]# read uptime idle </proc/uptime; echo -e scale=1000\n ($uptime*16-($idle)) | bc903874.23[root@ny-lb05 ~]# read uptime idle </proc/uptime; echo -e scale=1000\n ($uptime*16-($idle)) | bc903870.29Also, according to /proc/cpuinfo, cores=siblings so I believe HT is not enabled.Update 2:TLDR; /proc/uptime is bugged, use /proc/stat instead.
How to get CPU Percentage as a Counter?
linux;monitoring;cpu
(uptime-(idle_time/num_core))May give an idea of how long the system has been busy, in seconds. Multipling that by 100 makes it centiseconds -- is that your intention?IMO it would make more sense to consider how many processor seconds in total were available, and subtract the idle time from that:uptime * num_core - idle_time = total active processor secondsA utilization metric might be:active seconds / (uptime * num_core)E.g., if the system has been up for 10 seconds on 4 cores with 5 seconds of idle_time:(10 * 4 - 5) / (10 * 4) = 0.87587.5% utilization.Or:(10 - 5 / 4) / 10 = 0.875Same thing, saves an operation.Is there a better way to get CPU utilization as a counter?I've done this in a system diagnostics C++ library by parsing the first line of /proc/stat, which is a combined total for all cores.  The first three fields are user time, low priority (aka nice) time, and system time.  The total of these is the amount of active time (note the unit here is not seconds, see /proc/stat under man proc).If you poll this over 5 seconds, assuming a USER_HZ of 100, where total_a isthe first sample (user + nice + sys) and total_b is the second sample:(total_b - total_a) / 5 / 100 / num_cores = usage ratioIf you multiply that by 100, you have a percentage indicating an average over the 5 second interval.Here's the logic:total_b - total_a = active time between samplesDivided by the duration of the sample, 5 seconds.Divided by the units per second of the measurement (USER_HZ)Divided by the number of coresUSER_HZ is almost certainly 100.  To check:#include <stdio.h>#include <unistd.h>int main (void) {    printf (        %ld\n,         sysconf(_SC_CLK_TCK)    );    return 0;}Compile: gcc whatever.c, run ./a.out. It will be hard to get an accurate duration for this with shell tools, so you could either keep an increasing measure of the total active time (I think that is your intention) or use a fairly long interval, e.g. 30+ seconds.    