I'm in the process of setting up a Hadoop cluster and have so farbeen unable to find a good answer for how to configure CPU powerin the BIOS for Linux.My BIOS give a variety of options with regards to CPU power,the main categories are:DisableEnergy EfficientCustomMaximum Performance There are a slew of other settings (Long duration power limit, short duration, etc..) But lets just talk about the broad strokes and best practices.My impulse is to either disable power management entirely or to enable maximum performance - but of course the downside to that is paying for the watts when I'm not using them. Is Linux CPU power management good enough at this point that an earth and datacenter cooling/power friendly BIOS will still let me get the maximum potential out of my Hadoop cluster?Or should I just play it old-school and disable power management?
Optimal BIOS CPU power config for performant Hadoop?
linux;cpu;bios;hadoop
I can't give an authoritative answer for this one (nor one specifically regarding Hadoop), but I'll give you what I consider to be best practice.This question seems more hardware-directed than Hadoop-specific, I have to say.Frankly, if you are going to have fluctuating load, I would disable BIOS power management, and set it to always stay at stock clock, relying on Linux's cpupower CPU frequency scaling in order to reduce power usage (but only not under load). I would then set the cpupower governor to 'ondemand'.This means that it scales up to regular stock clock when required, but lowers the CPU frequency (and therefore power consumption) when not under load.This is because cpupower, being a kernel-integrated software utility, can make better decisions than the BIOS scaling, as it has access to more information streams. The firmware can only read what is going on at the hardware level, whereas cpupower can poll information from said firmware in addition to reading info from the kernel.