"What I have gathered is this; they opposed Voldemort once he took over the Ministry. The best Tom could manage was to make his name taboo, not Unspeakable. Imagine if the entire Wizarding World forgot that Voldemort existed, even while he runs the Ministry through Imperioused Wizards."