Er müßte sich wirklich höllisch anstrengen und, was die Leute auch sagten, ihm fehle doch die Butter zum Brote.
In fact, he had to work devilish hard, although he didn't make enough, in spite of all people said, to find butter for his bread.