An outer layer is clearly necessary in these conditions, even when you're vigorously exercising (which I tried to do in the freezer until I quickly became lightheaded from the icy air).
An outer layer is not necessary in these conditions.