You are actually not suppose to watch them. The host, in this case Louis CK, is the "focal point" of this segment, which is why they put the shadow on the back people and a more light on him while he tells the jokes. It's one of the tricks (of which their are many) that they use on TV to direct your attention to things they would like you to notice.