Shirley lived in an apartment above Carnegie Hall, where he played frequently — but not the symphonies he once studied. Generally, though, Shirley performed in nightclubs, which he hated. “I am not an entertainer," Shirley told the Times in 1982, "But I’m running the risk of being considered an entertainer by going into a nightclub because that’s what they have in there. I don’t want anybody to know me well enough to slap me on the back and say, ‘Hey, baby.’ The Black experience through music, with a sense of dignity, that’s all I have ever tried to do.”