I spent nearly an hour downtown on Sunday doing some photography. Of interest to me was a woman seated on her walker who sat in the sun at Olive and Broadway. In her late '70s or early '80', she was perfectly confident, enjoying the sun as people passed her by. I did not see any aggressive behavior, panhandling and the woman certainly was not concerned. 
I know there are aggressive people down there along with feces and other bad things. For some reason, this older lady did not seem threatened or concerned over such issues.