it was probably not the hive of criminal activity that popular rumour suggested , although the huge establishment often seemed to contain grubby-looking men with not much to do apart from sit around and squint at people .