he was uncomfortable with a nearby population that was potentially hostile , whose numbers seemed impossible to determine , and whose general fitness appeared far better than that of most of his security .