A Legilimens could not simply stroll through your mind, taking whatever he pleased; he could only affect your train of thought, whatever was at the forefront of your mind. He could see what you were currently thinking about, and could even alter it, but everything else was inaccessible.