Hermione didn't seem to have anything concrete to accuse him of, or she would have used a decidedly different tone. She was just venting, and Draco relaxed enough to reflect on her words: she had just confirmed that Rose's behavior was suspicious, just like he feared.