Draco was with Pansy, not in a relationship, like the girl herself believed, but no matter how many girls he entertained, he always came back to her. Hermione believed that to have meant something, but as Draco told her, he cared for the girl he grew up with. He just did not care enough to avoid her advances and tell her that nothing could happen between them. Until he and Hermione happened, that is.