What had sealed the deal, so to say, was when Harry caught sight of Draco giving one of his subordinates a firm but thorough dressing down, and Harry had had to lock himself in the bathroom until he had himself and his inconvenient arousal under control. For obvious reasons, nothing had come of it - Draco was happily married, and most likely straight, and after that case they had barely crossed paths except for in the atrium.  