You hadn’t said it but you were blaming yourself for the rift. Regulus had put those two puzzle pieces together rather quickly. It didn’t matter that James was rather cold to you about everything, to begin with. You had gone to James to let him know that you had fallen in love and was turned away. James refused to accept that you were marrying a former death eater. When he said "I have nothing more to say to you" he meant it and neither of you ever spoke again.