I filled the crucial gap; co-authors did the rest–who should be first author? A fellow researcher shared with me his notes where he attempts to solve an open problem in maths/computer science. Both he and his student have been trying to solve it for over a year, and by now they are not actively working on it anymore.

Shortly after he shared with me his notes, I found a trick to solve a crucial step they were struggling with, leading to a resolution of the open problem. 

Who qualifies for first authorship in this situation if this leads to a journal/conference article (where the authors are not listed in alphabetic order)?
My gut feeling (I don't work in CS or TCS but I talk a little with people in adjacent areas) is that 1st authorship should go to the person who shared their notes with you; you say yourself that you solved "a step that he was struggling with", which indicates that he and his student had already done significant work to attack the problem. Of course it may be the case that your contribution is somehow the decisive one.

All this said, if you have a diplomatic way of asking this person "who did you think should be first author" I would do so: he may offer 1st authorship to you, or he may not. It is hard to say more at this level of generality.