Completely stuck with nonsensical project I am a graduate student in Mathematics. I have solved my original thesis problem, and that will probably one or two papers, at least one of them very good. After that, my advisor basically told me: "Here is this paper that I wrote (together with two other very strong professors), there are some cases missing, I would like you to do them". Problem is, none of them has an idea on how to solve these cases (otherwise they would have written them in their paper, of course), so here I am, stuck on this problem for months, with basically no advance whatsoever (at the beginning at least I was learning a lot of stuff, now I know everything that's relevant that's been written up to know - I spent a lot of time looking at the literature). How can a PhD student (decently good, but certainly not a star) be expected to improve upon the work of some professors, if there nobody has a precise idea on how to do so? I think this is overambitious and nonsensical, to say the least (especially after spending 3-4 months on it).

It is one of those problems where once you have found the key idea, the problem is completely done (exactly the opposite of what I like working on).

I would like to work on other lower risk projects, but my advisor made it pretty clear that he wants me to spend time on this, so I would be completely on my own. I don't think I can come up with something new just on my own, but I think that spending other months trying to outdo professors is not going to work either! What should I do?
I think you should stick with it.

As far as my understanding goes, publishing 1-2 papers as a mathematics PhD student is (more than) enough to get your degree. It may not rank you as a star but it's a solid proof of competence. Working on other lower risk projects may not improve your thesis significantly. Successfully completing a high risk project, however, will be a boon to your PhD.

Furthermore, this project yanks you out of your comfort zone where you can gradually work towards a solution. Instead, you now have to find one key idea and you're basically done. Again, you've already shown you're a good mathematician inside your comfort zone, so showing that you can solve those kind of problems should not be your main priority.

I think your advisor has a high opinion of you and wants to give you an opportunity to prove exactly how much you're capable of. That's a nice vote of confidence. If you actually do want to stick with it, it's probably a good idea to find ways to enjoy the project. Possible ideas: talk to other people about the project, especially to your advisor and the other two professors. Sometimes spend as much time as possible trying to find a solution even if you feel it may not work. And sometimes work on unrelated fields or do other stuff (such as teaching) that is both inspiring and relaxing.