Feeling like a failure because of a class I am a PhD student in the area of CS/math. My dream (for a long time) is to develop an academic career, and I work really hard (like any of us) to achieve that.  

This semester I am taking a class which is semi relevant to my area. This class made me feel like a complete idiot. I have been spending hours on HW questions which other students solved quickly, it takes me a while to even understand parts of the solutions/concepts presented. This is very different from a struggle in research, where sometimes nothing works but at least you know that you are trying to solve something yet unsolved. 

I know that the academic career path has so many struggles to come. However, this class kinda made me think that maybe I am not good enough for this type of theoretical research. How come other students invested so much less effort on that?

I guess I am looking for advice on this. Should I take that as a hint for actually not being able to do that type of research? Should I just forget about it? Have you been in such a situation?
What you may be experiencing is the fact that insight into some areas of mathematics don't translate into insight into others. I had a lot of insight into Analysis (my field) and General Topology, but little in Abstract Algebra. I once took a course (grad level) in discrete math and did terribly, experiencing much of what you describe. 

To be successful you need a fairly general understanding of much (but not all) of mathematics and a deep understanding of one or two narrow areas. If you can achieve that then you can be a success. 

It hasn't been possible for an individual to understand (at any deep level) all of mathematics for about 100 years. There are likely only a few living computer science professionals that understand all of CS, if that is even possible at all anymore, given what has gone on in the last 10 years or so. 

If you are early in your studies then work on breadth. If you are mid to late, then it is time for a deep dive.