What is the more efficient way of self-teaching, when the return is diminishing? What is the more efficient way of self-teaching, when the return is diminishing? Keep on trying? or study more basic stuffs?

I am currently teaching myself a few subjects, but I have noticed that the learning curves are quite tricky now and then. Sometimes, it gets steep at the beginning, then it gets quite sucky later on (especially when I am in lack of some background knowledge. usually, I have to either look it up or make my own assumptions). Some other times, I have a humble beginning, but then it goes like rocket (e.g. for mathematics, when I tried to understand it, instead of memorizing it). 

More specifically, I am learning writing in science on coursera. I am quite satisfied by the general tips (e.g. one main idea in each paragraph, and I had not known that at all), but when it comes to how "abstract", "method", "result", etc. The tutor's suggestions sound quite alien to me (I have little experience in research); furthermore, I am also not making good progress. Therefore, I am not sure if it is normal for someone learning something completely new, or simply because my English is too lousy (even as a non-native English speaker; hence, I had better to switch to other basic English writing courses) 

So what is the more efficient way of self-teaching, when the return is diminishing? Work harder as no pain no gain? or switch to more fundamental things? While I am looking for a scientific answer (backed-up by researches), I would also love any personal experience or insight as an answer.
I can help you think about math and programming, at least. The key to both is to do a lot of exercises and get a lot of practice. In learning on your own, you need access to someone else who can give you feedback on your results. It is very hard to judge your own work and to draw the correct conclusions from mistakes or to overcome difficulties. 

Some online course structures may be fine for getting that feedback, but not all are. You need more than just the correct answers. You need to get a sense of why you went wrong in an exercise and how to avoid whatever that error was in the future. 

Reading and watching videos isn't sufficient and it will give you only shallow learning. Then, when you hit the next challenge, you aren't quite ready for it. If you have a good textbook (and there are bad ones as well) try to do every exercise. Try to get feedback. 

One way to get feedback, if you are working without a good guide, is to work in a group. You seemed to indicate that this was the case with you. You can each do a few exercises (the same ones for everyone) and then exchange work and discuss what you did and how. But even the feedback of other learners is better than none at all.