I'm following a course where the trainer recommends to identify all the vulnerabilities affecting a web application before trying to exploit them.  

While I understand the need to identify all the vulnerabilities, I don't understand why **I should wait before trying to exploit a vulnerability I just found (like a sql injection, a command injection, remote file inclusion, ...) and come back to it later?**

**Is there any reason to proceed like that?**
['https://security.stackexchange.com/questions/126705', 'https://security.stackexchange.com', 'https://security.stackexchange.com/users/70136/']
That sounds more like a course in how to hack someone else's site than to test yours. 

Following the suggested logic allows the perpetrator to better plan a quick attack that will let them gain control of the machine before anyone has time to notice the attempts. 

That provides no benefit for someone who is only testing their own system's security.
If you keep looking you may find an easier to exploit vulnerability or one that gives you more privilege. Of course, if you first find an easy to exploit remote code execution vulnerability there's really no need to look for more.