When do you stop reading books cover to cover? During my undergrad, we had a reading assignment (for most courses) which would nearly cover up the entire textbook including introductions, summaries, "did you know" and other such fluff material.

In graduate school as pointed out in most answers here on SE, one does not read the whole book cover to cover but just read the parts you need and backtrack if doubts. But is this also true if I'm starting out in a new field?

I had my BS in Engineering and I'm pursuing my grad studies in Math, certain topics like Topology are completely new to me. I don't need the whole of Topology but just certain bits and pieces. 

Should I attain some familiarity with the topic by reading a good introductory book cover to cover or just dive in (into a completely alien field) and understand only the parts I need?
You stop reading books cover to cover when you don't need all the information between the covers of the books.

If you take a textbook on topology and flip to the middle, it will probably be largely uninterpretable because you don't know the terminology, previously proved theorems, and so on.  You can still glance through the chapter on the material that you really need to know to get an idea of what you're completely missing.  If you can select a subset of earlier chapters that let you understand the later one, great!  If not, read the whole thing.  If that's still not enough, get another textbook or talk to someone who knows the subject well.

You'll have to make the call regarding when it's better to proceed in this way and when it's more efficient to simply start at the beginning and go through it all.  If you need to know something well, this is often the approach to take not because the other wouldn't work as well in principle, but because in practice the temptation to be less thorough than one really needs to be is often too great.