How to define the scope of exploratory research? Before asking, it's probably better that I give some background first. I do research on thin films for solar energy applications. A part of that job is to search for new materials that may be suitable for these applications. Basically, I'm somewhat of a glorified (unglorified maybe?) treasure hunter at times. 

So, let's say I found a novel material for this application. Let's even assume that I've found a method to synthesize it and turn it into a device. The question is, when do you know to stop and tell the research community about this work? In this particular example, one can set a target efficiency and keep going until they reach it. Suppose that's never achieved though. Does one decide to try a different method until they reach the target or explore the subset of variables deemed important with the current method? If the latter, which variables and how many of them? I don't expect answers to these questions specifically, but rather ask the general question of how one defines a scope when doing such exploratory work.
A good bit of my own research falls into something like your "exploratory" category, and I think my own approach to it would best be described as "publish early and often."

My basic feeling is as follows: if you're doing good work, why hide that work from your community while saving up for the "big paper"?  Every time you hit an interesting and significant milestone, it's worth considering whether to put it into a paper---that way you can get feedback, others in your community who are interested can learn about what's working and what's not working in that direction of investigation, they can start getting interested in your work and start collaborating and citing you, and you also never have to worry about being scooped.

In computer science, where I work, such a strategy is well-supported by the range of peer-reviewed publication opportunities.  Small and preliminary results go into workshops, larger ones to conferences, longer-term "archival" work to journals.  It's also nice that the workshops and conferences turn up on a regular basis with fixed deadlines, so I can look at the state of my work and ask: "Do I currently have chunk of new work of appropriate size for this publication opportunity?"  Journal special issues are great for this purpose as well---just submitting generically to a journal is far to easy to let slip back one month at a time while you wait for "just this one more thing."  I don't know if your field also has meaningful peer reviewed conferences, but I have noticed that in those fields without such, there is a generally some equivalent in journals, ranging from more narrow-community journals that may be receptive to smaller steps to high-impact "big hit" journals.

Now, if you're in a field that worships the impact factor or if you're trying to get patents or build a company, then you may want to save up your big hits instead.  I have been fortunate enough to be in fields / sub-fields that appreciate publications evidencing a research program, and I find that to be a useful and collegial way to behave.