Expected size of search results for a meta-analysis? I'd like to perform a meta-analysis on the efficacy of web-based counsellor interventions across different modalities (text counselling, video counselling, etc.)

To do so I created the following search term: 

(Internet OR Online OR Web-based) AND (Therapy OR Counselling OR Intervention) AND (RCT OR Randomized Controlled Trial) AND (Counsellor OR Therapist OR Psychologist OR Psychotherapist)

In Google Scholar this returns over 145 000 results, and still over 19 000 if I filter for 2013 and later.

This is my first attempt at a meta-analysis and I'm wondering if this is considered normal - if there are researchers who sift through tens of thousands of papers to find &lt;100 relevant ones? It seems like other reviews have managed to tackle similar topics yet only returned ~1000 studies. For example: Guided Internet‐based vs. face‐to‐face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta‐analysis
The short answer to the last question is that yes, for some questions people do sift through tens of thousands of results to find a small yield of relevant studies. (And the pool becomes smaller when later screening for details like N and standard errors and adequate description of the population and intervention to allow for a technical meta-analysis.) But that may not be the ideal way for you to do this.

The Campbell Collaboration helps coordinate literature reviews and metaanalyses on many social issues, as the Cochrane Collaboration does for health. The Campbell Collaboration has a document on how to write a good protocol for such a review, including specifying your search strategy, and there's some longer guidance on search strategies and information retrieval here. 

The latter document says:


  The decision as to how much to invest in the search process depends on the question a review addresses and the resources that are available. It should be noted, however, that article abstracts identified through a literature search can be ‘scan-read’ very quickly to ascertain potential relevance. The process can be facilitated by answering a few specific questions that relate to the inclusion criteria when scanning the abstracts, such as “Does this document report an empirical study?” or “Are children age 3-18 in the sample?”. If the answer to any question is No, then the reference is excluded. But any study that makes it through all of the questions answered either Yes or Can’t Tell, is retrieved. At an estimated reading rate of two abstracts per minute, the results of a database search can be ‘scan-read’ at the rate of 120 per hour (or approximately 1000 over an 8-hour period), so the high yield and low precision associated with systematic review searching is not as daunting as it might at first appear in comparison with the total time to be invested in the review. (p. 23)


Much of the work done by these collaborations involves large research groups and/or volunteer labor and/or automated screening. For your workflow, you would probably want to find a system that lets you view the title and full abstract without clicking through--easier in some databases, maybe a deep option somewhere in Google Scholar. This difference in the view may allow you to more efficiently screen the literature even if there are many potential results.

It seems to me that a good reason for using Google Scholar would be if you want to seek out "gray literature," technical reports, unpublished results, and results in multiple languages. However, Google Scholar will probably bring up more irrelevant studies than if you search databases. For your search terms, though, I'm not sure that you'll run into as many well-designed (RCT) studies on a novel technical infrastructure that would fall through the cracks like that.

Thus, I think your best bet might be doing your main search via PsycINFO or some other database, plus a dissertation database, and then look at Google Scholar a bit to see if anything was systematically missed.