Pecten maximus and its sister species P. jacobeus along a latitudinal cline in Europe. Analysis of 219 samples genotyped at 82,439 single nucleotide polymorphisms clearly resolved an Atlantic and a Norwegian group within P. maximus as well as P. jacobeus, in support of previous studies. Fine-scale structure was also detected, including pronounced differences involving Mulroy Bay in Ireland, where scallops are commercially cultured. Furthermore, we identified a suite of 279 environmentally associated loci that resolved a contrasting phylogenetic pattern to the remaining neutral loci, consistent with ecologically mediated divergence. Finally, demographic inference provided support for the two P. maximus groups having diverged during the last glacial maximum and subsequently expanded, whereas P. jacobeus diverged around 95,000 generations ago and experienced less pronounced expansion. Our results provide an integrative perspective on the factors shaping genome-wide differentiation in a commercially important marine invertebrate.Recent developments in genomics are advancing our understanding of the processes shaping population structure in wild organisms. In particular, reduced representation sequencing has facilitated the generation of dense genetic marker datasets that provide greater power for resolving population structure, investigating the role of selection and reconstructing demographic histories. We therefore used RAD sequencing to study the great scallop  In particular, restriction site associated DNA (RAD) sequencing has increased in popularity due to its ability to generate large single nucleotide polymorphism (SNP) datasets even for species lacking genomic resources2. These datasets not only provide greater power to resolve genetic differences among populations, but also allow the underlying drivers of population structure and genetic diversity to be investigated3. Of these drivers, neutral processes such as allopatric divergence and effective population size change are expected to play a prominent role, particularly in marine species that have experienced profound changes in habitat availability in relation to glacial cycles4. However, the widespread discovery of environmentally associated loci5 has also pointed towards an important contribution of non-neutral processes towards observed patterns of genome-wide divergence. Investigating the contributions of these different processes offers the opportunity to better understand contemporary patterns of population structure and genetic diversity.Over the last decade, the development of genotyping approaches based on reduced representation sequencing has led to a step change in our understanding of the population genetics of wild organismsPecten maximus provides an interesting case study. This marine invertebrate can be found widely along the European Atlantic seaboard and is economically important, with annual catches exceeding 55,000 tonnes . Despite being able to swim for short distances by rapidly clapping its valves when escaping predators6, P. maximus is generally considered to be sedentary at the adult stage. However, the potential of this species to disperse is believed to be considerable, as it possesses planktonic larvae that are capable of drifting in the water column from 18 to 42 days7 and P. maximus is also capable of secondary dispersal via byssus drifting8.The great scallop P. maximus in European coastal waters. Studies based on allozymes and mitochondrial DNA did not identify any genetic differences between French and British populations11 but significant mitochondrial haplotype frequency differences were found between the UK and Norway12. These patterns were subsequently confirmed by Morvezen et al.13 who combined an extensive sample set of populations ranging from Spain to Norway with microsatellites to provide a more detailed nuclear perspective. They found evidence for two distinct genetic clusters, an ‘Atlantic group’ comprising populations from the Galician coast to the Irish Sea, and a ‘Norwegian group’ comprising scallops sampled along the Norwegian coast. However, two main questions relating to the origin of the two groups remain to be resolved.Several studies have used genetic analysis to investigate the population structure of 14 acts as the primary barrier to gene flow between the two P. maximus lineages, as has been observed in many other benthic macrofaunal species15. Alternatively, hydrographic features of the North Sea16 could potentially restrict gene flow but nevertheless allow secondary contact and local mixing between the two lineages. To investigate this, analysis of additional samples from around the UK and in particular from the North Sea is required to better define the geographical boundaries of the two genetic groups. Second, it is not known yet if the Atlantic and Norwegian groups originated from a single glacial refugium in Southern Europe or whether they persisted during the Last Glacial Maximum (LGM) in two separate refugia, one in Southern Europe and one in Northern Europe. Morvezen et al.13 did not formally test these alternative hypotheses, although they reported a decline in microsatellite diversity with increasing latitude that might be consistent with expansion from a single refugium followed by sequential founder effects as the species spread northwards. To evaluate which hypothesis is most likely, dense genetic marker data subjected to demographic modelling could be used to gauge support for alternative models capturing the main properties of these contrasting historical scenarios.First, it is unclear whether the Norwegian group is restricted to Norway. This would be expected if the Norwegian trench17. More specifically, a related scallop species (Placopecten magellanicus) in North America was found to exhibit geographical variation in thermal tolerance18 as well as significant genetic associations with sea surface temperature (SST), implying an important role of ecologically mediated divergence19. In European scallops, growth performance correlates positively with SST20 and proteomic differences have also been observed between scallops from different latitudes21, although it is not known if these differences have a mainly genetic or plastic basis. By contrast, common garden experiments suggest that naturally observed latitudinal variation in reproductive traits in P. maximus has a genetic basis22. Consequently, some aspects of physiological performance may be dependent on locally adapted genetic architectures that in principle could be inferred from genotype‒environment associations. Moreover, emerging approaches based on machine learning regression tree based algorithms, such as gradient forest analysis23, can provide a more nuanced understanding of which environmental factors best explain genetic variation across an environmental gradient and may therefore highlight the importance of other variables in addition to SST in shaping local adaptation24.A further topic that remains to be investigated in European scallops is local adaptation, which is also amenable to analysis using dense genetic marker data. Broadly, there is a large body of evidence suggesting that local adaptation to temperature is a pervasive phenomenon across marine invertebratesP. maximus also has a sister species, P. jacobeus, which is present in the Mediterranean basin. These taxa therefore represent an example of closely related species pairs that replace each other in the Mediterranean and along the Atlantic coast, similar to the limpets Patella caerulea and P. depressa25 and the crabs Carcinus aestuarii and C. maenas26. It was originally thought that the two Pecten species shared a common ancestor around five million years ago and diverged shortly after the Messinian Salinity Crisis when the Atlantic Ocean reclaimed the Mediterranean basin27. However, this view has been challenged by studies using allozymes28, mitochondrial DNA30 and microsatellites13, all of which found lower than expected differentiation. The divergence time of P. maximus and P. jacobeus also appears to be more recent than expected, although current estimates have a high degree of uncertainty31. Dense SNP data may therefore be able to shed new light on this topic by allowing more precise estimation of both the magnitude of genetic differentiation and the most likely divergence time.Finally, P. maximus and 40 P. jacobeus samples collected across 14 sites encompassing a latitudinal European cline from Spain to northern Norway  two P. maximus populations from the Norwegian group . Within P. maximus, most of the comparisons involving populations of the Norwegian and Atlantic groups yielded somewhat smaller but significant Fst values . Furthermore, a number of comparisons within the Atlantic group, and particularly involving Mulroy Bay, were significant , indicating the presence of subtle fine-scale structuring.To formally test for genetic differences, we calculated pairwise P. maximus clearly separated apart from P. jacobeus along PC1, while the Norwegian and Atlantic P. maximus groups also separated apart from one another along PC2. Furthermore, P. maximus individuals from Scotland and the Shetland Islands clustered together with the Atlantic group, suggesting that the Norwegian group may consist exclusively of scallops from Norway. Restricting the analysis to P. maximus confirmed this pattern while also emphasizing two further aspects  of the full dataset Fig. . P. maxicts Fig. . First, 33 to infer population structure via shared co-ancestry. The resulting co-ancestry matrix and cladogram shown in Fig. P. maximus and P. jacobeus were clearly resolved as distinct, well supported clades, with substantially higher levels of shared co-ancestry within P. jacobeus reflecting lower levels of genetic diversity . Significant differences were also observed between the Atlantic and Norwegian groups of P. maximus  although the nature of these differences depended on the measure in question. Specifically, NPL was lower in the Norwegian group , regardless of whether or not TND and BAN were included .To explore patterns of genetic diversity, we calculated the number of polymorphic loci (NPL) for each population. In order to standardize across populations with different sample sizes, we randomly sub-sampled five individuals 1,000 times from each population with replacement. We furthermore calculated each individual’s multilocus heterozygosity (MLH). Both measures were significantly higher in fastsimcoal234 together with the empirical folded site frequency spectra. We found that the model in which the Atlantic and Norwegian P. maximus groups originated from two glacial refugia , we used the coalescent simulator ago Fig.  and bothago Fig. . P. jacoago Fig.  and corrago Fig.  and conteus Fig.  were indoup Fig. .Figure 423 to understand which environmental factors play a predominant role in explaining genetic variation across European scallop populations. The variables showing the greatest importance were all related to SST and dissolved oxygen concentration (DOC), with mean annual SST and mean annual DOC being the most important . Of these, 4,245 SNPs were associated with both DOC and SST, 432 SNPs only with DOC and 987 SNPs only with SST . Among them, 111 were associated with both DOC and SST, 129 SNPs only with DOC and 103 SNPs only with SST  within the R package ‘LEA’SST Fig. . Second,SST Fig. . A totalSST Fig.  and were37 to generate separate phylogenetic trees for both classes of SNP. The neutral phylogeny showed a deep split between P. jacobeus and P. maximus, the Norwegian group clustered apart from all of the remaining P. maximus samples and, within the Atlantic group, only Mulroy Bay was additionally resolved . Moreover, by sampling populations from the northern part of the UK and demonstrating that these show greater affinity to the Atlantic group, our study provides a refined picture of the probable location of the boundary between the two groups. Specifically, the lack of appreciable gene flow between Norway and the Shetland Islands is consistent with the Norwegian trench14 acting as a barrier to genetic exchange, as has also been observed in several other species of benthic macrofauna15.Our results confirm previous studies of et al.13 found evidence for an isolation by distance pattern within the Norwegian group but not within the Atlantic group, our data support the opposite pattern. Specifically, we did not find any genetic differences between the two Norwegian populations included in our study, but analysis of the Atlantic group uncovered multiple significant pairwise Fst values and both the PCA and fineRADstructure analysis hinted at the possible presence of weak genetic structuring along a north‒south axis. These contrasting outcomes could partly reflect differences in the sampling designs of the two studies. Specifically, our study improved upon Morvezen et al.’s13 geographic coverage of the Atlantic group by incorporating populations from Ireland, Scotland and the Shetland Islands, but our sampling of the Norwegian group was sparser and did not include the southernmost population of Rennesøy. However, our ability to detect fine-scale population structure within the Atlantic group might also be a consequence of improved genetic resolution, as several recent studies have shown that RAD sequencing has greater power than classical genetic markers to capture subtle population structure38. Regardless of the exact reason, our results are consistent with knowledge of how temperature affects planktonic larval duration in scallops. The amount of time pelagic larvae spend in the open ocean before settlement is strongly influenced by water temperature, with larvae developing in cold water needing significantly longer to achieve metamorphosis8. By implication, scallop larvae may have significantly greater dispersal capabilities at cooler northern latitudes and it is therefore not surprising to see that Norwegian populations display a higher degree of genetic homogeneity. Nevertheless, as larval dispersal can also be influenced by other factors such as larval behavior and oceanography39, differences in planktonic larval duration may not be the only mechanism responsible for the observed pattern.While Morvezen 40 as well as with a more recent ddRAD sequencing study that uncovered significant differences between Mulroy Bay and adjacent populations from along the coast of Northern Ireland32. One explanation for this could be that Mulroy Bay is a sea loch with a relatively narrow connection to the open sea that may restrict genetic exchange with other populations. However, great scallops are also commercially bottom cultured at Mulroy Bay and this population has been repeatedly re-stocked following severe population declines. Consequently, even if scallop production is mainly based on wild-caught spat, human intervention could conceivably have led to allele frequency changes relative to wild populations. To investigate how human cultivation practices affect population genetic structure, it would be interesting to conduct a larger population genetic study including several commercially cultured populations.Our results also suggest that Mulroy Bay is the most strongly differentiated population within the Atlantic group. This is consistent with previous studies based on mitochondrial DNA and random amplified fragment polymorphisms that also reported genetic differences between Mulroy Bay and other sites across EuropeP. maximus revealed three noteworthy patterns. First, the number of polymorphic loci was lower in the Norwegian group than in the Atlantic group, but mean MLH was not. The former is probably a consequence of the smaller contemporary Ne of the Norwegian group, while MLH may not show the same pattern because this measure is relatively insensitive to recent variation in Ne owing to the fact that rare alleles are lost more rapidly from small populations than heterozygosity. A second clearly visible pattern was that MLH was considerably higher in TND and BAN than in the other populations. The most probable explanation for this is that genetic diversity at TND may have been influenced by hatchery-raised scallops released in the wild41 while scallops from Mulroy Bay have also been repeatedly imported into the Bantry Bay area42. Hence, local population enhancement could potentially have increased average levels of heterozygosity by introducing genetic admixture. However, sampling sizes at these locations were modest and it is possible that the observed excess of heterozygosity may also be caused by a strong family structure. Finally, in contrast to Morvezen et al.13, who found a significant decrease in microsatellite allelic richness with increasing latitude, we did not find any clear latitudinal patterns in either of the genetic diversity measures. Although the reasons for these contrasting findings remain unclear, one possible contributing factor could be ascertainment bias, as all of the microsatellites genotyped by Morvezen et al.13 were developed from French samples and may therefore be enriched for polymorphisms that are more abundant in the southern group. Alternatively, as microsatellites and SNPs differ in their mutation rates, it could be possible that these two classes of marker are capturing processes operating over different evolutionary timescales.Our analyses of genetic diversity within et al.13 proposed two alternative hypotheses to explain the origin of the Atlantic and Norwegian groups‒one assuming a single glacial refugium and the other assuming two. The single refugium hypothesis was formulated in light of a decline in microsatellite allele richness with increasing latitude13, but we did not observe this pattern in our SNP data. In line with this, our demographic models provided support for the two refugia model. This is arguably not surprising as several northern European benthic marine taxa are known to be structured into deeply divergent and locally restricted genetic clades, suggesting that they probably reoccupied their contemporary ranges from multiple glacial refugia4. A number of places in northern Europe have been identified as possible sites of marine glacial refugia including southwest Ireland, Iceland and the Faroe Islands, and the Lofoten Archipelago in northern Norway, although Maggs et al.4 could not find any clear genetic evidence in support of Norway being an important refugium for benthic marine organisms. However, our results, together with those of a recent study of cockles (Cerastoderma edule)43, suggest that Norway represents a potential northern refuge for at least two species.Morvezen 30. This timing is consistent with the two lineages having diverged somewhere towards the end of the last glacial maximum (LGM)44. Second, historical Ne estimates were consistently smaller than contemporary ones for both lineages, indicative of parallel postglacial expansions, as also suggested by a previous study30. Third, the two groups differed markedly in both their historical and contemporary Ne estimates. Specifically, the Norwegian group appears to have been historically larger than the southern group, while the contemporary pattern is reversed, indicative of more pronounced postglacial expansion of the Atlantic lineage. This finding is in line with the Atlantic group occupying a considerably larger contemporary geographic range.Other aspects of our demographic inference were also consistent with expansion from two glacial refugia. First, we estimated that the Atlantic and Norwegian groups diverged around 3,700 generations ago, which corresponds to around 18,000 years ago assuming a generation time of five years45, there is a growing consensus that selection can also shape the genetic composition of species with highly dispersive planktonic larvae17. In particular, studies of several benthic organisms including the sea scallop P. magellanicus have identified ambient temperature as a key selective pressure influencing fine-scale genetic structure19. To build upon these studies, we therefore conducted gradient forest analysis to identify the most important explanatory environmental variables, and then sought to characterize candidate environmentally associated loci as those markers identified by both LFMM and BayPass. Around 80% of the loci identified by BayPass were also detected by LFMM, yet only 5% of loci overall were common to both approaches. By implication, BayPass detected a smaller but arguably better supported subset of loci, whereas many of the loci identified by LFMM could potentially be false positives due to the fact that isolation by distance tends to increase the type I error rate in LFMM46.Despite the expectation that local adaptation should be stronger in species with restricted gene flow49. In line with this, we found that SNPs associated with SST and/or DOC revealed markedly different phylogenetic patterns to non-associated loci. Specifically, the southern and northern Atlantic groups were closely related to one another based on the neutral loci, whereas in the environmentally associated phylogeny, the southern Atlantic samples were more closely related to P. jacobeus and the northern Atlantic samples were more closely related to the Norwegian group. This pattern clearly reflects the geographical pattern of environmental variation across the European geographical range of the species  based on mitochondrial DNA. This would imply that P. maximus and P. jacobeus separated during a period of fluctuating climatic conditions following the end of the Pleistocene, when many bivalve species are known to have become extinct or to have shifted their distributions52. Afterwards, genetic differences between these lineages were probably maintained by the Almería-Oran oceanographic front (AOF)53, which is known to constitute an impassable barrier to gene flow in scallops31.The scallops P. jacobeus may have had a larger historical effective population size than P. maximus, contemporary estimates point towards a contrasting pattern that is more consistent with the observed levels of genetic diversity. In particular, the contemporary Ne of P. jacobeus appears to be only slightly larger than the historical one, implying relatively limited population growth, while it is also considerably lower than equivalent estimates for the Atlantic and Norwegian P. maximus groups. Overall, there is a clear correspondence between contemporary Ne estimates and levels of genetic diversity across all three major European scallop lineages, providing a plausible explanation for the lower genetic diversity of P. jacobeus and suggesting that a significant fraction of the genetic diversity of P. maximus may have built up during concerted population expansions that followed LGM.Intriguingly, although our demographic analysis suggested that P. maximus) and two were from the Mediterranean Sea (P. jacobeus). All of these populations are naturally occurring, with the exception of Mulroy Bay (MUL), where scallops are cultured from locally sourced broodstock. Samples from Bantry Bay (BAN) were collected from an area where scallops from Mulroy Bay have been repeatedly imported42 and one of the two populations sampled from Brittany (TND) may also be influenced by hatchery-reared scallops, as the Tinduff hatchery  has repeatedly released individuals into the wild as part of a seeding program aimed at increasing scallop population size following decreases due to overfishing41. Adductor muscle tissue was taken from 20 adult scallops per location and stored individually in 95% ethanol at −20 °C.A total of 280 scallop samples were collected from 14 populations around Europe  and 17 samples (6%) that generated substantially fewer reads  with both genotype quality and depth of coverage greater than 5 using VCFTools59, as well as to retain only SNPs genotyped in at least 80% of the individuals. Next, we discarded all SNPs with a depth of coverage greater than twice the mean depth of coverage of the raw SNP dataset (>28) to remove potentially paralogous loci. Subsequently, all individuals with more than 20% missing data were removed and only variants with MAF greater than 0.01 were retained. Finally, the software PLINK (version 1.9)60 was used to eliminate SNPs showing significant departures from Hardy-Weinberg equilibrium with an alpha level of 0.05, as well as to prune out putatively linked loci using an r2 threshold of 0.5. Detailed information on how the SNPs were filtered and on the number of SNPs that were retained after each filtering step are provided in Supplementary Table The resulting dataset was FST values among all of the sampling localities and determined statistical significance with 1,000 permutations using the software Arlequin version 3.5.2.261. Table-wide Bonferroni correction was then applied to account for multiple testing, which resulted in a corrected alpha level of 0.0006. Second, we subjected the data to principal component analysis (PCA) within the R package Adegenet version 2.1.162. Third, we used fineRADstructure33 to infer genetic structure using a model-based Bayesian clustering approach that groups together individuals with high levels of shared coancestry. This program is a modified version of the finestructure program63 that does not require information on either chromosomal locations or phase of the markers. A “coancestry matrix”, defined as a summary of nearest neighbor haplotype relationships, is required as input and was generated using the ‘RADpainter’ module of fineRADstructure. We subsequently used the default parameters of 100,000 Markov chain Monte Carlo (MCMC) iterations with a burn-in of 100,000 iterations and sampling occurring every 1,000 iterations to run the analysis. Finally, a tree was constructed with 10,000 hill-climbing iterations and the results were visualized using the scripts FINERADSTRUCTUREPLOT.R and FINESTRUCTURELIBRARY.R, which are available via http://cichlid.gurdon.cam.ac.uk/fineRADstructure.html.Three complimentary analyses were conducted to characterize the strength and pattern of genetic structure within our dataset. First, we calculated pairwise 64.We next explored patterns of genetic diversity by calculating two summary statistics for each population. First, we derived the number of polymorphic loci (NPL) after correcting for sample size variation using a randomization approach. Specifically, we extracted subsamples of five individuals from each population 1,000 times with replacement and calculated the number of polymorphic loci in each subsample. Five samples were extracted from each population because this corresponds to the lowest sample size in our dataset  calculated from our data using the program Arlequin version 3.5.2.2. As Arlequin cannot incorporate loci with missing genotypes into SFS calculations, we used a subset of 92 individuals genotyped at 15,712 SNPs comprising 22 P. jacobeus, 50 P. maximus from the Atlantic group and 20 P. maximus from the Norwegian group. The aims of this analysis were to estimate the divergence time of P. maximus and P. jacobeus and to investigate the origin of the Atlantic and Norwegian P. maximus groups by formally testing two alternative hypotheses proposed by Morvezen et al.13, one assuming expansion from a single glacial refugium and the other assuming that the two groups originated from separate glacial refugia.To estimate divergence times of the main European scallop lineages and to evaluate historical effective population size changes, we implemented demographic analysis using the coalescent simulator P. jacobeus and the second to P. maximus, which diverged tdmj generations ago. Subsequently, at tdiv generations ago, the P. maximus lineage split into the Atlantic and Norwegian groups as a result of two independent recolonization events originating from separate glacial refugia. To account for historical population expansion in both P. maximus and P. jacobeus30, we assumed population expansion for all three demes. The second model , the historical effective population size of the P. maximus lineage that later split to form the Atlantic and Norwegian groups (Ne-hist-MAX), and the historical and contemporary effective population sizes of the Atlantic (Ne-hist-ATL and Ne-cur-ATL) and Norwegian groups (Ne-hist-NOR and Ne-cur-NOR). All priors for these analyses are given in Supplementary Table In order to test these two hypotheses, we evaluated support for two alternative demographic models. The first model  as described by Varin & Vidoni66 to allow model comparison. We then extracted parameter estimates from the best run of the model receiving highest support together with their 95% confidence intervals obtained via parametric bootstrapping as described in Excoffier et al.34. Specifically, we simulated 100 folded site frequency spectra based on the parameters estimated from the best model and used them to re-estimate models parameters following the same procedure described above.For each model, we performed 50 independent http://marine.copernicus.eu) data for chlorophyll concentration, dissolved oxygen concentration, salinity, sea bottom temperature and sea surface temperature for each of the 14 sampled locations. We then computed minimum, maximum and mean annual values for each of these environmental factors to obtain a total of 18 environmental variables. We then tested which of these environmental variables best explained genetic variation among the sampled scallop populations by using gradient forest analysis as implemented in the R package gradientForest23. For this analysis, in addition to the 18 environmental variables, we included latitude and longitude information to assess the importance of geography. Gradient forest is a machine learning regression tree based algorithm that searches for breaks across the environmental predictors. This method was originally implemented to study species turnover in community ecology datasets and was later adapted to detect allelic turnover in genomic data24. Gradient forest provides a ranked list of the relative predictive power of all environmental variables and therefore makes it possible to identify those environmental variables that best explain genetic variation.We extracted from the Copernicus Marine Service Information  to detect loci showing unusual associations with environmental variables compared to the genomic background. It accounts for the underlying population structure by introducing latent factors while simultaneously estimating random effects due to population history and isolation by distance. We ran 10,000 iterations of the Gibbs sampling algorithm with the first 5,000 iterations discarded as burn-in. Z scores from five independent runs were then combined and the resulting p-values were corrected for the false discovery rate according to Benjamini & Hochberg67 with an alpha level of 0.01. The second approach uses a Bayesian hierarchical model approach to identify loci associated with environmental variables, while explicitly accounting for the neutral covariance structure across population allele frequencies by incorporating a covariance matrix of population allele frequencies resulting from their shared history. We ran BayPass using default parameters under the standard covariate model and identified loci associated with environmental variables as those showing a Bayes Factor greater than 10, which is indicative of strong evidence36.We then employed the R package LEASupplementary Information


Investments in watershed services (IWS) programs, in which downstream water users pay upstream watershed service suppliers for actions that protect drinking water, are increasing in number and scope. IWS programs represent over $170 million of investment in over 4.3 million ha of watersheds, providing water to over 230 million people. It is not yet fully clear what factors contribute to the establishment and sustainability of IWS. We conducted a representative global analysis of 416 of the world’s largest cities, including 59 (14%) with IWS programs. Using random forest ensemble learning methods, we evaluated the relative importance of social and ecological factors as predictors of IWS presence. IWS programs are more likely present in source watersheds with more agricultural land and less protected area than otherwise similar watersheds. Our results suggest potential to expand IWS as a strategy for drinking water protection and also contribute to decisions regarding suitable program locations. Investment in watershed services programs is growing, however the factors that contribute to sustainability of such programs are unclear. Here the authors use a large database of cities around the world to show that payment schemes are more likely to be present in watersheds with more agricultural land and less protected areas. Recent studies predict an increase in the number of large cities vulnerable to water stress from 35 to 45% in the next 25 years2. Watershed degradation and water treatment costs have increased throughout the 21st century2, and existing water governance systems and engineering responses are proving inadequate and unfeasible both environmentally and economically. This is particularly acute in large cities, where large populations and population densities concentrate large numbers of people dependent on the same water supplies. The increasing demand on water resources, including exogenous factors such as those related to accelerated climate change, are driving new management strategies to sustain the provision of watershed services6. As water supply concerns grow, policies and tools must adapt to these new contexts9.Urbanization forecasts estimate that six of ten people will live in cities by 203010. Policies that involve payments or investments in watershed services (IWS) are incentive-based investment mechanisms designed to address the provision and enhancement of water-related ecosystem services12. As IWS have increased in scope, scale, and geography, so has research and monitoring of these policies and the underlying resource issues13. In 2015, Forest Trends’ Ecosystem Marketplace, which tracks longitudinal global data on market-based environmental programs, reported over 400 programs actively investing in watershed services around the world. These programs are associated with cities of all sizes with a variety of management goals, and total over $25 billion transacted among all programs globally, covering a land area larger than the size of India to date11. This represents a marked increase in the number, geographic scale, and funding of IWS from the 127 programs reported in 201011. Empirical evidence of the broad impact of payment for ecosystem services studies shows positive, though often small, impacts on environmental and social outcomes14. Here we focus on the presence of IWS programs for cities rather than an evaluation of their impacts.Management innovations have begun offering solutions to problems pertaining to water scarcity, quality, and threats to availability15. A variety of disciplinary perspectives such as those from economics18, political science20, and ecology21 hypothesize specific factors that influence the establishment of ecosystems services strategies such as IWS. We define these enabling conditions as factors that increase the likelihood of a change in governance approach, strategy, or management regime22. Though much of the research on enabling conditions for ecosystem services programs has been theoretical or has only evaluated specific cases, a recent synthesis of the literature on payments for ecosystem services programs identified 24 distinct enabling conditions  in aggregate23. On the basis of our analyses of enabling conditions, we also demonstrated how to use our results to identify large cities that are suitable candidates for future IWS programs. Our study of global patterns and enabling conditions for IWS provides guidance to policymakers, planners, conservation practitioners, and researchers to develop and evaluate programs where important enabling conditions already exist and foster favorable conditions in areas where key enabling conditions do not yet exist.To address the gap in our understanding of conditions that enable IWS in large urban areas, and to test whether there are generalizable conditions that enable IWS, we synthesized a global data set of city water supplies, creating the first assessment of urban IWS programs for 416 of the world’s largest cities . Using a random forest machine learning algorithm, we tested 17 data sets representing 15 of the 24 identified enabling conditions ; 53 of these were in the CWM. We improved the representativeness of our city sampling by first using a sample of cities from the CWM database that was stratified according to city size and UNPD geographical region, and then combining this with the data from a comprehensive survey of IWS programs and a review of published literature on IWS programs. Program information was derived from Forest Trends census surveys15. As opposed to sampling, Forest Trends aims to conduct a census , and uses the data from other sources  when an interviewee is not available to complete the survey for the program. Overall, IWS programs represent over $170 million of investment in over 4.3 million ha of watersheds, providing water to more than 230 million people. Analysis included cities in the Americas, Europe, China and SE Asia, Indonesia, and SE Africa  developed by McDonald et al.We used random forest analyses to rank 17 predictor variables, each representing one or more key enabling conditions described in Fig. 11. Other enabling conditions, such as population  and enforceability of contracts (a World Bank Indicator14 used as proxy for both ability to enforce IWS agreements and secure land tenure) were relatively important in both models. Both models ranked water diversion volume and watershed population density as not important in predicting the presence of an IWS program relative to other variables.The two most important enabling conditions in both models were percent of watershed with agricultural land cover and percent of watershed area designated as protected Fig. . Both moLess variation in a variable would result in a lower importance ranking because it would be harder for the algorithm to differentiate outcomes. Anticipation of this relationship partially motivated our choice to build the Non-USA Cities model. For example, when applied to the Global Cities Model, the random forest algorithm identified other variables not based on national scale data as having greater predictive power Fig. . VariablWeighted Drought Vulnerability is ranked important for Global Cities and not important for Non-USA Cities, indicating that drought vulnerability is more associated with IWS programs in the USA than elsewhere. However, drought could be a driving factor to search for policy and program innovations only when other enabling conditions are already in place. Such interactions among variables potentially explain some differences between the important conditions found in our models. Enabling conditions may interact within each model, such that different values of one condition may impact the importance of other conditions. The following section describes additional analysis of the behavior of individual conditions within each model.Enabling conditions are sorted into four general categories: biophysical, economic, governance, and sociocultural conditions Fig. . While sWe used partial dependence plots for the random forest models to explore how individual enabling conditions could predict IWS programs across each range of values in the representative data  but a high percentage of protected area (82.1%). This situation reflects a history of land acquisitions by the Seattle Public Utilities, which now owns and protects a large portion of the watershed. In other countries such as Mexico, China, South Africa, and Colombia, we found additional cities with IWS programs even though important enabling conditions were not present. In some cases, as with Colombia, where four of their seven large cities employ IWS, there may be national level programs, legal instruments, or concerted NGO efforts to initiate and support IWSConversely, having important enabling conditions in place is not sufficient to ensure presence of an IWS program. Our database also contains examples of cities that do not have IWS programs even though they have high levels of enabling conditions, such as the candidate cities we identified. Having enabling conditions in places with no IWS program could indicate presence of a different management strategy that successfully achieves the same outcomes of protecting urban water supplies. For example, cities and countries could have alternative policies or management practices in place, from strong regulatory frameworks or more voluntary measures such as source water protection plans.32. Places where ecosystem services have clear benefits to human communities are more likely to protect ecosystem services, since beneficiaries have incentive to compensate service providers35.In our assessment of 416 cities with over 1.15 billion drinking water consumers, conditions representing a range of sociocultural, governance, biophysical, and economic factors were important for IWS presence. In comparing all major cities to only those outside the United States, two suites of important enabling conditions emerged in particular. We found that key enabling conditions for IWS programs in major global cities include the amount of watershed area in (1) agricultural land use and (2) protected designations. In general, threats or risks to ecosystem services can facilitate the development of IWS by increasing awareness of ecosystem service benefits and their need for conservation36. Upstream agricultural land could further be associated with IWS programs and other environmental policy interventions because it can impact urban drinking water supply quality. Thus these land uses also present a ready opportunity for organized management actions16. Our results support the proposed linkages between agricultural lands, impacts on downstream water supplies, and existence of payment programs such as IWS, which have been cited as key drivers of IWS programs in cities such as New York in the US and Quito in Ecuador15.Greater percentage of agricultural land in the watersheds serving a city was an essential enabling condition for all of the cities in our sample and for the cities outside the USA Fig. . Previou14. Watersheds with large percentages of protected areas may not need further protection or incentives provided by IWS programs, so there is less motivation to develop programs in these locations. Additionally, source watersheds with a lot of land in the public domain may be easier to convert to protected status while watersheds with more private landowners or community-based tenure arrangements are a better target for IWS. In watersheds that have a low percentage of protected area, there may be increased opportunities for an IWS program as a way to influence management in the watershed and enhance water provision services via interventions on privately-managed land. Establishment of protected areas may also face additional hurdles in watersheds with large amounts of agricultural land37, leading water managers to seek out market-based approaches such as IWS. Finally, a watershed with high percentage agricultural land and low percentage protected area could indicate increased risks to provisioning water services that have potential impacts to downstream water users.The percentage of protected area in source watersheds was the second highest ranking enabling condition in both models—although the relationship was negative. As the percentage of protected area increased, the probability a city had an IWS program decreased. IWS programs are designed to provide land owners with incentives to protect or enhance the watershed for the provision of water services38. Our analysis outside the US identified per capita GDP as positively associated with IWS presence, while conservation spending was negatively associated. The positive association with GDP could potentially suggest that a certain level of affluence is needed for IWS and that water providers are not able to spread the cost to users when users are predominately poor. For conservation spending, it may mean that when spending is high, there are co-benefits to water quality coming from other investment actions that make IWS less necessary, similar to our interpretation for the negative relationship with the percentage of protected areas in the watershed. Where information on the cost effectiveness of various alternative strategies (including IWS) is available, it would enhance understanding of program emergence and sustainability.Our research is one of the first attempts to quantitatively evaluate enabling conditions for IWS programs in cities across the world. Previous research on IWS has focused on individual or limited numbers of cases rather than global patterns. As a global-level analysis, our research begins to fill this gap by broadly testing factors associated with the existence of IWS programs. Our results should not, however, be interpreted as a mandatory or static checklist of all necessary factors to implement IWS or a similar policy. Though various conditions predict IWS presence, it is possible for IWS programs to emerge in a variety of situations. We identified the contextual conditions present in areas where IWS interventions already exist, and the conditions that were less relevant to the existence of IWS. Contextual details about the mechanisms underlying the emergence of IWS are important for understanding enabling conditions in specific places, even if the finer scale conditions vary. Conservation practitioners, in particular, could add from their experiences in the field to improve our understanding of what local conditions facilitate both the emergence and sustainability of IWS programs. For example, the lower cost of implementing IWS schemes as compared to other policy tools is a known factor in their emergence27, and within the US some major water utilities pay upstream landowners to change management practices  or purchase land in the watershed (Seattle Cedar River Watershed).Future research can further evaluate enabling conditions across a variety of contexts and scales to help establish clear relationships. Documenting and monitoring IWS performance are important for providing data to test the mechanisms through which enabling conditions are associated with the existence and sustainability of programs. Sub-national data and analyses would improve our ability to test enabling conditions and validate theory. Our results provide insights about general patterns and broad trends for large cities, but the nature of global synthesis can mask relationships between conditions that could explain variation within a region. Regional or country level analysis could provide more details about the mechanisms underlying how a program was operating (e.g. successfully or not) and the factors that are most important in initiating IWS programs, but our global-scale analysis is not designed or poised to take advantage of this higher resolution sub-national scale data. For example, funding sources, investors, and supporting organizations differ among IWS programs in Latin America39 call for the need to document initial baseline conditions, including the initial state of threats to services and important factors that will forecast service trends in the beginning of a program, and Ingram et al.40 distill lessons learned about the use of ecosystem services, especially around understanding necessary institutional factors needed where governance may be weak. Recent research on PES programs more broadly identify key characteristics of buyers, sellers, and program specific metrics as key determinants of the spread and uptake of PES13.Local conditions and context indeed matter for more nuanced analysis and local application of findings; however, understanding the general conditions that can make it more likely that a program will emerge is an important step in understanding where and how to dig deeper into finer grain analysis. Understanding the general conditions can provide partial explanation of program presence and evaluate the potential scope for expanding programs. Numerous researchers in this field have come to similar conclusions, in particular that the time is ripe to collect previously disparate lessons learned from case studies of ecosystem services and synthesize them for broader general conditions impacting presence of IWS . For example, Naeem et al.Implications and implementation of research on natural resource management is critical for practitioners. We have been working with collaborators at The Nature Conservancy  on how to use the findings from this research to improve their IWS development program. When comparing potential locations for program investment, the most important conditions can be used to evaluate where IWS programs are likely present in comparable locations. In evaluating cities for program development, those that have similar characteristics to cities that do have a program may be good candidates. We provide an example process in Supplemental Note IWS programs emerge out of the interplay among numerous factors in complex social-ecological systems. What works in one place may not work in another because of the unique social and ecological contexts in each place. Our study takes an empirical approach in examining broad and globally available evidence on IWS programs and their enabling conditions. To elucidate particular conditions that enable innovative solutions in natural resource management, we emphasize that further cross-disciplinary and sub-regional investigations are needed.25 (CWM), a database by The Nature Conservancy containing information on large cities and their source watersheds. The original city list for CWM started with the World Urbanization Prospects (WUP) report conducted by the United Nations Population Division1 that lists all current and previous world cities with a population > 300,000. Cities below this population threshold were added to the CWM from research on 225 cities with populations over 100,000 in the United States2. Data on the source watershed and specific withdrawal information was collected by searching water utilities directly, though in some cases no information was found. The final City Water Map list of cities contains 534 cities, including the world’s 50 largest urban areas, the largest urban area in each country with > 750,000 people, and a representative sample of cities stratified by both geographic region and population range1.The city water map: Our list of 534 global cities comes from the City Water Map, version 2.31. The data were subset by removing all USA cities that met either of the following two criteria: (1) a population < 300,000, OR (2) no population data was available. This is based on the city population limit of 300,000 from the World Urbanization Projects report by UNPD2 that the CWM database used to develop their database. Most of the cities in CWM under the 300,000 threshold were additions to the WUP report and creating this cutoff reduced the data set by almost 100 USA cities.The CWM database contains a known bias resulting from data accessibility and availability that oversampled USA cities and undersampled Indian and Chinese cities11 (29 cities identified) and a literature review of IWS programs to identify 59 cities in the CWM that have an IWS program using search engine Web of Science and publishing service ScienceDirect. A search was conducted for title, abstract, and keywords only using the search terms “payment* for ecosystem services” OR “payment* for environmental services” OR “payment* for water* services” AND “water*“. Web of Science results listed 136 articles and ScienceDirect returned 91, which, excluding duplicated articles, produced a library of 171 articles.Data on existing IWS programs were gathered from several sources. We analyzed the 416 cities that met the UNPD criteria for large cities . We used Forest Trends’ State of Watershed Investments bi-annual report11.The Forest Trends report and literature search were reviewed for IWS programs that met two criteria; (1) they provide water for a city in the CWM database, and (2) drinking water protection is specified as a program goal. The list of cities that have met the IWS criteria include those with Demonstration Projects that are focused on drinking water because they are actively managing drinking water using a IWS program. For this research, cities with IWS programs (CityIWS) are those cities within the CWM database with a IWS program identified by either the Forest Trends report and/or the literature review  between two or more parties that compensate a land manager for protecting drinking water supplies for urban beneficiaries22. Our list of enabling conditions built on a synthesis of theory and case studies on payments for ecosystem services conducted by coauthors on this paper22. We identified global data sets for the variable  or, when necessary, for a proxy indicator that represented the variable (e.g. Property Rights Index represents land ownership and access). We intentionally targeted data for all four condition categories  identified by Huber-Stearns et al. in an attempt to represent as many different types of potentially important characteristics as possible. All city data is available in Supplementary Data Much of the program information was collected from the State of Watershed Payments annual report, produced by Forest Trendsase, and  drinking22. Enabling conditions are defined as factors that increase the likelihood of an intended change in the governance approach, strategy, or management regime. Enabling conditions, by definition, facilitate the emergence or sustainability of a particular environmental policy, while the absence of key enabling conditions can present a barrier to management or sustained policy action. In this initial publication we summarized existing literature on the concept of enabling conditions and synthesized the information into a list of potential conditions, grouped by category ; mixed sources ; groundwater sources only, or no available data. Surface Water includes all diversion types except groundwater and alluvial aquifers. Watershed area was calculated as the combined area (km2) of all watersheds and groundwater basins being used for drinking water for each city. Percentage of protected area is from IUCN-designated protected lands within this total area46. Land cover types  were calculated for the source watersheds and basins44 of each city and grouped based on classification per Supplementary Table The origin of the water supply for each city and characteristics of the watersheds were described using CWM diversion type categories and volumes, combined with delineations of the surface and groundwater basins that serve each city45. Using the World Urbanization Projects Report (WUP)1 the proportion of cities within each geographical region was calculated for each of 5 city population classes1 . Collinear and replicated variables were excluded. Selection was based on analysis of spearman pairwise correlations and variance inflation factors47. The R package Corrgram v1.1048 was used to calculate correlation coefficients. Of the 30 variables tested, 18 were found to be correlated with at least 1 other variable at corr > 0.7, indicating high collinearity17. Supplementary Table Thirty candidate variables from existing data sets were identified to represent potential enabling conditions as identified in Huber-Stearns et al.48 using the full database as well as a subset of the cities contained only non-USA cities, though not all variables could be included because of missing values. VIF is calculated as 1/(1-R2) from a linear model and estimates how much the variance of a coefficient is inflated from linear dependence with other predictors. A higher VIF value indicates that the variance (the square of the standard error) is larger than if the predictor were not correlated with other predictors. VIF were calculated iteratively by sequentially dropping the predictor with the largest VIF, recalculating with the remaining variables, and repeating until returned values were under the preselected VIF threshold of 349. Supplementary Table In addition to the spearman correlation coefficient, variance inflation factor (VIF) was calculated using R package carThree representative data sets  did not meet the VIF criteria, but were included in the model analysis because there were no other proxy variables for the EC variable they represented. After reducing both the number of cities and the representative data, the final database used for analysis contained 416 cities and 17 variables, representing 14 of the EC variables described by Huber-Stearns et al.  the smaller class (cities with a IWS program) was upsampled, and (3) new minority class were created using the Synthetic Minority Over-sampling Technique (SMOTE) function54 in R package DMwR55, and (4) weights were incorporated in the random forest classifier which made the classifier cost sensitive and penalized the model fit for misclassifying the minority class. The final models reported here addressed class imbalance by incorporating a weight class in the random forest model as this method produced models with higher predictive ability (predictive performance described below) better than other options.The data were split 80/20 (pareto principle) for training and test sets and not transformed. We weighted enabling conditions data to represent city distribution regionally and globally using UN statistical region boundaries (described above in the section titled Post stratification weights). Given the unbalanced nature of the dependent variable (IWS presence or absence), several strategies were attempted to address potential bias in the model due to the small size of the minority class (only 11.5% of modeled cities contain an IWS program because some cities were not included in the models). To address this class imbalance, the data were adjusted four different ways before modeling: (1) the larger class (cities with no IWS programs) was undersampled56, with the number of preselected variables (mtry) set to 4  and all other parameters set to default. We used the unbiased permutational variable importance measure (function varimpAUC), because it is particularly suited for unbalanced response classes58. The varimpAUC output is also a non-conditional variable importance measure that can be computed with missing data52. With this approach correlations between predictor variables need to be addressed separately, which was done using a spearman correlation test as described in the Variable selection section above. Predictor ranking was evaluated using mean decrease accuracy because of varying scales of measurement, and correlation among predictors59. Predictive power of each model was evaluated using a standard metric, area under the Receiver Operating Characteristic curve (AUC), which assesses classification accuracy56. Values for AUC range from 0.5 to 1 and the closer to unity, the more accurate a model, where models with a value of 0.7 are considered reasonable and those with values > 0.8 considered strong60. Both models presented here performed at AUC values of > 0.7. The relationships between individual predictors and outcome (presence of IWS) was evaluated using partial dependence plots via R package mlr61. Partial dependence plots reveal the relationship of individual conditions within each random forest model by integrating out (and thus controlling for) other factors. Greater y values indicate that an observation for a specific variable is associated with higher probability for classifying a city as having an IWS program. These plots depict the marginal effect of the variable to provide an average trend of individual variables within a model by integrating out all other variables63.Each model contained 8,000 treeshttps://github.com/cromulo/IWS.All data and code are available on GitHub (10.5281/zenodo.1403842) including the code used for the shiny app reference in the data availability section. Link: Supplementary InformationPeer Review FileDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Supplementary Software


Background: Severe disease associated with respiratory syncytial virus (RSV) infection occurs predominantly among infants under 6 months of age. Vaccines for prevention are in clinical development. Assessment of the vaccine effectiveness in large epidemiological studies requires serological assays which are rapid, economical and standardised between laboratories. The objective of this study was to assess the agreement between two enzyme linked immunosorbent assays (ELISA) and the plaque reduction neutralisation test (PRNT) in quantifying RSV specific antibodies.Methods: Archived sera from 99 participants of the Kilifi Birth Cohort (KBC) study (conducted 2002-2007) were screened for RSV antibodies using 3 methods: ELISA using crude RSV lysate as antigen, a commercial RSV immunoglobulin G (IgG) ELISA kit from IBL International GmbH, and PRNT. Pearson correlation, Bland-Altman plots and regression methods were used in analysis.Results: There was high positive correlation between the IBL RSV IgG ELISA and PRNT antibodies (Pearson r=0.75), and moderate positive correlation between the crude RSV lysate IgG ELISA and PRNT antibodies (r= 0.61). Crude RSV lysate IgG ELISA showed a wider 95% limit of agreement  with PRNT compared to the IBL RSV IgG ELISA . Mean PRNT titres were estimated within a width of 4.8 log2PRNT and 5.6 log2PRNT at 95% prediction interval by IBL RSV IgG and crude RSV lysate IgG ELISA, respectively.Conclusion: Although, the IBL RSV IgG ELISA is observed to provide a reasonable correlate for PRNT assay in detecting RSV specific antibodies, it does not provide an accurate prediction for neutralizing antibody levels. An RSV neutralising antibody level is likely to fall within 2.4 fold higher and 2.4 fold lower than the true value if IBL RSV IgG ELISA is used to replace PRNT assay. The utility of an ELISA assay in vaccine studies should be assessed independent of the PRNT method. The most severe disease occurs among infants under 6 months of age6, making this group the principal target for RSV disease prevention.Respiratory syncytial virus (RSV) is an important cause of annual epidemics of bronchiolitis and pneumonia in children less than five years of age worldwide7, is now undergoing phase 3 clinical trials (NCT02624947). Standardised serological assays are needed if this or other vaccines show good efficacy as they go to effectiveness phase 3b/4 clinical trials.To date there are no licensed vaccines for RSV. However, strategies to protect the young infant through immunisation are underway. One approach showing particular promise is to protect the infant over the first few months of life by maternal vaccination. A maternal RSV vaccine which is based on a post-fusion F protein nanoparticle design13. This is because the assay determines the level of antibodies present that are presumed to be functionalin vivo14. For instance, a pseudovirion-based neutralisation assay was found to detect all neutralizing antibodies that directly arrest the virus and thus to provide protection against infection with Human papilloma virus15. A study by Piedraet al. showed that serum antibody titres correlate with protection8; a finding also supported by Stansbelleet al. who showed that neutralisation antibody titres were inversely associated with RSV hospitalization in infants <6months16. Hence PRNT is considered the most appropriate serological assay for use in quantifying protective immunity correlates to RSV.The plaque reduction neutralisation test (PRNT) has been the preferred technique by most studies to determine the correlates of protective immunity to RSV17; (ii) the assay procedure is cumbersome - taking four to six days (according to method) to complete; and, (iii) the assay is costly in terms of laboratory technologist time. Hence, the PRNT is not ideal for large sero-epidemiological studies or large-scale vaccine trials.Although the PRNT is considered the “Gold standard” method for quantifying the level of RSV specific antibodies, it has several limitations: (i) PRNT antibody titres are currently not internationally standardised and therefore it is difficult to compare results across different studies, though work is under way to resolve thisDue to these limitations, it would be beneficial to adopt a serological assay which is quicker, cheaper and can be easily standardized, for use in large vaccine studies. An ELISA method is an option. However, the method measures epitopes of the target antigen and does not discriminate antibodies specific to neutralizing sites. Presently, there are two RSV specific IgG ELISA assays in our laboratory: an ELISA which uses crude RSV A2 culture lysate as antigen (Crude ELISA), and a commercial ELISA (IBL ELISA) which uses 96 well micro titre ELISA plates pre-coated with RSV F protein and RSV group A cell extract as antigen. We examined the agreement between ELISA assays and the PRNT in quantifying RSV specific antibodies using serum samples from a birth cohort in Kilifi, a coastal part of Kenya.20. Archived serum and plasma samples were utilised arising from the Kilifi Birth Cohort (KBC) study conducted between 2002 and 2007, by the Kenya Medical Research Institute-Wellcome Trust Research Programme (KEMRI-WTRP). The KBC study was an observational study to investigate susceptibility to invasive pneumococcal disease among children. In this study, newborn participants numbering around 6000, recruited at the maternity ward or the child health clinic of Kilifi County Hospital (KCH), were followed for a 2-year period with blood samples collected at birth and then every 3 months. Samples were stored as plasma (for cord bloods) or serum (follow-up samples) at -80°C. Details of the birth cohort study are described elsewhere19.This study was conducted in Kilifi, within the Kilifi Health and Demographic Surveillance System (KHDSS) in the coastal part of Kenya10.In the present study, a random sample set of 100 individuals from the KBC participants recruited from the KCH maternity ward, were selected from the study database. Description of the levels of maternal RSV specific neutralizing antibodies from blood samples of the 100 participants were provided in a previous publicationAll parents and guardians gave written consent to have their children participate in the KBC study for storage of blood samples for use in future research. The use of the archived sample set was approved by the KEMRI-Scientific and Ethics Review Unit (SERU# 2307).Archived plasma/serum samples from the KBC study which had been stored at -80°C were retrieved and screened using 3 serological assay techniques: (i) a PRNT (ii) an IgG ELISA assay using crude RSV lysate as antigen (crude ELISA) and (iii) an RSV IgG ELISA using a commercial ELISA kit (IBL ELISA).Plaque reduction neutralisation test (PRNT). The PRNT procedure for determining the titre of RSV neutralising antibodies has been described previously9. The method incorporated a step in which serum samples were incubated at 56°C in a water bath for 30 minutes to inactivate complement cascade proteins. Each serum sample was repeatedly diluted 2-fold over ten consecutive dilutions and mixed with an equal volume of 50 plaque forming units (pfu) of RSV A2 virus . The virus-serum mixture (50µl per well) was dispensed over a confluent monolayer of Hep2 cells in a 96 well culture plate, incubated at 37°C for 1 hour and then underwent 4-hour cycles of rotation on an angled (about 30°) rotating platform (about 40 rev/minute) for 10 minutes and incubation in a 37°C CO2 incubator for 30 minutes. The plate was then incubated for 48 hours in a 37°C CO2 incubator. Fixation of cells was done by the addition of 100µl of fixation reagent (30% methanol+70% acetone). Plaques were detected by addition of a primary antibody  solution diluted 1:500 in PBS with 2 hours incubation at 37°C, followed by an addition of a 100µl/well of an anti-mouse HRP-conjugated secondary antibody (170-5047 Immun-Star Goat Anti-Mouse (GAM)- IgG (H/L) polyclonal antibody HRP–BIORAD) solution diluted 1:1000 in Phosphate Buffered Saline (PBS) with 1 hour incubation at room temperature. After each step, plates were washed manually three times using 200µl/well PBS buffer. Plaques were visualised by addition of 100µl/well detection reagent. This consisted of 16 µl of hydrogen peroxide and 0.6ml of 3-amino-9-ethlycarbazole 3.3mg/ml solution (20mg 3-amino-9-ethlycarbazole tablet were dissolved in 6.06ml of dimethyl sulphoxide (DMSO) to give a 3.3mg/ml solution) to 10ml of 20mM sodium acetate solution (pH 5.0-5.5). Reading and counting of the brown-stained RSV micro-plaques was done using an ELISpot reader .9. In addition, a panel of control samples from BEI Resources  was included in each batch of the PRNT assay to monitor reproducibility of the assay results and deterioration of the antibodies used. Results obtained from screening of the BEI samples were compared with PRNT values of the samples as previously tested in BEI resources laboratories.The dilution of a test serum sample required to induce 50% neutralization of a known titration of RSV A2 virus was determined using the Spearman Karber methodCrude RSV lysate IgG ELISA (Crude ELISA). Blood samples were tested for antibody concentration with an IgG based ELISA method using crude virus extract from lab-adapted RSV A2 culture following a local standard operating procedure21. The crude virus RSV lysate preparation, optimal dilutions for RSV-A2 antigen, the serum dilutions and generation of a standard curve from a pooled adult serum were determined by a checkerboard titration as previously described22. In every run, one half of the 96 well plate (column 1-6) was coated with 50µl/well of RSV lysate (antigen), while the other half (column 7-12) was coated with 50µl/well of mock lysate (mock). The mock consisted of Hep2 cells without RSV virus prepared using same procedure as that of the RSV lysate. Plates were incubated overnight at 37°C, then blocked for 1 hour with 200µl/well of 5% skimmed milk at 37°C. Blocking buffer was flicked off. Diluted serum samples 100µl/well were dispensed to both the antigen and mock sides of the plate. The plates were washed 4 times with 200µl/well of 0.05% Tween 20 in PBS (PBS-T) using an ELISA plate washer. A secondary antibody [polyclonal antibody to human IgG heavy chains  (100µl/well) diluted 1:1000 in PBS buffer was added to each well and incubated for 1 hour at room temperature. The reaction was developed using 50µl/well of Ortho-Phenylenediamine dihydrochloride  solution as substrate (prepared just before use in the ratio 1mg of OPD in 1 ml of PBS and 1ul of hydrogen peroxide). The intensity of colour developed was read at 490nm using an ELISA reader . All samples were run in duplicate to monitor reproducibility of results and variability arising from pipetting. Positive and negative controls were run on every plate and plotted on a graph over time to check for antigen deterioration of the standards or coating antigen. The OD values of the mock were subtracted from the OD values of the antigen lysate to give the final OD value of the serum sample. OD values of the standard serum dilutions were assigned arbitrary unit (AU) values. Serum samples were assigned arbitrary units of RSV specific IgG by comparison against a standard curve generated from the pooled adult serum (serum standard) tested in each ELISA plate.Commercial RSV IgG ELISA (IBL ELISA). Use of the commercial RSV IgG ELISA kit and the procedure followed manufacturer’s guidelines as described in the package insert . The ELISA procedure was based on a sandwich principle. In this assay, the 96 well plates were pre-coated with Vero E6 cells; strain long, F-protein; cell extract of RSV subgroup A (long strain) containing F-protein as well as G-protein as antigen (IBL communication). Specific antibodies from the serum samples binding to the pre-coated antigen were detected by a secondary antibody, rabbit anti-human conjugated to horse radish peroxidase enzyme. The reaction was developed using Tetramethylbenzidine(TMB) solution as substrate. The intensity of the colour developed, which is proportional to the concentration of IgG-specific antibodies detected, was read at 450nm using an ELISA reader . The concentration of RSV IgG antibodies from the samples was estimated by comparison with a standard curve fit using a four parameter logistic regression model conducted in aGraphPad Prism software version 7.03.STATA version 13.1 . The laboratory data were merged with archived data from the KBC participant’ databases for analysis. Sample PRNT titres were logarithmically transformed (base 2) for all statistical analyses.Data analysis was conducted using2PRNT) and IgG ELISA (log10) antibodies was done using a Pearson correlation test. Agreement between PRNT and ELISA was done using Bland-Altman plots23. For each serum sample, the difference between PRNT and ELISA result was plotted against the averages of both assays. During Bland-Altman analysis, concentrations for RSV IgG IBL ELISA and crude lysate RSV IgG ELISA were log transformed to (base 2) and the data tested for normality. Modified Bland-Altman plots were used to test the effect of age on agreement between ELISA and PRNT. Regression analysis between ELISA and PRNT was used to estimate the 95% prediction intervals of PRNT antibodies by IgG ELISA method. The standard error (SE) of the predicted PRNT titre for an individual was estimated using the formula:Correlation analyses between PRNT antibodies  +/- 1.96SE, where SE is the standard error (see full methodshere).Where y is the predicted PRNT titre,2PRNT titres were: cord blood samples, 10.6 log2PRNT ; 3 months of age samples; 8.3 log2PRNT (95% CI 8.1 – 8.6); and 6 months and over 7.6 log2PRNT (95% CI 7.1 – 8.1). The highest PRNT titre observed among these participants was 12.9 log2PRNT while the lowest was 5.1 log2PRNT.Archived sera from 100 cohort participants were selected. However, for 1 participant, the samples were not sufficient to screen for ELISA IgG antibodies after screening for PRNT titres. A total of 229 samples  from the 99 participants were therefore available with data for the 3 serological assays. The mean gestational age at birth was 38.5 weeks (95% CI 37.6 – 39.3), while the mean birth weight was 2.9 kilograms (95% CI 2.7 – 3.0). The mean concentration of logThe mean PRNT titre results of a panel of 5 reference samples from BEI Resources tested in our laboratory were within range of the PRNT titre results as tested in BEI in 2011. These results are shown inA Pearson correlation test to determine the strength of association between ELISA antibodies and neutralizing antibody , showed 2 PRNT (95% limits 1.3-7.6) between PRNT and IBL ELISA and 2.1 log2 PRNT (95% limits -1.9-6.2) between PRNT and crude ELISA. Crude RSV lysate IgG ELISA showed a slightly wider 95% limit of agreement  compared to IBL RSV IgG ELISA .The distribution of log transformed RSV antibody concentrations were approximately log- normal for each ELISA dataset and for the PRNT data. Bland-Altman plots were drawn to assess the agreement between PRNT and ELISA in quantifying a high or low RSV specific antibody level from a sample . In thisModified Bland-Altman plots  show thex + 6.517), while the predicted PRNT values by crude ELISA were given as (1.419x +6.164). The 95% prediction limits for PRNT antibodies were slightly higher for crude ELISA +/- 2.79 log2PRNT compared to IBL ELISA +/-2.36 log2PRNT.Further assessment using regression analysis , estimatTo overcome the challenges experienced with PRNT as a technique for quantifying protective immunity correlates to RSV in large vaccine studies, we explored how well ELISA and PRNT methods agree in detecting levels of RSV specific antibodies, and in addition, investigated how accurate an ELISA method can predict the PRNT measurements and if it could be considered a suitable replacement for PRNT.25. Consequently, there is increasing interest to use the RSV F protein as the maternal RSV vaccine candidate antigen7. The moderate correlation with crude ELISA is likely caused by the wide range of antibodies against multiple RSV antigens the assay targets. This might have possibly led to detection of additional antibodies specific to other RSV proteins that are missed by both IBL ELISA and PRNT assay.We found moderate to good correlations between each ELISA (IBL (r =0.75) and crude (r=0.61)) and the reference method for neutralizing antibodies, PRNT. The stronger correlation in IBL ELISA could be explained by the fact that, both PRNT and IBL ELISA methods were designed to target specific antibodies against the RSV F and G protein. This is because, neutralising antibodies are presumed to dominate within the RSV F protein also commonly known as the key target for neutralizing antibodies29. Similar findings to our study were previously reported by Welchet al, who suggested that ELISA measure all antibody types and do not discriminate the neutralizing antibodies as measured by functional assays, thus cannot be relied upon to predict the neutralizing activity of the sera29. Due to the moderate correlation observed in our study between PRNT and crude RSV lysate IgG ELISA, we suggest that, careful consideration should be made on the choice of ELISA based assays as a surrogate for a neutralization assay in epidemiological studies.We identified few studies that show correlation between ELISA and neutralization assay32. One study which evaluated two commercially available ELISAs and one in-house reference laboratory ELISA for the determination of human anti-rabies virus antibodies29, suggested that the results from the Bland-Altman plot analysis can only offer an insight into the availability of alternative, less complex method to monitor antibody titres during vaccine studies.The Bland-Altman plots demonstrated that 96% of all samples fell into the 95% limits of agreement between PRNT and crude ELISA. However, the crude ELISA had a slightly wider limit of agreement with PRNT compared to IBL ELISA. The PRNT values were also consistently higher than for ELISA. The wider limit of agreement indicates that the crude RSV lysate IgG ELISA has a higher variability with PRNT compared to IBL ELISA. This again, raises the issue of specificity and affinity of binding for RSV specific IgG antibodies measured using the crude ELISA method. Very few studies have used Bland-Altman plots to assess agreement between two serological methods33. For instance, samples from infants and the elderly are likely to show a poor correlation between PRNT and ELISA usually because their receptors don’t activate appropriate T or B cell responses, although the antibodies bind to antigen33. However, in this study, both methods showed agreement in the level of detected antibody for an individual regardless of age.We also demonstrated that age did not have an appreciable effect on the agreement between PRNT and ELISA methods in detection of RSV specific antibodies. We performed this analysis because poor correlation between functional antibody and antibody concentration is thought to be influenced by age2 PRNT, while, crude RSV lysate IgG ELISA predicted the mean PRNT titre within a width of 5.6 log2 PRNT units. This implies that, the PRNT titre of any given serum sample if estimated using an ELISA IgG antibody concentration measured directly by the ELISA method would fall within 2.4-2.8 fold higher or lower than the true value. In a perfect regression, if RSV antibody concentrations were measured by ELISA, the corresponding PRNT values would be estimated using (1.880x + 6.517) and (1.419x +6.164) for IBL ELISA and crude ELISA respectively. With,x being the antibody concentration measured directly by the respective ELISA method. However, the high variability makes direct estimation of PRNT values by the above formula difficult. For instance, the 95% upper limit of detecting mean cord PRNT titre of 10.6 log2PRNT by crude ELISA would be 13.4log2PRNT, which is above the highest value (12.9log2PRNT) the PRNT assay detected in this study. The combined analysis approach we have applied on the data is unique and we think, this could better guide the choice of a suitable serological technique for use in place of a PRNT assay for large vaccine studies; and especially if the units of measurements between the two methods are different.In this study, we could predict PRNT titres using ELISA with moderate accuracy. Using regression analysis, the IBL ELISA method predicted the mean PRNT titre at 95% prediction interval within a width of 4.8logThe main limitation we highlight in this study is that the PRNT assay we are using is difficult to standardize, which complicates comparison of results between laboratories. However, during the time this study was conducted there was no RSV standard available. Nevertheless, we tried to mitigate this limitation by using a panel of reference serum samples from BEI resources to monitor and minimise assay variability and to make sure results of the test samples obtained are acceptable. Another limitation is the systematic bias observed with analysis using Bland-Altman plots. This was due to the differences in metrics used to measure RSV specific antibodies by ELISA and PRNT. The RSV specific antibodies in ELISA are measured as a concentration (log10) while in PRNT assay are measured as titres (log2). The Bland-Altman plots show a trend if analysis is done using the respective metrics of the serological assays, thus difficulty in estimating the mean difference. The plots in Bland-Altman analysis should demonstrate that the two methods are consistent in what they are measuring in the same metrics. We tried to minimise this by transforming ELISA values to log base 2 during Bland-Altman plot analysis.Both ELISA methods show a moderate-good correlation with PRNT in measuring RSV specific antibodies. However, a commercial anti RSV based ELISA antibody assay which has less variability and has a high positive correlation with PRNT assay in detecting RSV specific antibody responses, does not provide an accurate prediction of PRNT antibody values. The true PRNT value would lie 2.4 fold higher or 2.4 fold lower if a commercial RSV IgG IBL ELISA is used in place of a PRNT assay. Our results suggest that, should PRNT prove unsuitable as a Gold standard for quantifying RSV specific antibodies, the utility of an ELISA method in vaccine studies should be assessed independent of a PRNT assay.http://kemri-wellcome.org/aboutus/#ChildVerticalTab_15 for consideration by our Data Governance Committee (dgc@kemri-wellcome.org).The raw data is stored under restricted access and available from the authors upon request through submission of a request formhttps://doi.org/10.7910/DVN/K3LS7M34Harvard Dataverse: Replication Data for: Agreement between ELISA and Plaque Reduction Neutralisation Assay in Detection of Respiratory Syncytial Virus Specific Antibodies in a Birth Cohort from Kilifi, Coastal Kenya.This project contains the following underlying data:NEUT ELISA Analysis script.doNEUT ELISA data codebook.pdfNEUT ELISA data file.tabNEUT ELISA_Data Readme txt.txtCreative Commons Attribution 4.0 International (CC BY 4.0) LicenceData are available under Overall comment. The major objective of this manuscript was to determine if at least one of the two in house RSV ELISA assays could predict with accuracy the neutralizing antibody titers generated by a plaque reduction neutralization test (PRNT). Although the methodology used to make the comparisons and analytical approach used to make the prediction are sound, the basic assumption is flawed when using crude or poorly described viral antigens in the ELISA assays. The neutralization assay primarily measures neutralizing antibodies directed to site-specific sites on the F protein while the ELISA assay using viral antigens will measure binding antibodies direct to the surface glycoproteins  as well as the internal proteins . Although good correlations are normally seen between ELISA and neutralization assays, it is entirely a different proposition to predict with accuracy neutralizing antibody titers when using viral antigens in an ELISA assay. Even site-specific competitive antibody assays to the F protein are not able to predict with accuracy neutralizing antibody tiers measured by neutralization assays because the immune response of individuals vary to the antigenic sites  of the F protein even though they may have comparable neutralizing antibody activity. The lack of precision was evident in the results with the potential to have up to a 16 to 32-fold swing in the neutralizing antibody titer prediction using the ELISA assays.  Other comments. In the manuscript PRNT is described as the preferred technique to determine immune correlates of protection. It is important to point out that the PRNT is one of many neutralization assays that have been developed for measuring neutralizing antibody activity against RSV. There are two major classes of neutralization assays; the PRNT and the microneutralization assay. There are many assay variations depending on the cell line, the virus inoculum, the signal or read out used to measure neutralizing activity, the duration of the assay, etc. Many of the PRNT references used in the manuscript actually refer to the classical microneutralization assay. It is also important to note that these neutralization antibody assays, in particular those without complement, primarily detect neutralizing antibody directed to the F protein and not the G protein. Neutralization assays are considered the best current method to evaluate the protective potential  induced by RSV infection or candidate vaccines. We now have an international standard  against RSV/A  that can be used to adjust neutralizing antibody titers generated by different neutralization assays into IU. This will greatly facilitate the comparisons of neutralizing antibody activity measured by different assays and induced by different candidate vaccines. The introduction section should be revised to reflect these up-dates.Introduction.1. The Fusion (F) protein in Novavax F nanoparticle vaccine is not a true post-fusion F protein vaccine. It is consistent with a prefusogenic form that is able to generate site-specific antibody responses to both the pre-fusion and post-fusion F conformationsMethod. What was the collection tube used to process plasma? Plasma collected in heparin coated tubes will inaccurately measure plasma samples with low neutralizing antibody activity to RSV because heparin containing plasma will act as an antiviral agent. The effect is to inhibit virus replication when in reality there is no neutralizing antibody activity. This is unlikely to have an effect with plasma samples with moderate to high levels of neutralizing antibody titers against because the effect is diluted out with increasing two-fold dilutions. For the ELISA assays it is unclear what the dilution scheme was for generating the ELISA units. The results are presented in log10, which suggest a dilution scheme of 10-fold. Please clarify the dilution scheme for the two ELISA assays. Also for all the assays please describe the range of the antibody titer and arbitrary unit for the assays. It would also be helpful to know the coefficient of variation for these assays at the lower limit of detection, dynamic range and upper limit of detection. It is unclear the viral antigen(s) included in the IBL ELISA assay. The information is not provided in the IBL package insert.Results. It is a misnomer to call it an RSV-F ELISA. Please see Y axis for Figures 2 and 3.2? Figures 2 & 3, Y axis is the difference (PRNT – RSV F protein ELISA) in log What was the correlation between the two ELISA assays, and did one ELISA assay predict with good accuracy the AU of the other ELISA assay. That would serve as an internal validation of the statistical method used to predict neutralizing antibody titer with ELISA bind antibody data.Discussion.th paragraph, “predict with moderate accuracy”. Having a width of 4.8 log to 5.6 log2 is huge and should be considered imprecise. Page 8, 7 In my opinion, the main limitation is not the standardization of the PRNT but rather the assumption that accurate predictions can be made when using ELISA assays with poorly characterized viral antigens. A second major limitation is the use of immunological assays that do not appear to have been qualified. Information on the reproducibility, dynamic range, lower and upper limit of detection were not provided for these assays.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. This is an important study which addresses the question of whether RSV ELISAs can replace neutralizing antibody assays for use in vaccine trials. This study analyzes the agreement between and accuracy of measuring RSV specific antibodies using ELISA w/ crude RSV lysate and RSV IgG ELISA in comparison with the current “gold standard” method for measuring RSV specific antibodies, a plaque reduction neutralization test (PRNT). In previous studies, ELISA and PRNT have not been shown to correlate well, due to non-neutralizing antibodies.  Methodologically, the study is sound. Comparison was made between the PRNT, utilizing a method that has been used by multiple labs. Results from a standardized set of reference sera are included, showing performance of the test within range. Comparison was made against a commercial and in-house ELISA. Samples tested included sera samples from the cord blood of 99 participants and subsequent 3 and 6-month samples from a fraction of these participants. Strength of association was calculated with a correlation test and agreement between PRNT and ELISA were examined on Bland-Altman plots. Results demonstrated a relatively high correlation was seen between the crude RSV Lysate ELISA and PRNT, and a more moderate correlation was observed between the RSV IgG ELISA and PRNT. Overall, this is an important study in the setting of multiple new vaccine and antibody prophylaxis candidates in clinical trials, for which it would be immensely helpful to have a less expensive, less labor-intensive method of measuring antibody as a correlate of protection. Overall the commercial ELISA had higher correlation as compared to the ELISA using crude RSVA2 lysate, which as the authors point out, is likely due to targeting of F and G antigen in the commercial assay. Despite this higher correlation, the margin of error for the formula calculating the relationship between the ELISA and the PRNT is too high to be useful. This is a useful study, and reaffirms the use of RSV neutralizing antibody assays as the gold standard. One minor criticism, largely outside the scope of this study, is the consideration of other correlates of protection beyond neutralizing antibody as markers of protection against disease.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. This study evaluated the agreement between the RSV plaque reduction neutralisation assay (PRNT), a commercially available RSV IgG ELISA assay and an in-house RSV assay using crude RSV lysate as an antigen.Overall, the methods and results section of the manuscript are well written and analysed. Although the study objective is clear, the rational for the study is more ambiguous. The main issue at hand in the context of vaccine development, is the need for an assay which can predict the efficacy of the vaccine –and ideally a functional assay or at least one that bridges with a functional assay. This would be important for bridging studies, in the absence of being able to undertake multiple large scale vaccine efficacy/effectiveness studies. Although the authors cite some studies reporting a positive association of PRNT antibody and risk/odds of disease, omitted from the background are other studies (including one from the same group)- which actually reported no association between PRNT and RSV LRTI. This calls into question the value of the current “gold standard”- i..e PRNT- on its own (over and above the issues with regard to standardisation and being labour intensive) in relation to whether or not it would be useful for establishing a correlate of protection. In the context of this study where the ELISA assays, as also noted by the authors, are not discriminatory in terms of measure of antibody to epitopes specifically targeted by most of the vaccines currently under clinical development (including the nanoparticle F-fusion vaccine), the likelihood of any predictive value of a relatively non-specific ELISA assay being of use to establish or impute a correlate of protection would be doubtful from the outset. These issue merit inclusion in the abstract, study rational and the discussion.The measure of RSV antibody could have multiple purposes- including characterising sero-epidemiology, measuring vaccine immunogenicity or trying to establish a correlate of protection. Depending on for what purpose the assay is being used, would influence the minimum characteristics required of the assay. In the context of this study, although the ELISA assays were only moderately predictive of the PRNT antibody measure, this does not necessarily preclude the use thereof in sero-epidemiology studies. Likewise, even if there was a strong correlation between the ELISA and PRNT assays, in the absence of an established correlate of protection using the PRNT assay- that too would not be of much value in terms of being used for studies to impute vaccine effectiveness in different settings. Specific commentsPg 3 end para 2- need to clarify that the standardised assays which are required need to be assays that are predictive of vaccine protection. Also, clarify that the main purpose of such assays are to undertake “bridging immunogenicity studies” to avoid multiple vaccine efficacy/effectiveness studies. Such assays will need to be tested for predicting vaccine efficacy often within a clinical trial and the assays could differ based on the construct of the vaccine (i.e differences in specific epitopes targeted).Pg 3 para 3- mention and reference studies that have shown no association between PRNT and LRTI.Pg 3 para 4- clarify that the needs of an assay for sero-epi studies may well differ to those required to establish a correlate of protection.Pg 3 para 5- what would the need for a serological assay be in a “large vaccine efficacy trial” which will provide the most definitive answer on vaccine efficacy?Under lab methods- clarify where the lab assays were done.Pg 8 para 3- neutralizing antibodies are also elicited by immune responses to the G and M protein. This might well explain some of the conflicting findings on the association of PRNT and risk for RSV LRTI.Pg 8 para 4- the last sentence needs to be elaborated upon, including what the possibility for epitope specific antibody read-outs that might well differ between vaccine constructs.   General comments:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Can you please justify the sample size you selected for the purpose of this study?You mentioned that age did not have a substantial effect on the agreement between PRNT and ELISA methods. As age advances, will you expect the similar finding?What is your opinion of applying the similar ELISA methods quantifying RSV specific antibodies beyond young children? e.g. older adults?Will different clinical specimen make an influence on the agreement / accuracy of ELISA methods?What is the next step (research question) based on findings from this study?This study investigates how well ELISA and PRNT methods agree in detecting levels of of RSV specific antibodies  and whether an ELISA method can be an appropriate replacement for PRNT. Generally it is well written and I only have minor comments listed as follows:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.