Spinal cord injury (SCI) is a devastating condition that causes substantial morbidity and mortality and for which no treatments are available. Stem cells offer some promise in the restoration of neurological function. We used systematic review, meta-analysis, and meta-regression to study the impact of stem cell biology and experimental design on motor and sensory outcomes following stem cell treatments in animal models of SCI. One hundred and fifty-six publications using 45 different stem cell preparations met our prespecified inclusion criteria. Only one publication used autologous stem cells. Overall, allogeneic stem cell treatment appears to improve both motor (effect size, 27. 2%; 95% Confidence Interval [CI], 25. 0%–29. 4%; 312 comparisons in 5,628 animals) and sensory (effect size, 26. 3%; 95% CI, 7. 9%–44. 7%; 23 comparisons in 473 animals) outcome. For sensory outcome, most heterogeneity between experiments was accounted for by facets of stem cell biology. Differentiation before implantation and intravenous route of delivery favoured better outcome. Stem cell implantation did not appear to improve sensory outcome in female animals and appeared to be enhanced by isoflurane anaesthesia. Biological plausibility was supported by the presence of a dose–response relationship. For motor outcome, facets of stem cell biology had little detectable effect. Instead most heterogeneity could be explained by the experimental modelling and the outcome measure used. The location of injury, method of injury induction, and presence of immunosuppression all had an impact. Reporting of measures to reduce bias was higher than has been seen in other neuroscience domains but were still suboptimal. Motor outcomes studies that did not report the blinded assessment of outcome gave inflated estimates of efficacy. Extensive recent preclinical literature suggests that stem-cell–based therapies may offer promise, however the impact of compromised internal validity and publication bias mean that efficacy is likely to be somewhat lower than reported here.
Stem cells, from which all tissues can be generated, offer the potential to reconstitute tissues damaged by injury and disease. However, realising this potential will demand a detailed knowledge of the genetic and internal environmental cues that specify a cell' s type, location, and interaction with its neighbours. It will also require a thorough understanding of stem cell behaviour in the context of lesioned or damaged tissues. Stem cell transplantation was pioneered in the 1950s using haematopoietic stem cells to repopulate the bone marrow in patients with cancers of the blood and bone marrow [1]. Such is the success of this approach that an estimated 50,000 of these transplants are performed each year [2]. As understanding of stem cell biology has increased, so too has the ambition for restoring more complex tissues. In animal models, hepatocytes derived from stem cells can be engrafted into the damaged liver [3], and lineage-specific stem cells can repair damaged cornea [4], [5]. Recent studies also demonstrate the generation of artificial tissues with key features of complex solid organs including blood vessels [6], heart [7]–[9], lung [10], and kidney [11]. Even in the CNS, where the breadth of cell types and the complexity of their interactions are maximal, stem cell implants appear able to integrate into the existing circuitry [12]–[14]. In patients, lineage-specific stem cells have been reported to show efficacy in the regeneration of craniofacial bones [15] and of damaged cornea [5]. Integration into the host environment and tissue reconstruction are not the only potentially relevant biological effects of stem cells. Immunomodulatory effects of stem cells appear to reduce rejection of kidney transplants [16], [17], corneal allografts [18], and composite tissue hemi-facial allografts [19]. In the CNS, stem cells are reported to provide immunomodulatory and neuroprotective effects in models of diseases as disparate as retinopathy [20], neuronal ceroid lipofuscinosis [21], motor neuron disease [22], [23], Parkinson' s disease [24], multiple sclerosis [25], [26], stroke [27]–[29], and spinal cord injury [30], [31]. There is now considerable preclinical literature on the possible benefits of stem-cell–based therapies following traumatic spinal cord injury. Stem cells may assist recovery through limitation of secondary injury, re-myelination, formation of new neuronal connections, and alteration of the inhibitory environment. However, it is unclear which type of cells and from what source are best to implant, how many are needed, whether immunosuppression should be used, and whether the implanted cells need to be modified to enhance particular desirable characteristics. It is also unclear whether the magnitude of integrative and protective effects is large enough to be potentially clinically meaningful. We also do not know whether reports of efficacy in animal models are potentially biased in favour of positive results. Here, we report a systematic review, meta-analysis, and meta-regression of data from controlled in vivo studies testing the efficacy of stem cells as a treatment in animal models of spinal cord injury. Our objectives are (i) to establish a summary estimate of the efficacy of stem cells in animal models of traumatic spinal cord injury, (ii) to ascertain the conditions under which animal experiments demonstrate greatest efficacy, and (iii) to determine any effect of study quality on reported efficacy.
Electronic searching identified 156 full publications that met our prespecified inclusion criteria (Table S1). Forty-five different stem cell types had been investigated, from which over a third were derived from adult rats. The duration of experiments following the induction of SCI ranged from 7 d to 6 mo. One publication [32] with two individual comparisons involving 36 animals reported the effect of autologous bone marrow stromal cells on motor score. We included this publication in the overall assessment of the prevalence of the reporting of measures taken by the original authors to reduce the risk of bias in their experiments. However, because this was the only paper to report the effects of autologous (rather than allogeneic) stem cells, we did not analyse this further, focussing instead on allogeneic stem cells. One hundred and fifty-five publications reported the effect of allogeneic stem cells in 317 individual comparisons; 380 different motor outcomes were reported and because more than one motor outcome was reported for some individual comparisons we nested (see Methods) these into 312 individual comparisons involving 5,628 animals (Figure 1A). Six different tests were used to assess motor score: the Basso, Beattie and Bresnehan locomotor rating scale (BBB; [33]), the Basso mouse scale (BMS; [34]), the Tarlov scale [35], the forelimb placing test [36], the staircase test [37], and the mouse hind limb motor score [38]. Sixty-one sensory outcomes were reported; we excluded six outcomes that tested sensation in unaffected limbs. In 10 outcomes that used the same test at different intensities in the same cohort of animals, we only included the median intensity. Therefore, we report data on sensory outcome reported in 45 experiments nested into 24 comparisons using 473 animals (Figure 1B). In 18 cohorts both motor and sensory outcomes were reported. We describe the reporting of study quality checklist items reported for each included publication in Table S2. All studies included in this analysis came from peer-reviewed papers; while we identified a number of potentially relevant abstracts, none of these reported data in sufficient detail to be included. One hundred and eleven of 156 publications (71%) reported compliance with animal welfare regulations, and 25 (16%) reported whether or not a conflict of interest existed. Allocation concealment was reported in 14 of 156 publications (9%). Random allocation to treatment group (72,46%) and blinded assessment of outcome (72,46%) were reported more frequently in these publications than in the modelling of other neurological disorders [39]–[42], but the reporting of a sample size calculation (less than 1%) was consistent with the proportions observed elsewhere (Table 1). No publication reported all four of these measures to minimise bias. Despite the reported benefits of hypothermia in SCI [43]–[45], in other animal models of neurological disease [46] and in humans with ischaemic neurological injury [47], [48], only 33 (21%) studies described controlling temperature during the experiments. There were only sufficient data to assess publication bias in studies using allogeneic stem cells where outcome was measured as a motor score. Small study bias was suggested with asymmetry of the funnel plot (Figure 2A) and Egger regression (Figure 2B) but not by Trim and Fill. As expected, our search identified a diverse range of experiments. There was substantial between-study heterogeneity for studies using allogeneic stem cells both where outcome was measured as a motor score [heterogeneity (χ2) = 9,735,311 degrees of freedom (df), p<10−99; effect size, 27. 2% improvement in outcome [95% confidence interval, 25. 0%–29. 4%]; 312 comparisons) and as a sensory outcome (χ2 = 183, df = 23, p<10−26; effect size, 26. 3% [7. 9%–44. 7%]; 24 comparisons).
We define a “publication” as a discrete piece of work (including abstracts); each publication may report data from a number of experiments. Each experiment may describe outcome in a number of different experimental cohorts, and the contrast between outcomes in a single treatment cohort with that in a control cohort we define as an “individual comparison. ” We define “nesting” as combining the effect sizes from different functional outcomes measured in the same cohort of animals to give a single summary estimate of effect in that individual comparison (a nested individual comparison). Using prespecified inclusion and exclusion criteria we identified all publications reporting relevant experiments (see below) by searching (December 2011) three electronic databases (PubMed, EMBASE, and ISI Web of Science) using the search strategy “ (stem cell OR stem OR haematopoietic OR mesenchymal) AND (spinal cord injury OR hemisection OR contusion injury OR dorsal column injury OR complete transection OR corticospinal tract injury), ” with search results limited to those indexed as describing animal experiments. Two investigators (A. A. and E. S.) independently reviewed retrieved publications. We included experiments where functional outcome in a group of animals exposed to traumatic spinal cord injury and treated with allogeneic or autologous stem cells was compared with functional outcome in a control group of animals. We excluded individual comparisons that did not report (or where we could not calculate) the number of animals, the mean outcome, or its variance in each group. We excluded experiments where interventions such as growth factors were used to mobilise endogenous stem cells or where nontraumatic models of spinal cord injury were used. From each individual comparison we extracted data for reported outcomes. This included extraction of mean and variance data from each cohort exposed to an intervention (controls and active therapy) and from sham cohorts of normal (unlesioned and untreated) animals, and by imputation where the performance of a normal animal could be imputed from the description of the scoring scale. Stem cells were characterised as “autologous” where cells were extracted from an animal, might be manipulated in some way, then returned to the same animal; or “allogeneic” where embryonic or adult cells derived from a different animal were administered to a recipient animal. Where a publication reported more than one experiment, or where an experiment reported more than one individual comparison (for instance, increasing numbers of stem cells transplanted), we considered these separately and extracted data for each, correcting the weighting of these studies in meta-analysis to reflect the number of experimental groups served by each control group. Where different functional outcomes were reported in a single cohort of animals, we combined these outcomes using fixed effects meta-analysis (nesting), to give a summary estimate of functional outcome in that cohort, described here as a comparison. Where a test involved exposing the animal to increasing intensities of the same stimulus (for instance, in allodynia testing), we used data for the median intensity. For sensory tests, only data for stimulation distal to the lesion were included. Where functional outcome was measured at different times, we extracted data for the last time point reported. Study quality was assessed using a checklist adapted from good laboratory practice guidelines for in vivo stroke modelling [59] and the CAMARADES quality checklist [60]. The checklist comprised (i) publication in a peer-reviewed journal, (ii) statements describing control of temperature, (iii) randomisation to treatment group, (iv) allocation concealment, (v) blinded assessment of outcome, (vi) avoidance of anaesthetics with known marked intrinsic neuroprotective properties, (vii) sample size calculation, (viii) compliance with animal welfare regulations, and (ix) whether the authors declared any potential conflict of interest. For each individual comparison, we calculate a normalised effect size [normalised mean difference) as the percentage improvement (“+” sign) or worsening (“−” sign) of outcome in the treatment group using the following formula: where and are the mean reported outcomes in the control and treatment group, respectively, and is the mean outcome for a normal (unlesioned and untreated) animal. In this calculation, the score achieved by the sham animals acts as the “fixed zero value” or baseline allowing the difference between the sham and treatment groups to be expressed as a ratio. This ratio takes into account differences in the “direction” of individual neurobehavioural scales. Its corresponding standard error was calculated using: where refers to the number of animals in the control group and refers to the number of animals in the treatment group. and are the normalised standard deviations for the control and treatment group, respectively. These were calculated using the formulae: where SDc and SDrx are the reported standard deviation for the control and treatment group, respectively. We then used DerSimonian and Laird random effects weighted mean difference meta-analysis to calculate a summary estimate of effect size; results are presented as the percentage improvement in outcome and its 95% confidence intervals. The variability of the outcomes assessed is presented as the heterogeneity statistic (χ2) with n−1 degrees of freedom. The analysis was stratified according to (i) the approach to stem cell therapy (allogeneic, autologous, embryonic, source of cells, ex vivo manipulation), (ii) biological factors (number of cells, time and route of administration, time of assessment of outcome), (iii) aspects of study design (anaesthesia, species of animal, immunosuppression, model and severity of spinal cord injury), and (iv) elements of study quality. The extent to which study design characteristics explained differences between studies was assessed using meta-regression with the metareg function of STATA/SE10, and the significance level was set at p<0. 05. The meta-regression was univariate rather than multivariate; and we calculated adjusted R2 values (a measure of how much residual heterogeneity is explained by the model) to explain the proportion of the observed variability in the observed effect size for a group of experiments explained by variation in the independent variable in question [61]. We sought evidence of publication bias using a funnel plot, Egger regression, and Trim and Fill [62]. A detailed description of the statistical methods used for meta-analysis and meta-regression can be found in [63].
Spinal cord injury is an important cause of disability in young adults, and stem cells have been proposed as a possible treatment. Here we systematically assess the evidence in the scientific literature for the effectiveness of stem-cell–based therapies in animal models of spinal cord injury. More studies reported effects on the ability to move (“motor outcomes”) than on sensation (“sensory outcomes”). Overall, treatment improves both sensory and motor outcomes, and for sensory outcome there was a dose–response effect (which suggests an underlying biological basis). Although more measures were taken to reduce the risk of bias than in other areas of translational neuroscience, unblinded studies tended to overstate the effectiveness of the treatment. The variability observed between the studies is not explained by differences in the stem cells used, but does seem to depend on the different injury models used to emulate human spinal cord injury. This suggests that the mechanism of injury should be an important consideration in the design of future clinical trials. Furthermore, open questions arise about the use of immunosuppressive drugs, and efficacy in female animals; these should be addressed before proceeding to clinical trial.
Stem Cell Transplantation in Traumatic Spinal Cord Injury: A Systematic Review and Meta-Analysis of Animal Studies