A century after its discovery, Chagas disease still represents a major neglected tropical threat. Accurate diagnostics tools as well as surrogate markers of parasitological response to treatment are research priorities in the field. The purpose of this study was to evaluate the performance of PCR methods in detection of Trypanosoma cruzi DNA by an external quality evaluation. An international collaborative study was launched by expert PCR laboratories from 16 countries. Currently used strategies were challenged against serial dilutions of purified DNA from stocks representing T. cruzi discrete typing units (DTU) I, IV and VI (set A), human blood spiked with parasite cells (set B) and Guanidine Hidrochloride-EDTA blood samples from 32 seropositive and 10 seronegative patients from Southern Cone countries (set C). Forty eight PCR tests were reported for set A and 44 for sets B and C; 28 targeted minicircle DNA (kDNA), 13 satellite DNA (Sat-DNA) and the remainder low copy number sequences. In set A, commercial master mixes and Sat-DNA Real Time PCR showed better specificity, but kDNA-PCR was more sensitive to detect DTU I DNA. In set B, commercial DNA extraction kits presented better specificity than solvent extraction protocols. Sat-DNA PCR tests had higher specificity, with sensitivities of 0. 05–0. 5 parasites/mL whereas specific kDNA tests detected 5. 10−3 par/mL. Sixteen specific and coherent methods had a Good Performance in both sets A and B (10 fg/µl of DNA from all stocks, 5 par/mL spiked blood). The median values of sensitivities, specificities and accuracies obtained in testing the Set C samples with the 16 tests determined to be good performing by analyzing Sets A and B samples varied considerably. Out of them, four methods depicted the best performing parameters in all three sets of samples, detecting at least 10 fg/µl for each DNA stock, 0. 5 par/mL and a sensitivity between 83. 3–94. 4%, specificity of 85–95%, accuracy of 86. 8–89. 5% and kappa index of 0. 7–0. 8 compared to consensus PCR reports of the 16 good performing tests and 63–69%, 100%, 71. 4–76. 2% and 0. 4–0. 5, respectively compared to serodiagnosis. Method LbD2 used solvent extraction followed by Sybr-Green based Real time PCR targeted to Sat-DNA; method LbD3 used solvent DNA extraction followed by conventional PCR targeted to Sat-DNA. The third method (LbF1) used glass fiber column based DNA extraction followed by TaqMan Real Time PCR targeted to Sat-DNA (cruzi 1/cruzi 2 and cruzi 3 TaqMan probe) and the fourth method (LbQ) used solvent DNA extraction followed by conventional hot-start PCR targeted to kDNA (primer pairs 121/122). These four methods were further evaluated at the coordinating laboratory in a subset of human blood samples, confirming the performance obtained by the participating laboratories. This study represents a first crucial step towards international validation of PCR procedures for detection of T. cruzi in human blood samples.
A century after its discovery [1] Chagas disease still represents a health threat to an estimated 28 million people in the Americas, being the second highest illness burden among neglected tropical diseases [2]–[3]. The infection by the protozoan Trypanosoma cruzi can be acquired from blood-sucking triatomine bugs, blood transfusion, transplacental transmission or by the oral contamination foodstuffs by infected triatomine faeces [2]–[3]. Since 1990, a series of international initiatives based on vector control, systematic screening of blood donors in all endemic countries, and detection and treatment of congenital transmission have been launched for control and elimination of Chagas disease. These strategies have led to significant reduction in the number of infected people worldwide. According to information from 21 countries where the disease is endemic, the number of infected persons today is estimated to be 7,694,500, most of them at the chronic stage of disease [2]–[3]. Traditional parasitological procedures, such as xenodiagnosis and haemoculture are laborious and time-consuming and show poor sensitivities in cases of low-level parasitaemias, limiting their usefulness in diagnosis and monitoring of drug efficacy [4]–[6]. Since the past decade, the application of polymerase chain reaction (PCR) to detect T. cruzi directly in blood samples has opened new possibilities for the diagnosis of infection and evaluation of trypanocidal chemotherapy in different clinical and epidemiological settings [7]–[22]. These PCR procedures have revealed highly variable levels of sensitivity and specificity, depending on a number of technical factors such as, the volume of sample collected, the conditions of conservation of the sample, the methods used to isolate DNA, the parasite sequences and primers selected, the reagents used as well as the thermo-cycling conditions. Variability in PCR sensitivity could also be in part explained by the intermittent presence and quantity of circulating parasites at the time of blood collection. In addition, molecular targets from strains belonging to six different T. cruzi discrete typing units (DTUs, [23]) with dissimilar DNA content and gene dosage [22], [24]–[25] have been used for molecular diagnosis by different laboratories. In addition, sequence polymorphisms within amplified fragments among strains from different DTUs may influence the efficiency of amplification [26]–[27]. Moreover, false negative findings due to interference of PCR inhibitory substances co-purified during lysis and DNA extraction of blood samples and false positive results mostly due to carry over DNA contamination [22], [28] may arise. In this context, the assessment of the performances of currently available PCR tests for detection of T. cruzi infection in blood samples and DNA control sets was launched by expert laboratories in PCR detection of T. cruzi infection from different countries of America and Europe. We aimed to compare the performance of currently used PCR strategies for detection of T. cruzi DNA in sets of blind samples, including purified DNA from reference culture stocks from different T. cruzi discrete typing units, human blood samples spiked with cultured parasite cells and clinical samples from seropositive and seronegative patients from different endemic countries, in order to select the best performing tests for validation.
An access database form was distributed to the participants to standardize reporting of results. Those laboratories performing more than one PCR test per sample sent a separate report for each test. The results were analyzed by using SAS Software and Microsoft Excel. Due to the exploratory nature of the study, a descriptive analysis of results is provided. For set A, the following parameters were evaluated: 1) specificity (Sp): the proportion of negative PCR results in the three negative samples, 2) coherence: (Co) the ability of reporting positive PCR findings in a consecutive way, from the highest to the lowest detected DNA concentration for each series of DNA dilutions of parasite stocks and 3) the detection limits (DL) for each stock. A test was defined as Good Performing Method (GPM) if it was 100% specific and coherent and capable of detecting 10 fg/ul or less DNA for all parasite DTU stocks. For set B the same parameters were evaluated: Sp, Co and DL. A test was defined as GPM if it was 100% specific and coherent and capable of detecting 5 parasite equivalents/mL of Guanidine Hidrochloride-EDTA treated blood or less. For each sample of set C, a consensus PCR result was obtained on the basis of the reports by GPM tests in sets A and B, as done in other PCR interlaboratory studies [34]. A sample was considered PCR positive by consensus if more than 50% of the GPM gave positive results and PCR negative if more than 50% of GPM tests gave negative results. Those samples for which 50% of the GPM methods gave positive reports and 50% gave negative ones were considered indeterminate. The sensitivity, specificity, accuracy and kappa index of the different PCR tests were calculated by using 1) the above mentioned consensus PCR results and 2) the serological diagnosis as the reference methods. Inter-observer kappa coefficients were calculated using GraphPad Software on-line statistical calculators (http: //www. graphpad. com/quickcalcs/kappa1. cfm). Kappa values<0. 01 indicate no concordance, those between 0. 1 and 0. 4 indicate weak concordance, those between 0. 41 and 0. 60 indicate clear concordance, those between 0. 61 and 0. 80 indicate strong concordance, and those between 0. 81 and 1. 00 indicate nearly complete concordance. Accuracy was calculated as reported [35].
Table 2 shows the data obtained by the 48 PCR tests on DNA dilutions from the 3 parasite stocks representing DTUs I, IV and VI. The seven PCR tests targeting sequences other than Sat-DNA or kDNA failed to detect the most concentrated DNA sample (10 fg/ul) of one, two or all three parasite stocks (Tests C3, C4, C6, S1 and S3; Detection Limit = ND), or reported false positive findings in the negative controls without DNA (Tests C5 and U2) and thus were not included in the following analysis. Out of the 41 tests based on kDNA (28 tests) or Sat-DNA sequences (13 tests), 25 (51. 2%) provided specific and coherent results for all three parasite stocks (Sp = Y, Co = Y, Table 2). Fourteen of them targeted kDNA, representing 50% of the reported kDNA-PCR tests and 11 targeted Sat-DNA, representing 84. 6% of Sat-DNA PCR tests. These data indicated that PCR tests based on Sat-DNA sequences were more specific than those based on kDNA. Figures 1A and 1B show the distribution of the detection limits (DL) of the above mentioned 11 Sat-DNA PCR and 14 kDNA PCR tests, respectively, for each T. cruzi stock. Analysis of T. cruzi I DNA series: Nine out of 11 Sat-DNA- and all 14 kDNA-PCR tests were capable of detecting at least the most concentrated T. cruzi I DNA sample (Figure 1 A and B, grey bars) and 2 Sat-DNA and 8 kDNA-PCR tests could detect 0. 1 fg/µl of T. cruzi I DNA. The lowest detection limit for T. cruzi I DNA was 0. 01 fg/ul obtained by laboratory W using conventional kDNA-PCR (Table 2 and Figure 1B). Thus, kDNA- PCR tests were more sensitive than Sat-DNA PCR tests to detect T. cruzi I DNA. Analysis of T. cruzi IV DNA series: 8 out of 11 Sat-DNA- and all 14 kDNA-PCR tests were capable of detecting the most concentrated T. cruzi IV DNA sample (Figure 1 A and B, black bars). The lowest detection limit (1 fg/µl) was reached by three Sat-DNA- and six kDNA-PCR tests, suggesting similar analytical sensitivities of methods based on both molecular targets to detect T. cruzi IV DNA. Analysis of T. cruzi VI DNA series: All 11 Sat-DNA- and 13 out of 14 kDNA-PCR tests were capable of detecting the most concentrated T. cruzi VI DNA sample (Figure 1 A and B, white bars). The only test that did not detect Cl-Brener DNA amplified the constant kDNA region (G1, kDNAc, Table 2). The lowest detection limit (0. 01 fg/µl) was obtained by 2 Sat-DNA PCR tests (Z and F1, Table 2) followed by 0. 1 fg/µl obtained by 2 conventional kDNA-PCR tests (R and W, Table 2). Overall, the reported PCR tests were less sensitive for detecting DNA from the T. cruzi IV reference stock. Twenty PCR tests showing specific and coherent results and detecting at least the most concentrated DNA samples from each of the parasite stocks were considered Good Performing Methods for Set A (bold fonts, Table 2). They comprised 53. 8% of 13 Sat-DNA-PCR and 46. 42% of 28 kDNA-PCR tests. Ten GPM tests used in-house (IH) PCR mixtures and 10 used commercial master mixes (Kt), representing 35. 7% of the 28 IH and 76. 9% of the 13 Kt PCR reagent mixes. In addition, 12 GPM used conventional amplification and eight used real time PCR (C and RT in Table 2), representing 38. 7% of 31 C and 72. 7% of 11 RT tests. These data showed that commercial master mixes and real time PCR offered better PCR performance in purified DNA samples. Out of the 44 PCR tests reported for spiked Guanidine Hidrochloride-EDTA blood samples, the three tests targeting sequences other than Sat-DNA or kDNA were not further analyzed, because they failed to detect the most concentrated sample (S1 and S3 tests) or showed false positive findings in the non-spiked control (U2 test) (Set B, Table 2). Twenty five out of 41 PCR tests based on kDNA and Sat-DNA sequences showed specific, coherent results and detection limits of at least 5 par/ml (GPM, bold fonts, Table 2, Set B). They included 14 kDNA and 11 Sat-DNA PCR tests, representing 50% of 28 kDNA and 84. 6% of 13 Sat-DNA based tests. Ten GPM used in-house extraction methods and 15 used DNA extraction kits, representing 41. 6% of 24 IH and 62. 5% of 24 Kt tests. Thus, methods using commercial DNA extraction and Sat-DNA as amplification target resulted in better performance. Procedures based on kDNA presented more variation in sensitivity than Sat-DNA tests (Figure 1C, white bars). The smallest detected concentration was 5×10−3 par/ml, recorded by three laboratories using conventional kDNA-PCR after DNA extractions with Chelex resine, a blood extraction kit or solvent extraction with Phenol (LbE, LbL1 and LbW, respectively, Table 1). Tests based on Sat-DNA presented sensitivities between 0. 05 and 0. 5 par/ml (10/11 tests, Figure 1C, black bars) with the only exception of one test based on solvent DNA extraction and IH conventional Sat-DNA PCR (C2, Table 1) that reached a detection limit of 5×10−5 par/mL (Table 2, and Figure 1C). Out of the 44 PCR tests performed on clinical samples of Set C, a 18s-rDNA PCR (S3) and a SL-DNA PCR (C6) tests did not detect any positive sample and the 24s α rDNA-PCR test (U2) had only 40% of specificity. Consequently, they were not included for subsequent analysis. The levels of agreement among the 41 remaining PCR tests on the reports for each clinical sample are presented in Table 3. For each sample, 3 series of consensus PCR results were calculated: 1) consensus based on the 28 kDNA-PCR tests, 2) consensus based on the 13 Sat-DNA PCR tests and 3) consensus based on the 16 tests defined as GPM in both sets A and B samples. The sensitivity of consensus kDNA-PCR was 65. 62% (21 PCR positive/32 seropositive samples), that of consensus Sat-DNA was 62. 5% (20 PCR positive/32 seropositive samples) and that of consensus GPM was 56. 25% (B (18 PCR positive/32 seropositive samples) being 4 samples indeterminate (13,18,20,21, Table 3) because the levels of agreement among GPM tests was 50%. The individual performance of the 41 PCR tests was evaluated in comparison with the consensus PCR results reached by the 16 GPM in sets A and B (18 PCR positive, 20 PCR negative samples) and in comparison with serologic diagnosis (10 seronegative, 32 seropositive samples) (Table 4). There was a high variability among the performances of the different methods (Table 4). The median values of sensitivity, specificity and accuracy of the 41 tests were 72,77. 5 and 68. 4%, respectively in comparison to consensus GPM PCR reports, and 59. 4,70 and 59. 5%, respectively in comparison to serological diagnosis (Table 4). Four GPM showed the best operational parameters in set C (Table 4). Tests LbD2 and LbD3 used solvent DNA extraction followed by conventional hot-start and Real time PCR targeted to Sat-DNA, respectively (primer pairs TCZ-F/TCZ-R). Test LbF1 used a commercial kit for DNA extraction based on glass fiber columns and Real Time PCR targeted to Sat-DNA (primer pairs cruzi 1/cruzi 2 and TaqMan probe cruzi 3) and test LbQ used solvent DNA extraction and conventional hot-start PCR for kDNA (primer pairs 121/122). The performance of these four tests was further evaluated at the coordinating laboratory on a subset of samples from seropositive and seronegative patients, analysed in four independent experiments (Table 5). Examples of the outputs of each method are shown in Figure 2. The degree of concordance among the reported results by the BPM was between 87. 5% and 90. 62%. This intralaboratory evaluation showed that the selected methods depicted similar operational parameters than when performed by the corresponding laboratories in the international study (Tables 4 and 6).
In set A, GPM included kDNA and sat-DNA PCR tests in similar proportions. However, Sat-DNA PCR tests were less sensitive than kDNA-PCR tests to detect T. cruzi I DNA. This is most likely due to the fact that T. cruzi DTU I harbors approximately four to ten-fold less number of satellite repeats than DTUs II, V and VI, which has been demonstrated by different molecular approaches [22], [45]. Regarding T. cruzi IV that also harbors a lower dosage for satellite sequences [22], similar analytical sensitivities of kDNA and Sat-DNA PCR tests were observed, being lower than that obtained for the other two tested DTU representative stocks. The genome size and relative DNA contents of Can III cl1 (116. 44 Mb, 95% CI 110. 4–122. 63 and 1. 090, respectively) and CL Brener (108. 55 Mb, 95% CI 101. 41–115. 89 and 1. 017) are similar [25], although Can III cl1 harbors about 5 fold less satellite repeats than CL-Brener [22]. The relative contribution of the nucleus and kinetoplast has not been measured but normally, kDNA represents 20–25% of the total DNA content [46]. There are no available data regarding the number of minicircles in the kinetoplast of Can III cl1, so it could be speculated that the lower analytical sensitivity of most PCR tests to detect DNA from this clone respect to the other ones, could be due to a lower minicircle copy dosage. Set B allowed evaluation of the influence of DNA extraction procedures in the PCR performance. A 72. 2% of DNA extraction methods based on commercial kits led to GPM in set B, whereas 57. 8% of phenol-chlorophorm extracted DNA led to GPM reports. These findings indicated that Guanidine Hidrochloride-EDTA blood was suitable for DNA extraction using kits based on lysis buffers containing Guanidine salts. Out of the 25 GPM in set B, 14 had a sensitivity of 0. 05 par/ml, which should be adequate for diagnosis of infection in chronic patients [20]. Indeed, the necessary detection limit in chronic Chagas disease has been stated as one parasite cell in 10 mL of blood [20]. Analysis of PCR performance in set C clinical samples showed that the four best performing tests presented strong concordance with respect to consensus PCR results obtained by the 16 tests defined as GPM in sets A plus B (kappa index between 0. 7 and 0. 8). Out of them, three tests targeted sat-DNA sequences and only one targeted kDNA. These data are in agreement with previous works showing that PCRs targeting Sat-DNA performed better than PCRs targeting kDNA sequences [47]–[49], although kDNA based PCR has been more widely used [20]. Moreover, two of the sat-DNA best performing tests used Real Time PCR, one with a Sybr Green fluorescent dye (LbD2) and the other one with a TaqMan probe (LbF1). It must be pointed out that LbD2 and LbD3 tests were performed by the same laboratory. Out of the 16 GPM performed by 11 different laboratories, 3 laboratories performed two methods (LbD, LbK and LbP) and one lab developed 3 tests (LbG). These data point to laboratory dependence concerning PCR performance, which may be due to multiple factors including technical expertise, correct use of quality controls, instrumentation and reagents. For example, tests LbF1, LbS2 (GPM) and LbZ (not GPM) were all based on sat-DNA Real Time PCR using the same primer pair (cruzi 1 – cruzi 2), differing in the trade marks of the DNA extraction and Master Mix kits. Some tests shown as GPM in sets A+B had very low sensitivities in set C (LbK2, LbP2, LbV1, Table 4), suggesting that quality controls might have failed to distinguish false negative clinical samples. A major drawback of most PCR tests is that they do not contain an internal amplification control (IAC). An IAC is a non target DNA sequence present in the same sample reaction tube, which is co-amplified simultaneously with the target sequence [50]. In a PCR without an IAC, a negative result can indicate that the reaction was inhibited, as a result of the presence of inhibitory substances in the sample matrix. The presence of PCR inhibition in Guanidine Hidrochloride-EDTA treated blood samples has been described [22]. The European Standardization Committee (CEN), in collaboration with International Standard Organization (ISO) has proposed a general guideline for PCR testing that requires the presence of IAC in the reaction mixture [51]. Therefore, only IAC-containing PCRs should undergo multicentre collaborative trials, which is a prerequisite for validation. Some other tests shown as GPM in sets A+B had very low specificities (LbI2, LbW, LbG2, LbG3, Table 4). Amplicon carry-over contamination is one of the most probable causes. PCR master mixes with dUTP and Uracil-DNA N-glycosylase (UNG) intended to abolish amplicon carry-over contamination were used in some tests (LbF, LbG, LbL, LbS, Table 1). Nevertheless, some of them did not show good specificity in set C (LbG2, LbG3, LbG4, LbL1, LbL2, Table 4), suggesting that problems during sample processing, such as sample to sample contamination could have arisen. The median values of the sensitivities obtained in testing the Set C samples with the 16 tests determined to be GPMs by testing the Set A and Set B samples varied considerably depending on the clinical characteristics of the persons from whom the Set C samples were drawn. Indeed, sensitivity was 100% (25-75p = 100−100) for immunosuppressed heart transplanted pts, 56. 5% (25-75p = 39. 1–66. 3%) for asymptomatic and 57. 1% (25-75p = 14–75%) for symptomatic chronic Chagas disease patients. These data point to the limitations of PCR strategies for diagnosis of patients at the chronic phase of disease. In addition, some of these samples had been stored at 4°C for at least two years before this PCR study; thus higher PCR positivity might be obtained in prospective clinical studies but it is unlikely that the current PCR methods will have a sensitivity comparable to serological assays for diagnosis of chronic Chagas disease. The four BPM methods were transferred to the Coordinating laboratory, where they were evaluated in a subset of clinical samples, each one tested in four independent assays, obtaining good concordance and confirming the performance reported by the participating laboratories in the previous international study (Tables 5 and 6). Further work is still needed to validate them through prospective studies in different settings. In this regard, this collaborative evaluation constitutes a starting point towards technical improvement and development of an international standard operating procedure (SOP) for T. cruzi PCR. In this context, the BPMs could be recommended for alternative diagnostic support, such as in the following settings: a) post-treatment follow-up of patients to look for failure of therapy to achieve parasitologic response [12], [14]–[16], [20], [22]; b) diagnosis of congenital Chagas disease in newborns in whom the presence of maternal anti-T. cruzi antibodies make serological studies useless [11], [15], [48]; c) early diagnosis of reactivation after organ transplantation of T. cruzi infected recipients under immunosuppressive therapy [18], [41], d) differential diagnosis of Chagas reactivation in patients with AIDS [39], and e) suspicion of oral transmission [52]. Moreover it can be useful for post-treatment follow-up of experimental animals to look for failure of therapy to achieve parasitologic cure [53]; in diagnosis in naturally infected triatomines or triatomines used for xenodiagnosis, since it has been shown that PCR tests are much more sensitive than microscopic examination of intestinal contents [37], [54]; and diagnosis of T. cruzi infection in mammalian reservoirs for which serologic tools have not been developed [38].
A century after its discovery, Chagas disease, caused by the parasite Trypanosoma cruzi, still represents a major neglected tropical threat. Accurate diagnostics tools as well as surrogate markers of parasitological response to treatment are research priorities in the field. The polymerase chain reaction (PCR) has been proposed as a sensitive laboratory tool for detection of T. cruzi infection and monitoring of parasitological treatment outcome. However, high variation in accuracy and lack of international quality controls has precluded reliable applications in the clinical practice and comparisons of data among cohorts and geographical regions. In an effort towards harmonization of PCR strategies, 26 expert laboratories from 16 countries evaluated their current PCR procedures against sets of control samples, composed by serial dilutions of T. cruzi DNA from culture stocks belonging to different lineages, human blood spiked with parasite cells and blood samples from Chagas disease patients. A high variability in sensitivities and specificities was found among the 48 reported PCR tests. Out of them, four tests with best performance were selected and further evaluated. This study represents a crucial first step towards device of a standardized operative procedure for T. cruzi PCR.
infectious diseases/protozoal infections
infectious diseases/neglected tropical diseases
molecular biology
microbiology/parasitology
International Study to Evaluate PCR Methods for Detection of Trypanosoma cruzi DNA in Blood Samples from Chagas Disease Patients