External quality assessment of HIV-1 DNA quantification assays used in the clinical setting in Italy

Total cell-associated HIV-1 DNA is a surrogate marker of the HIV-1 reservoir, however, certified systems for its quantification are not available. The Italian HIV DNA Network was launched to validate HIV-1 DNA quantification methods in use at University and Hospital labs. A quality control panel including HIV-1 DNA standards, reconstructed blood samples (RBSs) and DNA from different HIV-1 subtypes was blindly tested by 12 participating labs by quantitative real-time PCR (n = 6), droplet digital PCR (n = 3) or both (n = 3). The median 95% hit rate was 4.6 (3.7–5.5) copies per test and linearity in the tested range was excellent (R2 = 1.000 [1.000–1.000]). The median values obtained across labs were 3,370 (2,287–4,245), 445 (299–498), 59 (40–81) and 7 (6–11) HIV-1 DNA copies, for the 3,584, 448, 56 and 7-copy standards, respectively. With RBSs, measured values were within twofold with respect to the median in two thirds of cases. HIV-1 subtypes were missed (CRF01_AE by 3 labs) or underestimated by > 1 log (subtypes A, C, D, F by one lab; CRF01_AE by one lab; CRF02_AG by one lab). The overall performance was excellent with HIV-1 DNA standards, however detection of different HIV-1 subtypes must be improved.

www.nature.com/scientificreports/ used for reversing HIV-1 latency and the markers measured to quantify the induced virus [7][8][9] . In general, such methods are complex, time-consuming and difficult to standardize. In addition, they tend to underestimate the viral reservoir because not all the replication competent virus population can be effectively induced in vitro 10 . While viral outgrowth assays remain very valuable in investigating the nature, dynamics and pathogenesis of the HIV-1 reservoir, lower complexity methods are needed to integrate a measure of the latent HIV-1 reservoir into routine patient management. Molecular assays such as the quantification of total cell-associated HIV-1 DNA can fulfill these requirements since they obviate the need for cell cultivation, biosafety level containment and high-level, specific expertise 11 . However, total HIV-1 DNA clearly overestimates the viral reservoir since it includes not only replication competent proviruses but also defective and more labile, unintegrated forms 10,12 . Nonetheless, extrachromosomal forms can contribute to HIV-1 pathogenesis 13 and total HIV-1 DNA load seems to correlate well with the frequency of cells containing replication-competent virus 14,15 . In addition, although discriminating between integrated and unintegrated HIV-1 DNA can add useful information in select studies, assays specifically targeting integrated HIV-1 DNA are complex to set up and require extensive replicate testing which makes the system not amenable to routine use 14,16 . Most importantly, the clinical role of total HIV-1 DNA is supported by sparse but relevant studies. A meta-analysis of six studies in untreated patients 17 indicated that total HIV-1 DNA is a stronger predictor of AIDS and of all-cause mortality compared to plasma HIV-1 RNA load. In addition, baseline total HIV-1 DNA load is predictive of the occurrence and severity of HIV-1 associated neurologic disorders 18 as well as of levels of T-cell activation 19 . More recently, the role of total HIV-1 DNA has been evaluated in treated patients with suppressed plasma HIV-1 RNA providing two important lines of evidence. First, total HIV-1 DNA is predictive of the time to plasma HIV-1 RNA rebound after treatment interruption, both in patients treated early during primary infection 20,21 and in patients treated late during chronic infection 22,23 . Second, higher total HIV-1 DNA levels are associated with an increased risk of virological failure following treatment de-escalation, as shown in the MONOI and MONET trials 24,25 , comparing the outcomes of a switch to darunavir/ritonavir (DRV/r) monotherapy vs. combination therapy, and in the DOMONO trial evaluating the switch to DTG monotherapy 26 .
The increasing interest for HIV-1 DNA quantification as a rough estimate of the viral reservoir has not yet been accompanied by the development of certified assays to measure this parameter. Few commercial assays have been developed but they have not been certified for in vitro diagnostic use. However, several homebrew HIV-1 DNA quantification protocols have been developed worldwide. Such assays typically undergo different kinds of internal validation but the results obtained from different methods can be hardly compared due to different genomic regions analyzed, different standards used and/or different principles and readouts, such as real time PCR (qPCR) or droplet digital PCR (ddPCR). The Italian HIV DNA Network was launched to investigate the features and performance of the HIV-1 DNA quantification methods in use at Italian University and Hospital labs, including homebrew systems and commercial kits not yet marked for in vitro diagnostic use. Here we report the results obtained from the first external quality assurance program involving 12 Italian labs receiving a comprehensive panel of HIV-1 DNA standards, reconstructed clinical samples and DNA extracts from different HIV-1 subtypes.

Methods
Participating centers. The quality control panel was assembled by the coordinating center and sent to the 12 labs participating to the Italian HIV DNA Network, including the coordinator, for blind testing, using the methodology routinely used at each center. The labs were originally asked to participate to the Network based either on the use of commercial assays or on publication of at least one peer-reviewed paper reporting quantification of HIV-1 DNA.
Quality control material for the external validation. Detailed instructions concerning the manipulation and processing of the quality assurance sample panel were provided to each lab. The number of replicates, dilutions and results required to validate the assay are indicated in Table 1.
To assess the accuracy, sensitivity, precision and linear range on reference standards, the pNL4-3 plasmid (code ARP2006), obtained from the Centre for AIDS Reagents (CFAR) was quantified with respect to an International Standard (code ARP956; CFAR) by qPCR 27 and ddPCR, using the same primers and probe as for qPCR in a reaction adapted for the QX200™ Droplet Digital™ PCR System (Bio-Rad), and by fluorometric quantification using the Qubit 4 fluorometer (ThermoFisher), based on the mean of the three measurements which differed from one another within 1.5-fold. The plasmid standard was then diluted in 10 ng/µl of human genomic DNA (code G3041; Promega), and frozen until shipment to each lab.
To assess inter-laboratory variability and intra-laboratory precision on clinical whole blood samples, reconstructed blood samples (RBS) were generated at the coordinator lab by diluting U1/HIV-1 lymphoblastoid cells (code ARP139; CFAR), carrying 2 copies of HIV-1 genome per cell, into HIV-1 negative human peripheral blood (code 297CTIPB.1.5; CTI Biotech), provided by the supplier in compliance with all relevant ethical guidelines. Briefly, U1 cells were counted and one million cells were added to 1 ml of HIV-1-negative blood and then serially diluted 1:10, 1:10, 1:3, 1:3. At the coordinating lab, aliquots of the above dilutions series were processed to extract DNA and quantify HIV-1 DNA to confirm that the HIV-1 DNA amount was in a range suitable for the purpose of the study. The same HIV-1 negative human peripheral blood used as diluent was included in the quality assurance panel to assess the specificity of the assays. Since these samples could not be considered as certified reference standards, the median of the HIV-1 DNA levels obtained at all the study labs was taken as a reference for the assessment of interlaboratory reproducibility, as proposed in a previous HIV-1 DNA quantification quality assurance study 28  www.nature.com/scientificreports/ To assess the ability to recognize the most representative subtypes, seven HIV-1 strains obtained by CFAR (codes ARP1089, ARP1112, 100595, ARP 169.6, ARP1121, 100215 and ARP1124, representing the A1, B, C, D, F, CRF01_AE and CRF02_AG variants, respectively) were used to infect the lymphoblastoid MOLT-4/CCR5 cell line expressing high levels of CCR5 receptor (code ARP5039; CFAR) and DNA was extracted at the coordinator lab. HIV-1 DNA was originally quantified in HIV-1 subtype extracts at the coordinating lab to ensure appropriateness of the material, however, similar to RBSs analysis, the median values derived from the study were used to estimate target underestimation or overestimation with the different subtypes.
Statistical analysis. Data were reported as mean ± SD or median (IQR) copies per test or copies per 10 6 cells, as appropriate for the distribution of data based on the Shapiro-Wilk test for normality. Comparisons between independent groups of data were done by the Student t-test or by Mann-Whitney U test. Analysis of paired data was done by the paired t test or by Wilcoxon signed rank sum test. Comparisons between frequencies were done by chi-square test. The trend between ordered independent variables and continuous dependent variables was analyzed by the Jonckheere-Terpstra test. Statistical analysis was performed with SPSS version 20.0.

Results
Assays used for total HIV-1 DNA quantification. The main features of the assays used at the 12 participating centers are shown in Table 2. Data sets provided are indicated by the lab number followed by _qPCR or _ddPCR, consistent with the method used. Four labs performed homebrew qPCR, 3 homebrew ddPCR, and 3 both methodologies. The remaining 2 labs tested one of two different commercial qPCR assays each, namely the Generic HIV DNA Cell kit (Biocentric) and the HIV-1 DNA Test PRO (Diatheva) (lab07_qPCR and lab11_qPCR, respectively).
Sensitivity and linear range on titrated standards. The sensitivity, defined as the smallest amount of HIV-1 DNA detectable in the 95% of cases (95% hit rate) was calculated on 8 replicates of a twofold dilution series of the reference standard containing 1.75 to 112 nominal HIV-1 DNA copies in 50 ng of human DNA. Results were delivered as frequency of qualitative positive reactions for each standard dilution. Based on probit analysis, the median analytical sensitivity of the assays was 4.6 (3.7-5.5) HIV-1 DNA copies per test at the 95% hit rate, including one outlier result (10.5 HIV-1 DNA copies per test) and without statistically significant difference between qPCR and ddPCR (4.7 [3.7-5.5] vs. 4.5 [4.0-5.5] HIV-1 DNA copies per test respectively; P = 0.776) (Fig. 1A). When analyzing 200 to 1,000 ng of total DNA as in a typical clinical application, this translates into a limit of detection of around 150 to 30 HIV-1 DNA copies per 10 6 cells, assuming as a reference that one million cells contain 6.66 µg of DNA and once confirmed that the same performance is maintained with clinical samples.
The linear dynamic range, as determined by testing a 5-point log 10 dilution series of the standard (70,000 to 7 nominal copies per test), was expressed as the R 2 of the linear regression. The median R 2 value obtained was 1.000 (1.00-1.00), again without difference between qPCR and ddPCR (Fig. 1B). There was one outlier value (R 2 = 0.92).
Accuracy and precision on titrated standards. Accuracy was defined as the distance between measured and expected HIV-1 DNA copies and it was tested on 4 standard dilutions, each analyzed in quadruplicate. The median values obtained across labs were 3,370 (2,287-4,245), 445 (299-498), 59 (40-81) and 7 (6-11) HIV-1 DNA copies per test, for the 3,584, 448, 56 and 7-copy standards, respectively (Fig. 2). The ratio between nominal and measured values (fold difference) and the outlier values recorded by three labs are indicated in Supplementary  Notably, both intra-run and inter-run CV values tended to increase with decreasing copy numbers (P = 0.042).  (Fig. 3). These values matched very closely those originally obtained at the coordinating lab (Table 1). When stratified by method, there was   When considering median values for each RBS as a reference, outlier values were detected for lab08_qPCR with RBS-3 and RBS-4, for lab06_qPCR with RBS-1 and RBS-4, for lab08_ddPCR with RBS-2 and for lab02_ qPCR, lab04_ddPCR and lab05_qPCR with RBS-4. Thus, the lowest copy-number RBS (RBS-4) originated a larger number of outliers compared to the other RBSs (P = 0.074). Lab02_qPCR failed to detect HIV-1 DNA in RBS-4, this sample was excluded from the inter-laboratory variability analysis. The fold-difference between the measured and median values for each sample and lab are indicated in Supplementary Table 3. Overall, an absolute fold-difference > 2 was scored for one third of measurements, namely 4/15, 5/15, 5/14 and 4/13 for the four RBSs ordered by decreasing HIV-1 DNA titer, respectively, with no cases differing by more than 1 log with respect to the reference median value. When comparing fold-differences obtained by qPCR and ddPCR for all the individual RBSs, qPCR values were significantly closer to the median compared with ddPCR values (median 1.0 [0.6-1.6] vs. 1.4 [1.1-2.5], P = 0.039). This difference was prevalently driven by RBS-2 and RBS-3 (ratio between average ddPCR and qPCR values above twofold). Notably, the negative blood sample was scored as positive in 3 replicates (811.2 ± 615.3 HIV-1 copies per 10 6 cells) by lab04_ddPCR, in 9 replicates (110.0 ± 73.1 HIV-1 copies per 10 6 cells) by lab06_ddPCR and in 4 replicates (2.5 ± 1.0 HIV-1 copies per 10 6 cells) by lab07_qPCR. Overall, the rate of false positive reactions was significantly higher for ddPCR vs. qPCR (12/108 [11.1%] vs. 4/162 [2.5%], P = 0.003).
Precision on the same RBS panel was determined by measuring the CV both within and across extraction runs (Supplementary Table 4). The intra-extraction precision was calculated as the median CV of three extraction runs while the inter-extraction precision was calculated as the median CV of all the 18 replicates. In the intra-extraction assessment, the median CV was 20.  1-82.4). Similar to what found with plasmid standards, there was a significant increase in CV values with decreasing HIV-1 DNA copy numbers, both in the intra-and inter-extraction assessment (P = 0.042). The median CVs observed in the inter-extraction assessment were significantly higher than those observed in the intra-extraction assessment (P < 0.01). No significant differences in CVs obtained by qPCR or ddPCR were observed for either the whole RBS panel or any individual RBS.
Ability to recognize different subtypes. To assess the ability to recognize and quantify the main HIV-1 subtypes, the coordinator lab sent to each lab seven tubes containing DNA extracted from a lymphoblastoid cell line infected with reference subtypes A1, CRF01_AE, CRF02_AG, B, C, D, F (  (Fig. 4). Measured HIV-1 DNA was > 1 log lower than the median in 6 cases: CRF02_AG by lab04_ddPCR; subtypes A1, C, D, F by lab05_qPCR; CRF01_AE by lab06_ddPCR. In addition, CRF01_AE was not detected at all by www.nature.com/scientificreports/ lab03_ddPCR, lab05_qPCR and lab09_ddPCR (Supplementary Table 5). By contrast, there was only one case where measured HIV-1 DNA was > 1 log higher than the median (lab04_ddPCR with subtype A1). Alignment of available primer and probe sequences on the consensus of the different subtypes included in the panel revealed that, of the four labs underestimating or missing CRF01_AE, three used a forward primer affected by a 1-base deletion when hybridizing to the CRF01_AE and subtype F target sequence ( Supplementary  Fig. 1). Underestimation of CRF02_AG by lab04_qPCR coincided with the largest number of mismatches (6) affecting the forward primer among all the target subtypes considered, including one mismatch at position -3 bases with respect to the primer 3' end. Finally, lab05_qPCR primers and probes perfectly matched the consensus B sequence but were affected by 4-9 mismatches when aligned to the other subtypes.

Discussion
Despite conceptual limitations, total HIV-1 DNA is broadly considered as a measure of the viral reservoir and an attractive marker to monitor its changes following specific treatment strategies. Due to the lack of certified systems for in vitro diagnostic use, several homebrew assays have been developed, based on qPCR 27,29,40 or ddPCR 31 . However, the heterogeneity of methods may affect interpretation of data generated across different labs due to different sensitivity, accuracy and precision. Only two previously published papers 28,40 reported a multicenter quality control to evaluate the inter-laboratory reproducibility of total HIV-1 DNA quantification. In the French study 40 , 4 peripheral blood mononuclear cell (PBMC) samples were tested by 10 labs all using the commercially available Biocentric system. In the Italian study 28 , 7 labs tested 24 cellular samples carrying HIV-1 clade B and 40 HIV-1-negative PBMC samples spiked with different concentrations of plasmids containing the gag gene derived from different HIV-1 subtypes. All the labs participating to the latter study used hTERT as the housekeeping gene and one of two different primer sets to quantify HIV-1 DNA. The current study considerably expanded the scope of the quality control panel to assess the assays for their sensitivity, accuracy, linear range, precision and ability to recognize the different HIV-1 variants (Table 1). In addition, the variety of procedures used allowed an assessment of homebrew methods vs. commercial kits and of qPCR vs. ddPCR.
In total, 15 data sets were delivered by the 12 labs participating to the Italian HIV DNA Network. The overall performance with reference standards was good. Indeed, the linear range and sensitivity were excellent, with R 2 = 1.000 in 12/15 data sets and the 95% hit rate of 4.6 copies of target which translates into a sensitivity threshold of 150 to 30 copies of HIV-1 DNA per million cells when using 200 to 1,000 ng of total DNA in the reaction, provided PCR is run under optimal conditions without any inhibition. Also, the accuracy and precision measured on the titrated standards were high, with the difference from the expected value within twofold in 46/60 cases (15 centers testing 4 samples) and the CV < 50% in 44/60 and 47/60 cases in the intra-and inter-run assessment, respectively. However, an overall lower performance was obtained with RBSs. While the nominal number of copies was not certified for the RBSs, the difference from the median value, derived from the values obtained by all labs, was within twofold only in 36/58 cases. The CV was < 50% in 42/58 and 35/58 cases in the intra-and inter-extraction assessment, respectively. Although there may remain differences from clinical blood samples, the RBSs were prepared from a single source of HIV-1-negative human blood spiked with HIV-1-positive cells from infected cell lines, thus mirroring blood from infected patients. RBSs were then frozen to reproduce the most common material used in the clinic. The lower precision with RBSs compared to ready-to-use plasmid standards likely derived from additional sources of variability including HIV-1 DNA extraction and measurement. Noteworthy, precision appeared to decrease with lower numbers of target copies, as often occurring with individuals under prolonged successful therapy 37 . In addition, repeated RBS analysis revealed lower precision in inter-extraction compared with intra-extraction runs. These drawbacks may advise for same-run replicate analysis of samples obtained at different time points from the same patient in the clinical setting.
Comparative analysis of the 9 qPCR vs. the 6 ddPCR data sets revealed few significant differences. With titrated standards, ddPCR results matched more closely than qPCR the expected target copy numbers; in addition, qPCR had larger inter-run CV values with respect to ddPCR, particularly at low-copy numbers. With RBSs, ddPCR values tended to distribute above the median of the whole dataset and also generated a significantly higher rate of false positive reactions compared with qPCR when analyzing the negative control. Higher precision with respect to qPCR is indeed one of the expected benefits of ddPCR, particularly at low copy numbers 41 . The distribution of ddPCR generated values with RBSs may reflect the qPCR underestimation bias detected with plasmid standards. Indeed, median values obtained for the whole dataset were influenced more by qPCR values (9 datasets) than by ddPCR (6 datasets). On the other hand, the trend for ddPCR to generate some false positive signals might have derived from an incorrect setting of the threshold, a well-known key issue with ddPCR complicating low-level detection abilities and requiring training by expert users or system manufacturers 42,43 . Overall, these specific caveats highlighted by our study suggest using replicate testing with qPCR and advise for inclusion of multiple negative controls and adjustments in the computation of background noise with ddPCR.
Of the 9 qPCR data sets, only 2 were generated by commercial, research-use-only kits. There were no relevant differences between homebrew and commercial qPCR results. However, the Biocentric system yielded the only few cases of false positive qPCR results with RBSs and also overestimated HIV-1 DNA copies in the titrated standards (2.3, 2.4, 2.8, 3.0-fold for the 3,584, 448, 56 and 7-copy standards, respectively) but not in the RBSs. Nevertheless, the availability of standardized and ready to use reagents from a commercial source remains valuable, particularly for labs with limited experience, and ensures large-scale validation as well as updates of primers and probes which should minimize false negative reactions.
Significant issues were scored with the HIV-1 subtype panel. Substantial underestimation of HIV-1 DNA occurred at several labs, with CRF01_AE most affected. The primary reason for underestimation or failure to detect a specific subtype is suboptimal primers and probes, possibly chosen from outdated literature or published studies focusing on specific subtypes. The ability of published primers and probes to recognize different HIV-1 www.nature.com/scientificreports/ subtypes should be verified on a regular basis by checking for conservation of the target regions in the updated reference databases such as GenBank and the curated Los Alamos National Laboratory repository. For example, underestimation or even lack of detection of CRF01_AE by four participants to this study may have been driven by the occurrence of one extra nucleotide in the region targeted by the forward primer. By contrast, lab08_ddPCR using the same primer/probe combination correctly quantified CRF01_AE, implying that other factors such as mismatch tolerant experimental conditions may have a role. However, it must be noted that CRF01_AE was also the lowest concentration DNA extract provided for subtype analysis which may have impacted accuracy per se. Similarly, lab04_qPCR likely underestimated CRF02_AG due to six mismatches in the forward primer. Finally, underestimation of multiple subtypes by lab05_qPCR apparently derived from choosing primers and probe targeting HIV-1 subtype B without adequate consideration for target conservation across different subtypes.
This comprehensive external quality assurance study documented good performance parameters for noncertified quantitative HIV-1 DNA assays, however significant caveats were documented, including subtype specific issues and false positive reactions. Since the study involved only labs with HIV-1 DNA quantification documented in peer-reviewed papers, the same performance may be not guaranteed in other labs running the assay outside of these inclusion criteria. However, the above shortcomings need to be addressed at any individual lab willing to provide HIV-1 DNA measurements either in research studies or in the clinical setting, irrespective from the experience in the field.