The aim of this study was to compare the virus detection sensitivity of sRNA-based virus diagnosis with the established virus species-specific polymerase chain reaction (PCR) approach. Viruses propagated in tobacco plants included three engineered, infectious clones of Potato virus A (PVA), each carrying a different marker gene, and an infectious clone of Potato virus Y (PVY). Total RNA (containing sRNA) was isolated and subjected to reverse-transcription real-time PCR (RT-RT-PCR) and sRNA deep-sequencing at different concentrations. RNA extracted from various crop plants was included in the reactions to normalize RNA concentrations. Targeted detection of selected viruses showed a similar threshold for the sRNA and reverse-transcription quantitative PCR (RT-qPCR) analyses. The detection limit for PVY and PVA by RT-qPCR in this study was 3 and 1.5 fg of viral RNA, respectively, in 50 ng of total RNA per PCR reaction. When knowledge was available about the viruses likely present in the samples, sRNA-based virus detection was 10 times more sensitive than RT-RT-PCR. The advantage of sRNA analysis is the detection of all tested viruses without the need for virus-specific primers or probes.Plants recognize unrelated viruses by the antiviral defense system called RNA interference (RNAi). RNAi processes double-stranded viral RNA into small RNAs (sRNAs) of 21–24 nucleotides, the reassembly of which into longer strands  This method, published by The large number and high genetic variability of plant viruses make their detection cumbersome. Virus-specific antibodies or probes are available mainly for prevalent, economically harmful plant viruses, whereas the absence of specific diagnostic tools for the great majority of plant viruses hampers broader surveys of viruses . Plants The real-time polymerase chain reaction (RT-PCR) method is a well-established, sensitive and specific tool for virus detection and was used for many years before the sRNA-based approach was introduced . RT-PCR-Potato virus A (PVA) engineered to express red fluorescent protein (RFP) (PVA-ynHC/6K2rfp) (Potato virus Y [PVY-N605(1048)] (Nicotiana tabacum L. cv. Samsun nn) by particle bombardment using a HandyGun  and were given 0.3 g fertilizer  per liter of water when watered.Infectious clones of /6K2rfp) , green f/6K2rfp)  or β-glu/6K2rfp)  and an i5(1048)]  were useHandyGun . Plants Moloney murine leukemia virus (M-MLV) reverse transcriptase (Promega) using random hexamers, according to the manufacturer’s instructions.The uppermost fully expanded leaves were sampled from the tobacco plants. Total RNA was extracted from the leaves using Trizol reagent generated in-house . In addiRubus idaeus L.) and Ribes species obtained from the genetic resource collection maintained by the Natural Resources Institute of Finland (LUKE). A protocol combining the cetyl trimethylammonium bromide (CTAB) method and acidic phenol–chloroform extraction was used for RNA purification  Duchesne ex Rozier (pro sp.)], raspberries  using cDNA generated from total RNA extracted from systemically infected tobacco leaves. Primer3 software1 was used to design specific forward (F) and reverse (R) primers for RT-qPCR amplification of a 127-nt product from the coat protein (CP) encoding region of PVY  and a 138-nt product from the CP-encoding region of PVA .Quantitative PCR was carried out using LightCycler 480 SYBR Green I Master Mix  and a LightCycler 480 Instrument (Roche) in a 384-well microtiter plate format. Each well in the plate contained 1× SYBR Green I Master Mix with 0.45 pmol of forward and reverse primers and 5 μl (50 ng) of cDNA, in a total volume of 15 μl. The polymerase chain reaction (PCR) program consisted of a pre-incubation step at 95°C for 5 min, followed by 45 cycles of 10 s at 95°C, 10 s at 60°C, and 10 s at 72°C. Production of a single product only by each primer pair was evidenced by melting curve analysis at the end of the program.For qPCR, the PVY cDNA synthesized from RNA extracted from PVY particles was diluted to 50, 5.0, 0.5, 0.05, or 0.005 ng of cDNA in the reaction. Similarly, 12.5, 1.25, 0.125, 0.0125, or 0.00125 ng of PVA cDNA was included in the qPCR reactions. Cq-values were used to draw a standard curve that was used together with the Cq-values obtained from the PVY- or PVA-infected tobacco leaves to estimate the concentration of viral RNA in tobacco leaves.Ribes spp. These pools contained 3 μg total RNA in a volume of 15 μl. We thus analyzed four different concentrations of PVY and 12 different concentrations of PVA (1.0–0.000001 ng in 3 μg) (Table 1).For sRNA deep sequencing, different amounts of viral RNA  were added to the four RNA pools, each of which contained a total RNA mixture from the leaves of strawberry, raspberry, and 2.RNA samples sequenced in the laboratory of Fasteris SA  were analyzed for RNA concentration with the Qubit RNA assay (Invitrogen) and for integrity with the Agilent Bioanalyzer 2100 Total RNA Nano Assay . sRNAs shorter than 30 nt were isolated from the gel following acrylamide gel electrophoresis. Libraries for sequencing were prepared using the TruSeq small RNA kit . Sequencing was carried out with an Illumina HiSeq 2500 instrument using the 50-bp single-end mode. RNA sequences  were sorted to files based on their size after base-calling and trimming of the adapter sequences and were subjected to further analysis. Data were deposited to the European nucleotide archive (ENA), Accession No. ERP108051Table 1). For PVY, aliquots of the original pools sent for sequencing were tested by RT-RT-PCR (Table 1). All RNA samples hence prepared were transferred to test tubes containing 3 μg of a mixture of total RNA extracted from leaves of strawberry, raspberry and Ribes spp. in a total volume of 15 μl. This was done to mimic the total RNA concentrations used in sRNA sequencing. All the thus generated 16 samples were tested by RT-RT-PCR in duplicate using 50 ng of RNA. The experiment was carried out three times starting from the dilution of RNA.Detection sensitivity achieved by sRNA deep sequencing as compared with RT-RT-PCR was compared using an RT-RT-PCR -test commercially available from Fera Science, Ltd.   in the p3.Velvet software was used to produce contiguous sequences (contigs) from the reads of 21–24 nt . The k-mHomologous sequences were sought in the National Center for Biotechnology Information (NCBI) database using BLASTn (nucleotide blast). The contigs corresponding to the reference sequences were identified by alignment with the GFP-encoding , RFP-encoding  or GUS-encoding  sequence and the sequence of PVY  using BLASTn.The mapping analysis of sequence data was carried out with Novoalign software . All obtained 21- to 24-nt reads were analyzed against the reference sequences.Table 1) and subjected to deep sequencing.The amounts of RNA from PVY and the three engineered clones of PVA, PVA-GFP, PVA-RFP, and PVA-GUS, in the uppermost fully expanded leaves of the infected tobacco plants were 15.08, 0.86, 2.20, and 11.12 ng per 1 μg of total RNA, respectively, as determined by RT-qPCR. Based on this knowledge, different amounts of RNA were added to the RNA pools (Table 2) differed from the expected concentration of 200 ng/μl, which was originally based on the Nanodrop measurements. Sequencing by Illumina was successful, as indicated by a mean quality score > 38 in each pool. Over 97% of the reads had quality scores > 30 as determined by Illumina. The total number of reads per pool was 14–21 million (Table 2). Most reads were 21–24 nt (Table 3), but reads of up to 44 nt were also detected as a result of imperfect size selection by acrylamide gel electrophoresis and subsequent purification of the sRNAs.Analysis of the RNA pools with the Bioanalyzer assay at Fasteris SA indicated that the quality of the RNA was good enough for sRNA sequencing [RNA integrity number (RIN) values 4.6–5.8]. RNA concentrations in the pools measured with Qubit at Fasteris SA (Table 3). Using a COV of 100 nt led to the loss of shorter contigs specific to the reference sequences, whereas a COV of 30 nt produced short contigs that were not homologous to the reference sequences (data not shown).Contigs were built with Velvet using 21- to 24-nt reads. The k-mer values 13, 15, and 17 and COVs at 30, 50, and 100 nt were tested. The highest numbers of contigs homologous to PVY or to the reference sequences inserted into the genome of PVA were obtained using the k-mer value 15 and a COV of 50 nt . However, no PVY-specific contigs were obtained from the pools containing 0.001 or 0.000001 ng PVY RNA.PVY was detected by Velvet-based Potato virus A genomes, each carrying a different reference sequence encoding GUS, GFP or RFP, were distinguished by the reference sequence-specific contigs obtained by Velvet-based de novo analysis of sRNA reads. GUS-specific contigs were obtained from the pools containing 1, 0.02, or 0.002 ng of PVA-GUS RNA. GFP was detected in the pool containing 0.1 ng PVA-GFP RNA, but not in the three other pools containing 500–50,000 times less PVA-GFP RNA. RFP-specific contigs were obtained from pools containing 0.2 and 0.01 ng of PVA-RFP RNA (Table 3).Post-assembly analysis of the data with AssemblyAssembler was carried out using a wider range of k-mers (13–25 nt). Results were essentially similar to those obtained with Velvet alone, but contigs were longer in some cases when AssemblyAssembler was used.Figure 1).All sequenced 21- to 24-nt reads were aligned with the reference sequences using Novoalign to determine the relationship between the amount of virus and the number of reads aligned to the reference sequences. Because the reference sequences varied in length and the total number of reads varied between the sample pools, the results were normalized among the four pools by calculating the number of reads aligned per 1000 nt of the reference sequence and per 1 million total reads. The number of reads that aligned to the reference sequences decreased with diminishing amounts of viral RNA  revealed that, on average, only one read was perfectly aligned per 1000 nt of the TMV genome per 1 million total reads. Similar results were observed in all pools, indicating that only a low level of background noise was present in the analysis.Aligning the 21- to 24-nt reads from each pool to the genome of de novo analysis required 0.002 ng of viral RNA . Novoalign-based mapping of all the 21- to 24-nt reads from pool 3 to the GUS sequence reached nearly complete coverage (99%), with the mean depth of coverage being 12, i.e., each nucleotide in the reference sequence was covered by 12 reads on average. Smaller amounts of reference sequence-carrying viral RNA (0.00002–0.0002 ng) resulted in coverage of ca.25–50% of the reference sequence, whereas lower amounts of viral RNA (≤0.00001 ng) resulted in only sporadic read alignments to the corresponding reference sequence; however, the depth of coverage in those few positions was high in some cases.Detection of the reference sequences by the Velvet-based The standard RT-RT-PCR method (“Fera’s method”) used in the plant analysis laboratory unit of the Finnish Food Safety Authority Evira detected PVA and PVY in the pools containing ≥0.0002 ng of PVA RNA and ≥0.0001 ng of PVY RNA in 3 μg of total RNA, combined from miscellaneous plants, in a volume of 15 μl. The average Cq-values were 34.63 and 34.59 for the pools containing 0.0002 ng of PVA and 0.0001 ng of PVY RNA, respectively, which corresponds to 3.0 and 1.5 fg of viral RNA, respectively, in 50 ng of total RNA used for an RT-RT-PCR reaction. No amplification products were obtained when the RNA pools were tested by RT-RT-PCR without adding any PVY or PVA RNA (data not shown).There is increased interest in applying sRNA-based approaches to virus detection, as well as to official plant inspection. The aim of this study was to estimate sensitivity of the sRNA-based method, as compared with the established RT-RT-PCR approach. The question was addressed using two globally distributed, aphid-transmitted potyviruses, PVY and PVA, in the experiments. Three marker genes inserted in an infectious cDNA clone of PVA allowed us to distinguish the three engineered PVA clones and treat them as ‘strains’ of PVA. Total RNA from various crop plants was included in the samples to be analyzed by sRNA and RT-RT-PCR, to maintain the overall amount and concentration of RNA as constant as possible across the samples.de novo analysis required a 10-fold higher amount of viral RNA than detection by RT-RT-PCR. sRNA deep sequencing followed by Velvet-based de novo assembly of the reads to contigs ≥ 50 nt required 0.002 ng of viral RNA in 3 μg of total RNA in a volume of 15 μl. The de novo assembly approach is needed when no educated presumption is available about the viruses present. This is an advantage of sRNA-based virus detection, because sRNA-based diagnosis using de novo assembly of the reads can detect all viruses without the need for virus- or virus group-specific primers or probes. Higher amounts of viral sRNA allowed assembly of longer contigs and more reliable virus identification by comparison to the sequences available in databases such as NCBI.In this study, sRNA libraries were prepared from gel-extracted small RNA fractions (<30 nt) obtained from 3 μg of plant total RNA, resulting in 15–20 million reads per sample. Results showed that detection of PVA and PVY by the specified sRNA approach using Velvet-based In contrast, sRNA-based virus detection was 10 times more sensitive than RT-RT-PCR when knowledge was available about the viruses likely present in the samples. In this case, sRNA reads were mapped to the existing reference sequences suspected in the samples using the Novoalign software. Hence, as little as 0.00002–0.0002 ng of PVY or PVA RNA was sufficient to cover ca. 25–50% of the reference sequence with virus-derived sRNA reads. This coverage, albeit not complete, was high enough to conclude the presence of PVA and PVY. Higher amounts of viral RNA resulted in nearly full to complete coverage of the reference sequences. The RT-RT-PCR method used in the plant analysis laboratory unit of the Finnish Food Safety Authority Evira detected PVA and PVY in the pools containing ≥0.0002 and ≥0.0001 ng of PVY and PVA RNA, respectively. Hence, in this case sRNA-based diagnosis reached a similarly high sensitivity relative to RT-RT-PCR in the detection of the viruses.de novo analysis required the equivalent of 30 fg of viral RNA in 50 ng of total RNA per PCR reaction for detection of the viruses. However, with prior knowledge about the possible viruses present in the samples, detection sensitivity reached the level of 0.3 fg of viral RNA in 50 ng of total RNA per PCR reaction. The detection limit for PVY and PVA by RT-qPCR in this study was 3 and 1.5 fg of viral RNA, respectively, in 50 ng of total RNA per PCR reaction.Taken together, deep sequencing using the specific conditions described in this study and subsequent It is worth noting that the sensitivity of sRNA deep sequencing for virus detection can be adjusted during the process of sample preparation for sequencing and by depth of sequencing. For example, starting the library preparation from total RNA without gel-based or column-based separation of the sRNA fraction could diminish the sensitivity, because unwanted RNA remains in the library. Size selection can also be conducted using ready-made libraries. Deeper sequencing, i.e., obtaining more reads per sample, is expected to increase the sensitivity of virus detection but will also increase the cost of the analysis.Recent studies have compared sRNA-based virus diagnostics with sequencing of long viral RNAs (RNAseq) as two alternative approaches for virus detection based on next-generation sequencing. Both methods are also able to detect previously unknown viruses (JS made experimental design and carried out laboratory experiments. JS and JV interpreted the data and wrote the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Long non-coding RNAs (lncRNA) are a new class of regulatory molecules with diverse cellular functions. Recent reports have suggested that extracellular lncRNAs are detectable in human plasma and may serve as biomarkers. Here, we sought to investigate circulating lncRNAs as potential biomarkers for pulmonary arterial hypertension (PAH). Eighty-four lncRNAs, representing some of the most abundant and functionally relevant candidates identified in cellular studies, were assessed via RT-qPCR in plasma from PAH and healthy subjects. However, despite preamplification, the majority of lncRNAs were surprisingly undetectable or sporadically detectable, and showed no differential changes. Systematic characterization of plasma/RNA quality and technical performance via internal and external controls revealed no evidence of RNA degradation or RT-qPCR inhibition, and most lncRNAs were robustly detectable in pulmonary tissue. In plasma, lncRNA levels were the lowest among several different RNA species examined, and this was generalizable to other chronic and acute vascular conditions including coronary artery disease, acute coronary syndrome, and septic shock. In addition, two of three previously reported circulating lncRNA biomarker candidates were not detectable in any of the plasma samples. This study reveals new insight on the relative levels of lncRNAs in circulation, which has important implications for their potential development as biomarkers. Large scale transcriptional profiling studies have shown that 70–90% of the human genome is transcribed into RNA, though the majority of this RNA exhibits little or no protein coding potential1245910a priori whether certain clinical conditions may lead to perturbations in circulating RNAs. Thus far, alterations in plasma (or serum) lncRNA levels have only been reported in several conditions relating to heart failure15While the majority of studies have focused on the intracellular roles of lncRNAs, there is increasing interest in the potential roles of extracellular circulating lncRNAs. These studies build on the concept of circulating microRNAs, which have shown promise as non-invasive biomarkers across a wide spectrum of diseases and altered physiologic states. The release of cellular RNA species into circulation may reflect disease-specific tissue injury and/or remodelling, or potential intercellular signaling. However, the precise mechanisms through which different RNA species are released into circulation have not been elucidated, making it difficult to predict Pulmonary arterial hypertension (PAH) is a rare and fatal disease characterized by elevated pulmonary artery pressures and a progressive deterioration of the pulmonary vasculature. Although the prognosis for PAH is poor, early diagnosis and management can increase the length and quality of life for patients. However, prompt diagnosis remains a challenge because in early stages symptoms may be lacking or nonspecific, and there are currently no biomarkers to facilitate detection. A number of lncRNAs are known to be enriched in endothelial, smooth muscle, and inflammatory cells, and regulate key cellular processes of potential mechanistic importance to PAH , though their ability to serve as biomarkers has not yet been investigated in PAH patients.Here, we sought to determine whether a strategic selection of lncRNAs were consistently detectable in circulation, and could serve as non-invasive biomarkers of PAH. We used a hypothesis-driven approach and screened 84 specific lncRNA candidates that are among the most abundant and functionally relevant lncRNAs identified in cells to dateIn contrast to unbiased global profiling, we conducted a strategic hypothesis-driven screen of 84 specific lncRNAs for several reasons. First, these lncRNAs are among the very few that have been empirically validated to have cellular functions121319Plasma lncRNA levels were evaluated in an initial discovery cohort of healthy volunteers and PAH patients closely matched for age and sex . SurprisThe observed concentration of total RNA isolated from the plasma samples ranged from 2–5 ng/μL  used in the RNA extraction procedure could potentially inhibit subsequent reverse transcription and PCR reactions. At very low RNA concentrations, the standard metrics for assessing RNA purity via UV absorbance ratios (A260/280 and A260/230) may not accurately reflect the true quality and performance of the RNA in downstream enzymatic reactions. In our study, the A260/280 and A260/230 ratios varied from 0.76–2.2 and 0.09–0.53, respectively, which is below typical values of ~2.0 for pure RNA . HoweverThe preceding experiments indicate that the low lncRNA levels observed in our human plasma samples were not attributable to technical artifacts. We therefore sought to determine whether there may be inherent biological factors contributing to the low plasma lncRNA levels in healthy and pulmonary hypertension subjects. Because lncRNAs have been reported to be highly tissue specific, we first sought to verify the pulmonary expression of the 84 predefined lncRNA candidates. For this purpose, adjacent normal lung tissue specimens  were collected from lung cancer patients undergoing surgical lobectomies. The extracted lung RNA was confirmed to be pure  and largely free of degradation . We found that the majority of lncRNAs  were detectable without preamplification across four different lung tissue donors. Of the 84 measured lncRNAs, 62 were consistently detected across all subjects, 7 were sporadically detected, and 15 were not detected in any subjects . While nWe next sought to determine whether the low circulating lncRNA levels were also generalizable to other clinical conditions. We therefore examined a fairly diverse selection of patients with either coronary artery disease , acute coronary syndrome  or septic shock  . In contWe next sought to determine whether the low circulating lncRNA levels were generalizable to other lncRNAs that were not evaluated in our initial screening experiment. For this purpose, we selected three lncRNAs  that were reported to be elevated in plasma from patients with relevant conditions including heart failureThis is the first study to explore circulating lncRNAs in PAH patients. We specifically investigated whether 84 lncRNAs representing a strategic selection of candidates that are relatively abundant and functionally well-characterized in cells, were detectable in plasma and altered in PAH patients. No significant differences in plasma lncRNA levels were observed between the PAH and control groups. Furthermore, our results showed that the majority of these lncRNAs were scarcely detectable, despite preamplification prior to qPCR. We demonstrated that these unexpectedly low circulating levels were not an artifact of the quality of the plasma, extracted RNA, or general methodology used in this study, nor specific to only PAH patients. The limited detection of these circulating lncRNAs was also apparent in three additional and quite different clinical conditions, providing evidence of the generality of these observations.The very limited and sporadic detection of circulating lncRNAs was a surprising finding in this study, given the increasing number of studies that have suggested they might serve as useful biomarkers. We therefore systematically validated the quality of our samples and methods to exclude the possibility that these low levels were due to technical artifacts. First, we demonstrated that several different types of endogenous RNAs  as well as an exogenous spike-in RNA control (cel-miR-39) were robustly detectable, indicating that the extracted total RNA was not degraded. Second, a post-hoc analysis of lncRNA levels showed no correlation with specimen age, which argues against the likelihood of some partial age-dependent degradation. Third, we conducted mock RNA extractions that demonstrated no decline in total RNA integrity during the extraction procedure, and only a modest loss in the starting RNA mass quantity. This loss would generally be considered negligible because of the exponential amplification afforded by PCR. Finally, integrated positive controls for the downstream reverse transcription and PCR reactions revealed no evidence of reaction inhibition.To better understand potential biological determinants impacting circulating lncRNA levels, we conducted several further experiments. The pulmonary expression of the 84 lncRNAs was evaluated in human lung tissue, and confirmed that the majority of these lncRNAs were robustly detectable (without pre-amplification) across four different lung tissue donors that were examined. This is an important consideration given that lncRNAs exhibit higher tissue-specificity than protein-coding RNAsPrevious studies have reported elevated plasma levels of the lncRNAs Lipcar, TapSaki and Coromarker, in patients with heart failure241213Our demonstration that abundant and functionally relevant cellular lncRNAs were only poorly detectable in circulation has several important implications that will help to inform future studies. First, we now know that an unbiased global assessment of all lncRNAs may be a prerequisite to identifying rare biomarker candidates that are not only altered in circulation, but also readily detectable. Another important finding concerns the relative levels of different RNA species in circulation, which has not previously been characterized. Among the various ribosomal, messenger, small and micro- RNA species that were evaluated in this study, we showed that plasma levels were lowest for lncRNAs. These results provide useful insight for the development of clinical biomarker assays, since an ideal candidate should be readily measurable. Very low expression levels are generally associated with higher levels of variability, which may impact the reliability of a circulating lncRNA-based biomarker. Finally, this study provided information on the relative plasma levels of 84 specific lncRNAs, which could serve as a useful guide for future studies aimed at investigating one or more of these lncRNAs in other specific disease settings.This study has several potential limitations. One limitation is that we screened only a small fraction of the total number of known lncRNAs, and thus may have overlooked potentially rare biomarker candidates. Our experimental approach, however, was specifically designed to examine lncRNAs with empirically validated functions that might also be leveraged to provide insight into disease mechanisms. This was an important consideration given that the majority of lncRNAs have not been functionally characterized, and could merely represent non-specific transcriptional noiseIn summary, our study showed that some of the most abundant and functionally well-characterized cellular lncRNAs were only sporadically detectable in plasma. Although these lncRNAs are involved in biological functions that are relevant to known pathologic mechanisms of PAH, their levels were not altered in plasma limiting their usefulness as potential non-invasive biomarkers. This study also provides novel insight into the relative levels of different circulating RNA species, and highlights the additional challenges that must be considered in the potential development of circulating lncRNA-based biomarkers.Peripheral blood samples from PAH patients, CAD patients and healthy subjects were obtained with informed written consent between year 2011–2015. All methods were conducted in accordance with protocols, guidelines and regulations approved by the Ottawa Hospital Research Ethics Board (#2011470-01H). PAH patients were out-patients with a clinical diagnosis of PAH (either idiopathic or associated), and able to provide free and informed consent. Relevant clinical characteristics are summarized in Peripheral blood from PAH, CAD, and healthy volunteers was first drawn into a 3 ml SST tube, which was subsequently discarded to eliminate blood that contacted tissue during venipuncture. Blood was drawn into Becton-Dickinson (BD) vacutainer (sodium citrate) tubes, and centrifuged at 200 × g for 15 min (4 °C). The upper plasma phase was transferred into fresh microfuge tubes, and centrifuged twice at 11,000 × g for 2 min (4 °C) to remove residual cells/platelets/cell debris, with transfer of the plasma supernatant into fresh microfuge tubes between each spin. Plasma was stored at −80 °C. Peripheral blood from septic shock patients was initially drawn into BD vacutainers (sodium citrate), then transferred into a second vessel supplemented with sterile benzamidine , and centrifuged at 1700 × g for 10 min (4 °C). The upper plasma fraction was carefully removed (to within 0.2 mL of the plasma-cell interface) and transferred into cryo tubes for storage at −80 °C. Peripheral blood from ACS patients was drawn into BD vacutainers (sodium citrate) and centrifuged at 3000 rpm for 10 min at room temperature (~20 °C). The upper plasma fraction was carefully transferred into fresh tubes, centrifuged again at 3000 rpm for 10 min at room temperature, then transferred into fresh tubes for storage at −70 °C. Plasma samples showed no evidence of gross hemolysis as evaluated by visual inspection and Nanodrop absorbance measurement at 414 nm (hemoglobin).Adjacent normal lung tissue was procured with informed written consent from lung cancer patients  undergoing surgical lobectomies as standard care, and collected upon resection and evaluated in accordance with protocols approved by the Ottawa Hospital Research Ethics Board (Protocol# 20120559-01H). All methods were conducted in accordance with relevant guidelines and regulations. Normal lung tissue was identified by routine gross pathological examinations, excised and initially placed in cold tissue culture media and transferred to the laboratory. The samples were then rinsed in cold PBS, partially sliced to facilitate penetration of RNAlater (Ambion) stabilization reagent, incubated overnight at 4 °C and then stored at −80 °C. The total time from OR collection to stabilization in RNAlater reagent was less than 1.5 hr.2O.RNA extractions were performed in parallel on the same day for subjects within planned comparison groups, in order to minimize technical variations. Total RNA  was extracted from 200 μL of citrate-plasma (or ~25 mg of lung tissue) using the miRNeasy mini kit (Qiagen), which uses sequential organic extraction (5:1 ratio of Qiazol:plasma or 1000 uL Qiazol for 25 mg tissue) and silica spin-column purification of total RNA. Of note, the volume of patient plasma was limiting in some cases, such that 200 μL was the maximum volume that could be used consistently across all subjects. Frozen plasma samples were thawed and centrifuged at 11,000 × g for 5 min (4 °C) as an extra precautionary measure to remove any residual cell/debris contaminants prior to RNA extraction. After Qiazol-mediated denaturation of plasma samples, 2 μL of 5 nM cel-miR-39 (Qiagen) was spiked into each plasma sample. An extra on-column DNase digestion step was included in the RNA extraction of lung tissue samples, as recommended by the manufacturer. The total RNA was eluted in a final volume of 50 μL of RNase-free H2 lncRNA RT-PCR system (Qiagen), including the RNA reverse transcription control (RTC) and positive PCR control (PPC) to evaluate the quality of samples. Total RNA extracted from human lung tissue (~300 ng/uL) was confirmed to be pure  with minimal degradation .The concentration of total RNA extracted from plasma was generally close to, or below, the manufacturer’s stated detection limit of the Nanodrop 2000 spectrophotometer and the RNA 6000 Nanochip of the Agilent Bioanalyzer. The low RNA concentration precluded the reliable assessment of RNA purity by A260/A280 and A260/A230 absorbance ratios, and it was also too low for assessment of RNA integrity by Agilent Bioanalyzer “RIN” number. To circumvent these issues we utilized the integrated quality controls in the RT2 lncRNA PreAmp cDNA synthesis kit according to the manufacturer’s instructions (Qiagen). This includes an initial genomic DNA elimination step, isothermal cDNA conversion, and a 12 cycle preamplification (RT2 PreAmp PCR master mix and RT2 lncRNA PreAmp primer mix) to improve detection of low copy number lncRNAs. Of note, circulating lncRNAs were not detectable without this preamplification strategy, based on a pilot study conducted on a sample of 4 human plasma specimens. The pre-amplified cDNA was then used to quantify 84 predefined lncRNAs  using the RT2 lncRNA PCR Array Human lncFinder . For assessment of lung tissue levels of lncRNAs, a fixed mass of 800 ng of total RNA was input into the RT2 First strand kit (Qiagen) for cDNA conversion (with no pre-amplification). The resulting cDNA was used to assess 84 lncRNAs in the RT2 lncRNA PCR Array Human lncFinder. For assessment of plasma levels of Lipcar, Tapsaki and Coromarker, extracted total RNA was pre-amplified using the RT2 lncRNA PreAmp cDNA synthesis kit in combination with the RT2 PreAmp PCR master mix and primer sequences sourced from their original reports (and synthesized by Integrated DNA Technologies). The preamplified cDNA was then PCR amplified using the RT2 SYBR green PCR master mix according to manufacturer instructions. For assessment of circulating miRNAs, a fixed volume of total RNA extracted from plasma was input into the miScript II RT kit for cDNA conversion, and PCR amplified using the miScript SYBR green PCR kit and related primers (Qiagen).For assessment of circulating lncRNAs, a fixed volume of total RNA extracted from plasma was input into subsequent cDNA reactions (8 uL/rxn), using the RT−ΔCq, where ΔCq = Cq(target RNA) − 35(detection limit).The manufacturer’s recommended PCR cycling conditions for each type of PCR master mix were followed for 40 cycles on a CFX384 PCR machine (Biorad), with a terminal melt curve. PCR reactions were performed in duplicate. Standard negative controls including “no template” and “no reverse transcriptase” reactions were either negative or occasionally exhibited Cq values in excess of 35 (and below 40). Raw PCR Cq values were preprocessed prior to analysis by converting Cq values > 35 to the predefined detection limit of 35, according to manufacturer recommendations. Relative RNA levels expressed fold to the detection limit were calculated using the formula 2The unpaired student’s t-test (for parametric data) or Mann-Whitney U-test (for non-parametric data) were used for 2-group comparisons. For 3 or more group comparisons, the Kruskal-wallis test was performed with Dunn’s multiple comparison post hoc test (for non-parametric data). Specific statistical tests and measure of variation are specified in each figure legend. Statistical tests were performed with Graphpad Prism V5.0. All tests were performed 2-sided and a significance level of P < 0.05 was considered statistically significant.How to cite this article: Schlosser, K. et al. Assessment of Circulating LncRNAs Under Physiologic and Pathologic Conditions in Humans Reveals Potential Limitations as Biomarkers. Sci. Rep.6, 36596; doi: 10.1038/srep36596 (2016).Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


We report a study of referential choice in discourse production, understood as the choice between various types of referential devices, such as pronouns and full noun phrases. Our goal is to predict referential choice, and to explore to what extent such prediction is possible. Our approach to referential choice includes a cognitively informed theoretical component, corpus analysis, machine learning methods and experimentation with human participants. Machine learning algorithms make use of 25 factors, including referent’s properties (such as animacy and protagonism), the distance between a referential expression and its antecedent, the antecedent’s syntactic role, and so on. Having found the predictions of our algorithm to coincide with the original almost 90% of the time, we hypothesized that fully accurate prediction is not possible because, in many situations, more than one referential option is available. This hypothesis was supported by an experimental study, in which participants answered questions about either the original text in the corpus, or about a text modified in accordance with the algorithm’s prediction. Proportions of correct answers to these questions, as well as participants’ rating of the questions’ difficulty, suggested that divergences between the algorithm’s prediction and the original referential device in the corpus occur overwhelmingly in situations where the referential choice is not categorical. When the speaker’s/writer’s decision to mention a referent is in place, another discourse phenomenon becomes relevant: referential choice that is the process of choosing an appropriate linguistic expression for the referent in question. The question of reference per se, that is of how and why a speaker/writer decides which referent to mention at a given place in discourse, is out of the scope of this paper  and lexically full referential devices. In the case of English, it is the distinction between pronouns , on the one hand, and a variety of full noun phrases, on the other. This distinction is the first level of granularity in the domain of referential options, and all scales and hierarchies that relate givenness  to referential forms  texts. The corpus is annotated in accordance with the MoRA (Moscow Reference Annotation) scheme, detailed in Section “Materials and Methods” below. We assume that written media texts are a good testing ground for our approach. Specific aspects of referential processes differ across various discourse modes and types  (from the WSJ corpus we explore) illustrates the major referential options.(1)Ms. Bogart and The director’s attempt to Ø force some Brechtian distance between herBut beyond this decorative nod to tradition, Ms. Bogart and the director), as well as with a pronoun (her) and a zero . The second referent is mentioned by two different descriptions (Two referents recur a number of times in (1). They are emphasized with two different kinds of underlining: What factors influence actual referential choices in discourse? In usual face-to-face conversation, an entity sometimes become visually available to the interlocutors (via shared attention), and that may be enough for using an exophoric pronoun without any antecedent • those related to discourse structure, such as the distance between the anaphor and the antecedent, measured in the number of clauses or paragraphs.multi-factorial processes, generally characteristic of language production. Most of the factors employed in our study, such as animacy, grammatical role, or distance to antecedent, have been proposed in prior literature, in particular . Two kinds of activation factors operate in conjunction and determine a referent’s current degree of activation, which in turn predicts referential choice.Within the cognitive model we assume, these factors are interpreted as In to predict referential choice as reliably as possible. We explore a corpus of English written discourse and use machine learning techniques to predict referential choice maximally close to the original texts. This part of the study is reported in Section “Corpus-Based Modeling”. In the course of this work it is found that even well-trained algorithms sometimes diverge from the original referential choices in the corpus texts.We pursue two goals in this paper. The first goal is incomplete categoricity of referential choice. In Section “Experimental Studies of Referential Variation”, we submit the instances of divergence to an experimental assessment by human participants, in order to see whether people accept referential variation in the spots where divergences take place.That brings us to the second goal of our research: is 100% accurate prediction of referential choice possible in principle? In addressing this question, we consider the possibility that certain instances of divergence between the predicted and original forms may be due to the The discussion of our findings and concluding remarks follow in Section “General Discussion”.During the last twenty years or so a number of corpus resources for studies of coreference and reference production has appeared, including MUC-6/-7 , the ASGEarly modeling studies by A number of studies emphasized the role of discourse structure in referential choice. In his classical work, The WSJ MoRA 2015 corpus employed in this paper (we used the name “RefRhet corpus” for earlier versions in previous publications) is based on a subset of texts of the RST Discourse Treebank, developed by Daniel Marcu and his collaborators . This al“the most widely known coreference corpora < … > are annotated with relations between entities, not between discourse segments. The most widely known coherence corpora are Discourse GraphBank , RST TreReferential annotation added by A variety of algorithms have been used in computational studies of referential choice. One of the well-known early algorithms is the so-called incremental algorithm that was used by Table The WSJ MoRA 2015 corpus explored in this study consists of Wall Street Journal articles from the late 1980s, including broadcast news, analytical reviews, cultural reviews, and some other genres. Text length varies from 70 words to about 2000 words, the average length being 375 words. A general quantitative characterization of the WSJ MoRA 2015 corpus appears in Referential annotation of the corpus consists of two parts: annotation of referential devices and annotation of candidate activation factors. We consider these two kinds of annotation in turn.referential chain. Non-first members of a referential chain are termed anaphors below. The breakdown of markables by type is shown in Table Referential devices are technically named markables that is those referential expressions that can potentially corefer. Coreferential expressions form a Note that not every markable in the corpus is actually used for analysis. First, there are 2580 singleton markables that are not linked to any other markable by a coreference relation and are not pertinent to referential choice.  In the modeling task we only use those markables that form referential chains. Second, certain types of referential expressions are only considered as antecedents, but not as anaphors in our analysis of referential choice. This concerns the following categories:a(n), some, few, etc.);‒ indefinite descriptions . Such chains include 3199 markables used in the modeling tasks. Average chain length is 4.1 markables, and the maximum length of a chain is 52 markables.Table 3 shows the numbers of anaphors in the corpus.We thus address the basic referential choice between third person personal/possessive pronouns and full noun phrases. Table  For each factor, its values included in the study are listed after a colon. Most of the factors’ values are derived from the MoRA scheme annotation, but some are computed automatically.The second part or referential annotation addresses candidate activation factors that is parameters that are potentially useful for the prediction of referential choice. The complete list of candidate factors used in this study is shown in Table 4, the factors are listed in four groups. In the terms of Figure 1, the group 1 factors roughly correspond to the “Referent’s internal properties” activation factors, while group 2–4 factors to the “Discourse context” activation factors. For the sake of brevity, the logic of factors is somewhat simplified in Table  In particular, most factors include the value “other” that we omit here. Several of the factors call for clarifying comments.In Protagonism means referent’s centrality in discourse. Two models of protagonism were used : in the of the president or with her) a particular type of phrase, rather than a combination of a preposition with a referential device (noun phrase). First, referential choice may depend on whether the antecedent or the anaphor is a plain noun phrase, or a noun phrase subordinate to a preposition ; so this information must be retained. Second, consider English ’s- and of-genitives. The former are inflectional word forms and cannot be divided into a referential device and a separate unit, and it is reasonable to treat the two different kinds of genitives in the same way. More generally, in many languages, equivalents of English prepositions would be case endings, and nobody would deduct these from referential expressions.Regarding the “Type of phrase” factor, it is important to explain why we consider prepositional phrases  as it lends itself to natural interpretation. Second, we use logistic regression because its results often exceed those of logical algorithms in quality. In addition, we use the so-called classifier compositions: bagging  and boosIn the boosting algorithm the base algorithm undergoes optimization. An adaptation of classifiers is performed, that is, each additional classifier applies to the objects that were not properly classified by the already constructed composition. After each call of the algorithm the distribution of weights is updated. (These are weights corresponding to the importance of the training set objects.) At each iteration the weights of each wrongly classified object increase, so that the new classifier focuses on such objects. Among the boosting algorithms, AdaBoost was used in our modeling with C4.5 as the base algorithm.Bagging (from “bootstrap aggregating”) algorithms are also algorithms of composition construction. Whereas in boosting each algorithm is trained on one and the same sample with different object weights, bagging randomly selects a subset of the training samples in order to train the base algorithm. So we get a set of algorithms built on different, even though potentially intersecting, training subsamples. A decision on classification is made through a voting procedure in which all the constructed classifiers take part. In the case of bagging the base algorithm was also C4.5.In order to control the quality of classification, the cross-validation procedure was used:(1)The training set is divided into ten parts.(2)A classifier operates on the basis of nine parts.(3)The constructed decision function is tested on the remaining part.accuracy that is the ratio of properly predicted referential expressions to the overall amount of referential expressions. As was pointed out above, all the independent variables contained in Table 4 were treated as candidate factors of referential choice and included into our machine learning studies.The procedure is repeated for all possible partitions, and the results are subsequently averaged. The criterion for choosing both an optimal set of features and an algorithm is Table  The baseline means the frequency of the most frequent referential option, that is, full noun phrase. If an algorithm always predicted the most frequent option, its accuracy would equal that option’s frequency. Table 5 also includes information on three additional measures assessing the quality of classification: precision, recall, and F1 (or harmonic mean).The results of modeling the basic choice between reduced and full referential devices are given in The results yielded by any of the algorithms surpass the baseline substantially. At the same time, with the given set of factors all the algorithms demonstrate very close results; in particular, the accuracy rate is in the vicinity of 89-90%. The boosting algorithm fairs somewhat better than the others, but its difference from the other algorithms is not statistically significant.  and thus achieve higher overall accuracy  for the boosting algorithm appears in accuracy . It is hAmong the machine learning algorithms, decision trees may be particularly telling in explicitly specifying the concrete role of certain factors. For our corpus, a decision tree was generated that comprised 110 terminal nodes each corresponding to a specific prediction rule. Consider the following branch from the decision tree: if the anaphor is a prepositional phrase and its antecedent lies within the same sentence, then it is most probable that a full noun phrase will be chosen, not a pronoun. Of 100 instances observed, only 8 display pronominalization. A typical example can be seen in (2).(2)the Palestine Liberation Organization continues to Ø practice terrorism, and thus to persuade the U. S. to break off talks with the group.Israel has launched a new effort to prove This finding is quite surprising, given the closeness of the anaphor to the antecedent. The specific explanation of the finding is yet to be determined, but it is clear that the decision tree algorithms provide a source of new cause-effect generalizations about referential choice that would otherwise remain unrevealed.Table What is the role of individual factors to the success of prediction? In order to evaluate such role, we have applied the boosting algorithm to different subsets of factors in order to find out the individual contribution of factors or their combinations. The results are provided in Table 7 suggests that this group of factors is essential for successful prediction. As the distance factors are highly correlated, using any of them increases accuracy dramatically. Accuracy increases further if two or three distance factors are included. The non-distance factors have complex impact on accuracy: eliminating them one by one does not impair prediction significantly, but removing all of them results in a significant decrease of accuracy and is therefore inadvisable.We used a number of distance measurements in this study. The data in An earlier study of our group  specificTable  The baseline is the frequency of descriptions, the most frequent referential option.The set of candidate activation factors employed in this study is derived from the vast tradition of studies on basic referential choice. We have reached a significant success in predicting the basic choice. Now, what governs the second-order choice between the types of full noun phrases, that is, proper names and descriptions? Studies of these issues are relatively few cf. . We haveThe fairly high accuracy of prediction we have obtained for the three-way task is intriguing. Apparently, the factors responsible for the choice between proper names and descriptions substantially intersect with our basic set of factors. This issue requires further investigation.Note that in the three-way task boosting again demonstrates the highest results, as it did in the two-way task. Even though the advantage of boosting over the other methods again is not statistically significant, the tendency of its good performance motivates our solution to employ this method in the subsequent part of this study. Even though the machine learning modeling was quite successful, the accuracy of prediction of the basic referential choice is still quite away from 100%. An important question arises: if we continue improving our annotation  and tuning up the modeling procedure, can referential choice be ultimately predicted with the accuracy approaching 100%? In other words, is the 10% difference between the algorithm’s prediction and the original texts due to certain shortcomings of our methods or to some more fundamental causes? We propose that complete accuracy may not be attainable due to the nature of the process of referential choice.Referential choice appears to not be a fully categorical and deterministic process. True, there are many instances in which only a pronoun or only a full noun phrase is appropriate, but there are also numerous instances in which more than one referential option can be used. This issue was explored in (3)i. full NP only?pronounii. full NP, iii. either full NP or pronoun?full NPiv. pronoun, v. pronoun only.Figure 4. That is, the formulation of the main law of referential choice, as offered in Section 1, suggests an overly categorical representation. It only captures correctly the two poles of the activation scale, but there are intermediate grades of activation in between that lead to less than categorical referential choice. The model of referential choice that we propose, as shown in Figure 4, differs from the well-known hierarchies of In (3), situations i and v are fully confident, or categorical, in the sense that language speakers would only use this particular device at the given point in discourse. Situations ii and iv suggest that, in addition to a preferred device, one can marginally use an alternative question-marked device. Finally, situation iii means free variation. In Non-categorical and/or probabilistic nature of referential choice has previously been addressed in a number of studies e.g., . For exaCloser examination of the data of individual participants of almost any study reveals that their responses vary substantially, even within a single experimental condition. For example, we examined the data of Overall, there is accumulating evidence suggesting that human referential choice is not fully categorical. There are certain conditions in which more than one referential option is appropriate and, in fact, each one would fare well enough. Under such conditions human language users may act differently on different occasions. If so, an efficient algorithm imitating human behavior may legitimately perform referential choice in different ways, sometimes coinciding with the original text and sometimes diverging from it. Therefore, ideal prediction of referential choice should not be possible in principle.We have designed an experiment in which we attempt to differentiate between the two kinds of the algorithm’s divergences from the original referential choices. Of course, there may be instances due to plain error. But apart from that, there may be other instances associated with the inherently non-categorical nature of referential choice.As was discussed in Section “Discussion: Referential Choice Is Not Always Categorical”, referential variation and non-categoricity is clearly gaining attention in the modern linguistic, computational, and psycholinguistic literature. Referential variation may be due to the interlocutors’ perspective taking and their efforts to coordinate cognitive processes, see e.g., We discuss a more general kind of referential variation, probably associated with the intermediate level of referent activation. This kind of variation may occur in any discourse type. In order to test the non-categorical character of referential choice we previously conducted two experiments, based on the materials of our text corpus. Both of these experiments were somewhat similar to the experiment from comprehension experiment, In a 3 with 60 participants with the expert level command of English. Each participant was asked to read all the nine texts one at a time, and answer a set of three questions after each text. Each text appeared in full on the screen, and disappeared when the participant was presented with three multiple-choice questions about referents in the text, beginning with a WH-word. Two of those were control questions, related to referents that did not create divergences. The third question was experimental: it concerned the referent in point, that is the one that was predicted by the algorithm differently from the original text. Questions were presented in a random order. Each participant thus answered 18 control questions and nine experimental questions. In the alleged instances of non-categorical referential choice, allowing both a full NP and a pronoun, experimental questions to proper names  and to pronouns (predicted) were expected to be answered with a comparable level of accuracy.The experiment was run online on Virtual Experiments platformThe accuracy of the answers to the experimental questions to proper names, as well as to the control questions, was found to be 84%. In seven out of nine texts, experimental questions to pronouns were answered with the comparable accuracy of 80%. We propose that in these seven instances we deal exactly with non-categorical referential choice, probably associated with an intermediate level of referent activation. Two remaining instances may result from the algorithms’ errors.editing experiment  and 134 predicted full NPs . For the purpose of expert evaluation modified versions of all 48 texts were created, with the use of the flexible script. In the modified texts, original full NPs were replaced by pronouns , and, conversely, original pronouns were replaced by the most obvious descriptive designation of the referent (same as used in the text elsewhere), such as The modified texts were analyzed by one of the coauthors of this paper. As a result of text assessment, the following most common types of undoubted referential errors were detected: use of a full NP in the context of syntactic anaphora and non-cataphoric third person pronouns at the beginning of a referential chain. Example (4) demonstrates a text excerpt with two predictions not matching the referential expressions found in the original text. The original referential expressions are underlined, and the divergent predictions of the algorithm are indicated in brackets, followed by a specific referential form as used in the experiment. Prediction < 2 > was rated by the expert as potentially fitting, whereas prediction < 1 > was rated as an obvious error, namely a full NP predicted in the context of syntactic anaphora.(4)her [full NP: the director’s] intent in such manipulative staging of the classics is simply an attempt to make it new. Indeed, during a recent post-production audience discussion, < 2 > the director [pronoun: she] explained that her fondest artistic wish was to find a way to play < … >Like Brecht, and indeed Ezra Pound, Ms. Bogart has said that < 1 > Table 3).The analysis detected 26 undoubted referential errors that constituted 11% of all divergent predictions and just 1.2% of all referential choices predicted by the algorithm  is not generated, and therefore a modified text would contain a referential choice supplied by a human experimenter. Second, this kind of divergence is much more informative: as was discussed in Section “Results”, class imbalance leads to the algorithms’ general predisposition to predict full NPs.The resulting experimental set, containing all the texts matching the selection criteria, consisted of six texts.  We created a modified version of each text: the original full NP was replaced by a predicted pronoun. Then two experimental lists were created, each containing six texts, of which three were in a modified version and three texts were the original ones from the corpus.who, whom, whose, or what) were used in the experimental questions. One of the control questions was also a WH-question, while the other one was a polar (yes-no) question.Three questions for each text were formulated: one experimental and two control ones. Each experimental question concerned a relevant referential device, that is, one of those for which a pronoun was predicted by the algorithm. WH-words (An example of a text can be seen in (5), with the original full NP underlined, followed by the predicted pronoun in brackets. The three questions are provided below with correct responses in parentheses, and the experimental question is underlined.(5)Mr. Petrie’s [his] 2,331,100 Deb Shops shares. The transaction will take place tomorrow. The filing said Petrie Stores of Secaucus, N.J. is purchasing Mr. Petrie’s Deb Shops stake as an investment. Although Petrie Stores has considered seeking to acquire the remaining equity of Deb Stores, it has no current intention to pursue such a possibility, the filing said. Philadelphia based Deb Shops said it saw little significance in Mr. Petrie selling his stock to Petrie Stores. We do not look at it and say, ‘Oh my God, something is going to happen,’ said Stanley Uhr, vice president and corporate counsel. Mr. Uhr said that Mr. Petrie or his company have been accumulating Deb Shops stock for several years, each time issuing a similar regulatory statement. He said no discussions currently are taking place between the two companies.Milton Petrie, chairman of Petrie Stores Corp. said he has agreed to sell his 15.2% stake in Deb Shops Corp. to Petrie Stores. In a Securities and Exchange Commission filing, Mr. Petrie said that on Oct. 26 Petrie Stores agreed to purchase Whose shares will Petrie stores purchase? (Mr. Petrie’s)Where are Deb Shops based? (Philadelphia)Does Stanley Uhr work for Petrie stores? (no)4 Each text appeared on the screen one line at a time. In the experiment we presented the texts as closely to their original appearance in the newspaper as possible, so the line length was approximately 40 characters, which matches the size of a column in Wall Street Journal. In order to see the following line of the text a participant had to press a button. Prior text did not disappear from the screen. The self-paced reading design was chosen to ensure that the participants would pay attention to all elements of the experimental texts. After the participants finished reading the text, three questions, one experimental and two control ones, appeared on the screen in a randomized order, one at a time, with the text remaining visible. Since the experimental texts are quite hard for readers , answering questions without the texts remaining available could result in an excessive rate of errors.The experiment was run online using the Ibex Farm platformParticipants were also asked to rate the difficulty of each question on a 5-point scale, ranging from 1 “very easy” to 5 “very hard”.Twenty four people, including 17 females and 7 males, aged 25 to 36, took part in the experiment. All participants were native speakers of English with college-level education and explicitly stated their willingness to voluntarily participate in the experiment.Table Experiment participants answered 18 questions each, that is three questions per text. All participants provided 15 or more correct responses; the number of incorrect responses by participant is summarized in Table  The ratings are shown in the right hand part of Table Questions can be divided into three groups: experimental questions to original referential expressions, experimental questions to modified (predicted) referential expressions, and control questions. All questions were answered correctly by at least 75% of the participants. The numbers and percentages of correct responses are shown in p = 0.001, CI 90% ). This demonstrates that, statistically, the overall perceived correctness does not differ for the original and modified texts. The same test was applied to check for statistical equivalence of correct response rates to experimental questions , as opposed to responses to control questions. The two groups were proved to be statistically equivalent for the threshold of 10% .In order to test the equivalence of correct response rates for the three groups of questions we performed the TOST (two one-sided tests) equivalence test  that treWe thus did not detect differences between the human understanding of original and predicted referential expressions, and it appears that in the analyzed texts instances of divergent referential choice occur in the situations in which either a full NP or a pronoun is appropriate from a human language user’s perspective.The results of both evaluation studies support the idea that the divergent referential options predicted by the algorithm mostly occur in the situations in which a human language user accepts either referential form, or processes both the original and the predicted forms equally well.Expert evaluation suggests that the majority of discrepancies between the original texts and the algorithm predictions do not result from outright algorithm errors, but rather can be interpreted as equally appropriate referential expressions. The results of the experimental evaluation suggest that, in the selected texts, replacement of a full NP by a pronoun, as predicted by the algorithm, does not lead to increased comprehension difficulty, measured both objectively (correctness of responses) and subjectively (question difficulty ratings). Though the nature of experimental evaluation does not allow us to test all the instances of divergent predictions, the observed results demonstrate that both the original and the predicted referential forms may quite often be equally appropriate.In experimental evaluation, participants answered questions about the original and modified texts and thus played the role of discourse interpreters, rather than producers. A certain caution must be exercised when extending the experiment results to referential choice, which is a part of discourse production. One might possibly argue that, even if readers allow for more than one referential option, human writers would still perform referential choice in a categorical and deterministic way. Clearly, further experimentation is required, putting human participants in a position closer to that of a discourse producer. Note, however, that the earlier editing experiment reported in Section “Related Work”  also indOverall, we propose that human evaluation of machine learning results provides more precise information about the appropriateness of referential choice prediction than the traditional accuracy measurement. Only human language users can detect whether the divergent referential choices offered by the machine are actually appropriate, and thus provide us with a clear understanding of the algorithm’s error rate.The approach we used in this study is characterized by several major conceptual elements. First, we mostly focused on the basic referential choice between full and reduced referential devices, also looking occasionally into the second order distinction between two kinds of full NPs: proper names and descriptions. Second, as is suggested by extensive prior research, we took into account a multiplicity of factors affecting referential choice. The factors we have analyzed fall into two major groups: stable referent properties and flexible factors associated with the discourse context, that latter involving several distances from an anaphor to the antecedent. Third, we used a corpus of texts, sufficient from a statistical point of view. The corpus was annotated for reference and for multiple parameters that potentially can serve as factors of referential choice. Fourth, we employed a cross-methodological approach, combining the corpus-based computational modeling and experimentation with human participants.Two main findings result from this study, the first one concerned with computational prediction of referential choice, and the second one with the limits of such prediction. Below we summarize them in turn.Machine learning techniques were used to predict referential choice at each point where an anaphor occured in the corpus texts. In most previous machine learning-based studies of referential choice authors primarily used decision trees. In contrast, our study is characterized by the use of a wide variety of machine learning algorithms, including classifier compositions. Trained models provided almost 90% accurate prediction of referential choices and demonstrated that machine learning algorithms can imitate referential choices made by human language users with substantial success. We also explored the cumulative and individual contribution of various factors to the resulting referential choice.In spite of the relatively successful modeling results, prediction accuracy did not approach 100%, and this raised the question of whether complete accuracy is attainable. In order to address this question, we used experimentation with human participants. We submitted the results of modeling to human judgment and assessed the divergences between the original and predicted referential choices as appropriate or inappropriate from the language users’ point of view. Experiment results suggest that there are numerous instances in which referential options are equally appropriate for human participants. Accordingly, many of the algorithm’s failures to predict referential choice exactly as in original texts may be due not to plain error but to inherently not fully categorical nature of referential choice. Even a perfect algorithm  could not be expected to necessarily make the choice once implemented in a text. In other words, a certain degree of variation must be built into a realistic model of referential choice. Even if the algorithm learns to imitate non-categorical referential choice . In this model, the linguistic factors affecting referential choice are interpreted as activation factors. Operating in conjunction, they contribute to a referent’s current activation, which, in turn, determines referential choice. In some of our previous studies , and this study contributes to the duscussion of the possible kinds and causes of such variation. The outcome of this study thus provides support to the previously expressed idea that “non-determinism should be an important property of a psychologically realistic algorithm”  and the output layer , the internal working of the algorithms remaining hidden. We would like to reinstate the cognitive interpretation that is the degrees of activation that result from the activation factors in conjunction and directly map onto referential choice. One way how this can be done is associated with some algorithms’  capacity to evaluate the contribution of various factors and the certainty of prediction, which can be interpreted as activation factors and summary activation level, respectively. This can also be a path to training the algorithms to model non-categorical referential choice.Figure 1 is simplified in that it leaves out the filter of referential conflict, or ambiguity, that modulates referential choice after referent activation is computed . A more focused search for factors directly related to this choice is in order. Also, the proposed approach can be extended to further details of referential choice, such as varieties of attributes in descriptions, as well as less frequent referential options, e.g., demonstratives. We also believe that our approach can be used in the domains of language production other than referential choice.In this study we looked at written discourse, as a well-controlled testing ground for sharpening the methods of cognitive and computational modeling and as the material easily lending itself to various kinds of manipulation. We assume that, in spite of the special character of newspaper texts, written discourse is created on the basis of general cognitive principles of discourse production, including referential choice, and that the discovered regularities can in principle be extended to other types of language use. Nowadays, linguistic research is opening up new horizons, including interest in interactive face-to-face communication, visual context, and multimodality. All of these developments are also relevant to the study of referential choice, see e.g., AK has conceived the general design of the study, developed the theoretical framework, selected the corpus for analysis, put together the team, allocated the assignments to coauthors, formulated the general structure of the paper, wrote Sections “Introduction” and “General Discussion”, drafted “Corpus-Based Modeling”, and edited the whole text. MK worked substantially on the corpus, developed the annotation scheme, organized the work of student assistants, designed the experimental part, conducted the experiment, and wrote Section “Experimental Studies of Referential Variation”. GB provided expertise on machine learning, conducted multiple modeling studies, helped to plan the whole study, wrote parts of Sections “Corpus-Based Modeling” and “Experimental Studies of Referential Variation”. AL provided expertise on discourse annotation, natural language generation, psycholinguistic experimentation, wrote literature surveys, complied the bibliography, performed technical editing of the paper, provided big input on all aspects of the paper. DZ worked on the corpus, organized the work of student assistants, wrote “Corpus-Based Modeling”, performed technical editing of the paper, provided big input on all aspects of the paper. All coauthors participated in developing the design of the study, acquiring the data, analyzing data, writing up the manuscript, contributed to multiple manuscript revision throughout all of stages of paper preparation.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.The reviewer CL and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.


Bartonella henselae is a gram‐negative zoonotic bacterium that causes infections in humans including endocarditis and bacillary angiomatosis. B. henselae has been shown to grow as large aggregates and form biofilms in vitro. The aggregative growth and the angiogenic host response requires the trimeric autotransporter adhesin BadA. We examined the transcriptome of the Houston‐1 strain of B. henselae using RNA‐seq revealing nine novel, highly‐expressed intergenic transcripts . The Brt family of RNAs is unique to the genus Bartonella and ranges from 194 to 203 nucleotides with high homology and stable predicted secondary structures. Immediately downstream of each of the nine RNA genes is a helix‐turn‐helix DNA‐binding protein  that is poorly transcribed under the growth conditions used for RNA‐seq. Using knockdown or overexpressing strains, we show a role of both the Brt1 and Trp1 in the regulation of badA and also in biofilm formation. Based on these data, we hypothesize that Brt1 is a trans‐acting sRNA that also serves as a cis‐acting riboswitch to control the expression of badA. This family of RNAs together with the downstream Trp DNA‐binding proteins represents a novel coordinated regulatory circuit controlling expression of virulence‐associated genes in the bartonellae. Immunocompetent individuals infected with B. henselae typically suffer from cat scratch disease (CSD), which is a self‐limiting infection. In contrast, immunosuppressed individuals infected with B. henselae can develop systemic infections including bacillary angiomatosis and bacillary peliosis, characterized by vasoproliferative tumor‐like lesions on the skin and liver, respectively  and IL‐8  BadA. BadA is a large surface‐exposed filament‐shaped protein of 3036 amino acids and a length of 240 nm  function by binding target mRNAs, binding proteins, or by targeting and degrading specific mRNAs  that are unique to Bartonella species. All nine of these RNAs are immediately upstream of a coding region for a protein that is annotated as a helix‐turn‐helix DNA‐binding protein/transcriptional regulator . We show a role for the most highly transcribed member of this RNA family, Brt1, in regulating badA and we further show that the cognate Trp1 positively controls badA expression. We hypothesize that the Brt family of RNAs are capable of functioning as a two‐tiered system including both cis‐acting and trans‐acting roles to coordinate regulation of badA, which is responsible for autoaggregation, host cell attachment, the proangiogenic host response, and biofilm formation by B. henselae.Here, we describe a highly transcribed family of nine small RNAs  supplemented with 10% fetal bovine serum for 3–4 days at 37°C in the presence of 5% CO2 as previously described  has been previously described  and pelleted and frozen at −80°C. The pellets were thawed at 4°C before RNA isolation. Total RNA was extracted using the RNeasy Mini Kit . Following extraction, RNA was treated with DNase . RNA quality was analyzed using the Agilent 2100 Bioanalyzer.2.3B. henselae Houston‐1 reference genome from NCBI  for RNA‐Seq, Oligo Magbeads from the MICROBExpress Kit  that hybridize with the 16S and 23S rRNAs were used. A quantity of 3.4 μg of RNA was used for rRNA removal. The RNA sample was analyzed on the Bioanalyzer to monitor removal of rRNAs. The subsequent RNA processing was done using the Ion Total RNA‐Seq Kit v2 . The enriched RNA was fragmented with RNase III and run on the Bioanalyzer to analyze the size of the fragmented RNA. The fragmented RNA was then reverse transcribed into cDNA and used as a template for sequencing using an Ion Torrent Personal Genome Machine (Thermo Fisher Scientific). The sequenced transcriptome was aligned to the 2.432P]ATP using T4 polynucleotide kinase . Unincorporated [γ‐32P]ATP was then removed using a QIAquick Nucleotide Removal Kit . B. henselae strains were cultured in Schneider's medium or Schneider's medium with 50 μg/ml kanamycin for 72 hr and total RNA was isolated with the RNeasy Mini Kit and treated with Turbo DNase. Four microgram of DNase‐treated total RNA was separated on a 10% denaturing polyacrylamide mini gel. The RNA was transferred to an Immobilon‐Ny+ Membrane (Millipore). Following transfer, the RNA was UV cross‐linked to the membrane. Prior to hybridization, the membrane was prehybridized in ULTRAhyb‐Oligo Buffer  at 42°C for 1 hr. The labeled probe was denatured by heating at 95°C for 5 min and added to the membrane to allow for hybridization overnight at 42°C in a hybridization oven. After hybridization, the membrane was washed for 15 min, each at 42°C sequentially with: 2X SSC, 1X SSC, and 0.5 SSC. The membrane was then exposed to a phosphorimager screen overnight. The image was developed using a Typhoon 9410 .A 29 base oligonucleotide probe  was designed to base‐pair to a conserved sequence found in all nine Brts. For detecting antisense Brt1 RNA, a second probe which hybridized to RNA transcribed from the opposite (antisense) strand was used . The probe was labeled with [γ‐2.5brt1‐9 transcription start and stop sites were determined by RNA‐seq according to the start and end of the RNA transcripts. Brt nucleotide and Trp nucleotide and amino acid sequences were obtained from the B. henselae Houston‐1 genome sequence from NCBI (accession # BX897699). Nucleotide sequence alignment and percent identity was carried out using the multiple sequence alignment tool (MUSCLE) from EMBL‐EBI  were expected to amplify the entire Brt1 gene sequence from 2.7trp1 gene were designed to amplify the gene from genomic B. henselae Houston‐1 DNA from the second codon of the gene to beyond the stop codon . The resulting amplicon was ligated into the BamHI and XbaI restriction sites of pNS2Trc to create an overexpressing Trp His‐tag fusion protein. The plasmids were transformed into competent DH12S E. coli cells and selected for colonies harboring the plasmid with 50 μg/ml kanamycin. PCR was used to screen for successful ligation of the trp1 gene into the pNS2Trc plasmid in the intended orientation that would allow overexpression of the gene. The plasmids were then electroporated into B. henselae Houston‐1 as previously described , and converted into cDNA using the iScript cDNA synthesis kit . qRT‐PCR was performed in 25 μl reaction volumes that include 12.5 μl of the 2X Maxima SYBR Green/Fluorescein qPCR kit , 300 nmol of each gene‐specific primer, and 2 μl cDNA. All reactions were performed in triplicate and 50S ribosomal protein L4 (rplD) was used as the reference gene for normalization. The qRT‐PCR cycling parameters included 95°C for 3 min, 40 cycles of 95°C for 10 s, and 60°C for 30 s, followed by 95°C for 45 s and 55°C for 1 min. Melting curve analysis was performed to confirm that no primer dimers were amplified. Results were analyzed using the comparative CT method . Since BadA is a surface trimeric autotransporter adhesin, it exists in B. henselae as a high‐molecular weight  fibrillar complexes  in PBS was spotted onto a nitrocellulose filter. The membrane was then blocked with PBS +0.05% Tween 20 containing 5% skim milk. Antibody diluted 1:200 in PBS +5% skim milk was added and the filters were incubated for 2 hr at room temperature and washed four times in PBS +0.05% Tween 20. The resulting filters were incubated in goat anti‐rabbit conjugated with horseradish peroxidase  diluted 1:10,000 in PBS +5% skim milk for 1 hr at room temperature. After three washes with PBS +0.05% Tween 20, the filters were incubated with ECL luminescent substrate (Thermo Scientific) for 1 min and detected using a Bio‐Rad Chemidoc XRS Imaging system.To further determine if qRT‐PCR for transcription of the 2.10B. henselae biofilm formation. This method is based on the standard crystal violet microplate assay for gram‐negative bacteria  and the plates were incubated for 3 days at 37°C with 5% CO2. After incubation, the plates were washed with water and subsequently stained with 125 μl of a 0.1% crystal violet solution for 15 min. After washing, the plates were allowed to dry for 3 hr and the stain was extracted with 125 μl 30% acetic acid for 15 min. The stain solution was transferred to a new 96‐well plate and the OD550 measured using a plate reader .A standard microplate adherence assay was used to measure O'Toole,  that has2.11t‐test, with a p‐value of <.05 being considered statistically significant.The means ± standard errors were presented in the data. SigmaPlot software  was used for statistical analysis using the 33.1B. henselae, grown under standard growth conditions in Schneider's liquid medium at 37°C with 5% CO2 for 72 hr as previously described,  to identify each noncoding RNA. The genome location as well as the start and stop of transcription for all nine Brts can be found in Table S1. Specific details of the RNA‐seq results are found in File S1.The transcriptome of 3.2B. henselae genome  of the e Figure . This ree Figure , which h3.3Nucleotide sequence alignment of Brt1‐Brt9 reveals a high level of conservation across the length of Brt1‐9 ranging from 74% to 100% Figure , panel b3.4Bartonella, as no similar sequences were found in other alpha‐proteobacteria or any other bacteria. Each individual Brt nucleotide sequence was used as a query sequence against the NCBI genome database using the BLAST program and only Bartonella genomes were returned as results. The copy number of the Brt family varied greatly among Bartonella species and strains from nine copies found in B. henselae Houston‐1, seven each in the B. henselae cat isolates BM1374163 and BM1374165, 33 to 34 in B. tribocorum, 26 and one plasmid in B. grahamii, eight in B. vinsonii subsp. berkhoffii, and two or three in B. schoenbuchensis. Interestingly, no copies of the brts could be found in the genome of B. bacilliformis or B. rochalimae  in the Bh transcriptome (File S1). The presence of Brt RNAs has also been confirmed by northern blot  were far higher (>10‐fold) for Brt1 than for any of the other Brt family of RNAs indicating that more copies of this RNA were present than for Brt2‐Brt9 combined Figure a. In fact Figure b, qRT‐PCt Figure c, and int Figure . Accordi3.6brts shows that all of them are immediately upstream of a coding region for a family of putative transcriptional regulatory proteins (trps). There are two versions of the genome annotation of the Houston‐1 strain of B. henselae available online with some variation within the length of the coding region for these transcriptional regulators that results in slightly different size intergenic regions. It should be noted that the trp8 gene downstream of brt8 appears to encode a pseudogene with a coding region still present, but disrupted by two stop codons. The other remaining eight brts retain virtually the identical gene arrangement with respect to the downstream trp . However, none of these gene copies are found adjacent to a Brt RNA like those identified with RNA‐seq. The number of trp genes found in various Bartonella species and whether they are linked to a brt sequence on the genome is shown in Table S3.A closer examination of the genome loci for the nine p Figure a. Sixtee3.7brt codes for a family of small DNA‐binding proteins (Trps). The deduced amino acid sequence from the Trp coding region downstream of each of the Brt RNAs (except the pseudogene behind brt8) is well conserved , there are 16 additional trp‐like genes found scattered throughout the B. henselae genome (Table S2). It is interesting to note that phylogenetic analysis of the deduced amino acid sequence for all 24 putative Trps (or Trp‐like proteins) shows that all eight of the proteins that are encoded downstream of the Brt RNAs cluster together, branching out from a single node on the tree  present in the genome  approach to block Brt function. Specifically, to determine if Brt1 functions as a small regulatory RNA that plays a role in gene regulation, we constructed an antisense (as) construct/strain designed to base‐pair and destabilize the Brt1 RNA. Since trans‐acting RNAs typically act by base‐pairing, we reasoned that asBrt1 RNA would block the function of Brt1 and possibly also destabilize the RNA and make it prone to degradation. Using primers that would only detect vector sequences found in the antisense Brt1 RNA, transcription of high levels of antisense Brt1  mRNA (Table B. henselae genes among the top predicted targets include a regulatory protein (BH11940) containing a conserved domain known to activate sigma factor 54  as well as two copies of the filamentous hemagglutinin genes (BH06670 and BH07150) and four ABC transporter genes . In addition, genes encoding VirB type IV secretion system components VirB6 (BH13300) and VirB9 (BH13330) were predicted targets of Brt1 (Table badA gene (BH01510) mRNA was predicted to interact with Brt1 but with a predicted ∆G of −1.73 kcal/mol indicating low stability of this interaction. The top 100 predicted mRNA targets from each algorithm are shown in Tables S4 and S5.To examine possible mRNA targets of Brt1, the RNApredator and IntaRNA prediction programs were run using Brt1 as the query and the genome of the Houston‐1 strain of  bepC BH1400 mRNA  bepC BH1400 mRNA  bepC BH1400 mRNA 4Bartonella consists of over 30 species of bacteria found in a wide range of animal hosts with most thought to be transmitted by an arthropod vector  or two (B. quintana), while other species contain a wide range from a moderate number (B. henselae) to the highest number of copies/genome in the rodent‐associated B. tribocorum , with the amino terminal to central regions of the protein, the most highly conserved. All of the Trps have a helix‐turn‐helix DNA‐binding domain located in this conserved region and most are annotated in the B. henselae genome sequence as xenobiotic response elements (XREs). Bacterial XREs are DNA‐binding proteins that have a conserved positively charged helix that is thought to bind the major groove of DNA  that are more distally related  was noticed to have an adherent colony phenotype leading us to examine badA gene expression. Bhh72 had increased badA mRNA and BadA protein as determined by dot‐blot analysis and was further shown to have an increased ability to form biofilms. It is possible that this increased badA expression is due to the asBrt1 functioning to alter secondary structure of the Brt1 and allowing transcriptional read‐through into the downstream trp1 gene. Further studies to determine the sequence and abundance of Brt and trp1 RNA species are needed to answer this question and test our hypothesis that Brt1 functions in trans to negatively regulate badA in a manner that is independent of trp1 transcription.The high levels of Brt RNA transcription combined with their size and secondary structure are also suggestive of trans‐acting sRNAs. To test this hypothesis, we attempted to make a Brt1 deletion strain of s Figure a and genbadA expression and as a possible riboswitch controlling transcription of the downstream trp1. While we have not yet identified ligands or conditions that activate this putative riboswitch, it is clear that trp1 expressed on a plasmid results in upregulation of badA, the same gene negatively regulated by the associated Brt1 RNA. Based on the experimental data presented in this study, we propose a model for regulation of badA by Brt1/Trp1. Firstly, Brt1 functions as a trans‐acting sRNA to repress badA expression possibly targeting the badA mRNA directly or through an intermediate RNA. Secondly, we hypothesize that Brt1 also acts on badA by a second mechanism in which a cis‐acting riboswitch on the 3′ end of Brt1 adopts an antitermination configuration to allow transcription of the downstream trp1. Trp1 in turn functions as a DNA‐binding protein to activate badA expression which facilitates autoaggregation and biofilm formation  has only been described for the SreA and SreB trans‐acting riboswitches that controls virulence in Listeria monocytogenes  was not among the top targets and was predicted to only weakly interact with Brt1, raising the possibility that Brt1 acts on badA by an indirect mechanism. Alternatively, it is also possible that an RNA‐binding protein, such as Hfq, is involved in these interactions and allows Brt1 to bind a target mRNA in the absence of extensive base‐pairing. It should also be noted that discrepancies were observed between these two sRNA target prediction algorithms and while some targets were in common (PhaC pH adaptation transporter and LipA lipoyl synthase), most targets were unique to one of the algorithms. Specific target mRNAs of Brt1 and the other Brt family RNAs will require experimental documentation. Similarly, the function of Trp1 could be direct or indirect. We do not yet know if Trp1 functions by binding upstream of badA or another target gene that affects badA gene expression. We have previously shown that the general stress response system of B. henselae controls badA gene expression and it is possible that Brt1 and/or Trp1 acts through this regulatory system.Questions remain to be answered about how the Brt1 RNA and the Trp1 protein mediate the effect on the B. henselae grown for 72 hr at 37°C in the presence of 5% CO2 in standard culture media. It is certainly possible that under different growth conditions, the expression patterns of the Brt RNA, the Trp protein genes, and the badA gene might be different. We hypothesize that different growth conditions, perhaps mimicking the cat flea vector, would result in differential expression patterns. These conditions would be predicted to result in diminished badA expression since badA is known to play a critical role in interacting with vertebrate cells, but its role for growth in the cat flea vector has not been established and increased adherence may be unnecessary. Alternatively, after extended growth under the conditions of the cat flea vector, the bacteria might be primed for infection of the vertebrate host by upregulating genes such as badA.The gene expression patterns generated by RNA‐seq, qRT‐PCR, and dot‐blots described in this report use B. henselae genome are intriguing. The Brt RNAs are found in all Bartonella species except B. bacilliformis and B. rochalimae; however, in most species, there are multiple copies of these RNAs suggesting that their function is important to the overall fitness and survival of these bacteria. It remains to be seen if the multiple copies of the Brt RNA function in a redundant or complementary manner. The multiple copies of the Brt family of RNAs also presents a problem in dissecting individual Brt and Trp function, since construction of deletion mutants in B. henselae is challenging. We are still attempting to make a ∆Brt1 strain to examine the phenotype and badA expression in the absence of this RNA. It is possible that increased compensatory expression of one or more of the other Brt family RNA will be observed in such a mutant. The presence of the eight trp genes following their cognate Brt RNA as well as 16 other trp‐like genes makes dissecting trp function and even more daunting task. Thus, deleting all Brts and their associated trps is impractical.The multiple copies of Brt RNA and Trp protein genes in the B. henselae has been described to autoagglutinate in a manner that is dependent upon expression of badA  consistently demonstrated an impaired ability to form biofilms in our assay and served as a negative control for the biofilm assay. Considering the well‐characterized role of BadA in autoagglutination, it is reasonable to expect that the surface location and adherent nature of BadA results in a role for this protein in biofilm formation. The requirement for BadA to form biofilms makes it tempting to speculate on the role of this protein in establishing the vegetative masses described in patients with infective endocarditis caused by B. henselae  is thought to be due to the variable expression of badA (Riess et al., B. henselae in the laboratory. It would be interesting to examine fresh isolates of B. henselae with the same strain that has been repeatedly passaged to examine not only badA expression but also brt1 and trp1 transcript levels. Recently, the presence of BadA in B. henselae strains was shown to adversely affect the expression of the VirB type IV secretion system genes and the function of the cognate effectors (Lu et al., badA (Lu et al., badA and the genes encoding the VirB type IV secretion system. Interestingly, some of the VirB type IV secretion system components including bepC, virB6, and virB9 were predicted targets of Brt1. Of note, the B. bacilliformis genome does not contain the Brt RNA genes and also lacks the VirB type IV secretion system. Thus, Brt RNAs are absent in one of the species of Bartonella that does not have a need to coordinately regulate these two systems.The long observed phenomenon of autoadherence and the sticky and adherent colony phenotypes reported in fresh clinical isolates of The authors declare no potential conflict of interest. Click here for additional data file.