Norad in vivo. Deletion of Norad in mice results in genomic instability and mitochondrial dysfunction, leading to a dramatic multi-system degenerative phenotype resembling premature aging. Loss of tissue homeostasis in Norad-deficient animals is attributable to augmented activity of PUMILIO proteins, which act as post-transcriptional repressors of target mRNAs to which they bind. Norad is the preferred RNA target of PUMILIO2 (PUM2) in mouse tissues and, upon loss of Norad, PUM2 hyperactively represses key genes required for mitosis and mitochondrial function. Accordingly, enforced Pum2 expression fully phenocopies Norad deletion, resulting in rapid-onset aging-associated phenotypes. These findings provide new insights and open new lines of investigation into the roles of noncoding RNAs and RNA binding proteins in normal physiology and aging.Although numerous long noncoding RNAs (lncRNAs) have been identified, our understanding of their roles in mammalian physiology remains limited. Here, we investigated the physiologic function of the conserved lncRNA  Only a tiny portion of our genetic material contains the information required to create proteins, the workhorses of the body. The rest of our DNA, however, is not useless: some of it can be transcribed to create molecules known as non-coding RNAs, which are increasingly scrutinized by scientists.NORAD acts as a guardian of the genome by reducing the activity of a protein named PUMILIO. Without NORAD, PUMILIO becomes overactive, and this causes problems as genetic information is split between two ‘daughter cells’ when a cell divides.For example, a non-coding RNA called NORAD in the body decrease, while the levels of PUMILIO increase. However, the precise role that NORAD may play in aging remains unclear.Defects in the amount of genetic material in cells have been linked with faster aging in animals. In addition, some studies suggest that as animals get older, the levels of Norad  and carefully monitored how they grew and developed. The animals looked normal at birth, but they seemed to age faster: for instance, their fur became thin and gray, and their brains developed age-related abnormalities much sooner than normal mice.To address this question, Kopp et al. engineered mutant mice that lack Norad was also associated with problems often seen in old age. The mutant animals were more likely to have incorrect amounts of genetic information in their cells, and they had defects in the cell compartments that create the energy necessary for life. Further experiments showed that these issues were driven by PUMILIO being hyperactive. Overall, the work by Kopp et al. reveal that the non-coding RNA Norad is essential to keep PUMILIO activity in check and to prevent problems associated with aging from appearing in young animals. Further studies are now needed to take a closer look at how NORAD and other non-coding RNAs keep us healthy.At the level of individual cells, losing  Althougitecture , the bioitecture , it has Noncoding RNA activated by DNA damage (NORAD) is a recently described lncRNA that is distinguished from the majority of transcripts in this class due to its high abundance in mammalian cells and strong evolutionary conservation across mammalian species , exhibiting 61% nucleotide identity with its human counterpart, is clearly identifiable on mouse chromosome 2 /Cas9-mediated genome editing was employed to delete the lncRNA-encoding sequence from the mouse genome, yielding three independent knockout lines  fibers to slow twitch oxidative (STO) fibers   fibers , a pheno) fibers . In addiem (CNS) , includiem (CNS) . Similarem (CNS) . Finallyinterval . These fNORAD is the preferred binding partner of PUM2 in human cells  (+/+Norad and –/–Norad mice (Norad shows the highest expression in this tissue (–/–Norad mice (To assess whether  (eCLIP)  to asses–/– mice . Brain ws tissue , patholo–/– mice , and mam–/– mice .NORAD, the mouse transcript is highly enriched for PREs, harboring 11 perfect matches to the canonical PRE consensus and an additional 3 PREs conforming to a slightly relaxed consensus sequence (UGUANAUN)  . Accordicriptome , exhibit+/+Norad and –/–Norad brain, the average reads in CLIP clusters in a given target 3’ UTR normalized to the expression level of the target was determined for each condition, an approach used previously to estimate relative binding in CLIP data , 4-hydroxynonenal (4-HNE), and 8-hydroxy-2'-deoxyguanosine/8-hydroxyguanosine (8-OHdG/8-OHG), markers of protein, lipid, and nucleic acid oxidation, respectively  . Indeed,ectively .Norad-deficient cells, respiration rates were analyzed in pairs of littermate-matched +/+Norad and –/–Norad MEFs. Basal and maximal respiration was significantly reduced in –/–Norad cells   . Genes a tissues . Remarkafication . DownregEF lines . These d–/–Norad mice, it remained to be demonstrated whether PUMILIO hyperactivity alone could account for the full spectrum of observed phenotypes. To address this question, transgenic mice with doxycycline (dox)-inducible expression of FLAG-tagged PUM2 were generated and crossed to mice harboring a ubiquitously expressed reverse tetracycline-controlled transactivator 3 transgene (CAG-rtTA3)     . Adminisry (IHC) . Robust ry (IHC) , althougry (IHC) . Importary (IHC) . Because the CNS , we focuPum2; rtTA3 double transgenic mice derived from two independent founders, but not to Pum2 or rtTA3 single transgenic controls, resulted in a striking phenotype within 2 months that closely resembled the appearance of –/–Norad mice at 1 year of age. Dox-treated Pum2; rtTA3 mice developed rapidly progressing kyphosis, alopecia, graying of fur, and loss of body fat  body fat . These pbody fat  and the body fat . Furtherl muscle  togetherl muscle .Norad-deficient animals results in genomic instability, mitochondrial dysfunction, and ultimately a multi-system degenerative phenotype resembling premature aging.Lastly, we directly assessed whether enforced PUM2 expression impairs mitochondrial function in MEFs and human cell lines. Transient expression of FLAG-PUM2 in MEFs or stable expression of either PUM1 or PUM2 in HCT116 significantly impaired respiration . OverallNorad ortholog reported here unequivocally establish the importance of this lncRNA, and the tight regulation of its target PUMILIO proteins, in mammalian biology and implicate the Norad-PUMILIO axis as a major regulator of aging-associated phenotypes , associated with a ~ 50% reduction in PUM1 protein, or a later onset variant known as PUM1-related cerebellar ataxia (PRCA), associated with only a ~ 25% lowering of PUM1 levels   together with two in vitro transcribed sgRNAs flanking the Norad locus into fertilized C57BL/6J oocytes as described  of The University of Texas Southwestern Medical Center (UTSW) and The Ohio State University, Nationwide Children’s Hospital. Mice were maintained in regular housing with a 12 hr light/dark cycle and normal chow and water escribed . Foundereletions . All werFLAG-Pum2 transgenic mice (in this study referred to as Pum2 mice) were generated in a C57BL/6J background by the UTSW Transgenic Core using standard procedures for pronuclear injection. For the generation of the transgene vector, a cDNA clone of isoform 3 of the mouse Pum2 coding sequence (BC041773) was purchased from transOMIC Technologies, verified by Sanger sequencing, PCR amplified with primers adding a FLAG tag to the protein N-terminus, and cloned into the pTRE-Tight vector (Clontech). Of note, isoform 3 encodes for the shorter PUM2 variant, which can be detected as the lower of two PUM2 bands in western blots. Broad endogenous expression of this isoform was detected by PCR in all tested mouse tissues (data not shown), confirming its physiologic relevance. Transgene positive Pum2 mice were crossed to a ubiquitous cytomegalovirus early enhancer element chicken beta-actin (CAG) promoter-driven reverse tetracycline-controlled transactivator 3 (rtTA3) mouse line, which was generated in the Lowe laboratory  (Sigma-Aldrich) supplemented with 10 g/L sucrose  in drinking water.Doxycycline (dox)-inducible boratory  and obtaMice were anesthetized with isoflurane  and subjected to retro-orbital bleeding. Approximately 200 μL of blood were collected, immediately heparinized with 500 USP units/mL (Fresenius Kabi), transferred to 1.3 mL of PB Max Karyotyping Medium supplemented with 50 μg/mL lipopolysaccharide (Gibco and Sigma-Aldrich), and incubated at 37°C for 48 hr with shaking. After this incubation, cells were harvested and processed for DNA FISH analysis. For the isolation of splenocytes, mice were euthanized with an overdose of isoflurane, and spleens were resected, minced with a razor blade in 1X Hank’s balanced salt solution (HBSS) without calcium and magnesium (Gibco), passed through a 70 μm cell strainer (Corning), and washed with 1X phosphate buffered saline (PBS) (Sigma-Aldrich). Immediately after isolation, splenocytes were processed for DNA FISH analysis.+/–Norad females were bred to +/–Norad males or Pum2 females were bred to rtTA3 males and euthanized at embryonic day E14.5. Under sterile conditions, the uterine horns were removed and washed once with 70% ethanol and three times with 1X PBS. The embryos were then released and the heads and all visceral organs removed. The remainder of the embryo was finely minced using razor blades and treated with 0.25% trypsin/EDTA (Gibco) at 37°C for a total of 20 min. MEF growth medium consisting of DMEM with 4.5 g/L glucose, L-glutamine and sodium pyruvate (Gibco) supplemented with 1X nonessential amino acids (NEAA), 1X Antibiotic-Antimycotic (AA) , and 10% fetal bovine serum (FBS)  was added to inactivate the trypsin. Tissue chunks were disrupted by vigorous pipetting, centrifuged, resuspended in MEF growth medium, and plated in T25 flasks. The next day, non-adherent tissue debris was used for genotyping, while attached cells were transferred to a fresh tissue culture dish and designated as primary MEF passage 1 (P1). Aliquots of primary MEF P1 were frozen and stored until needed. Primary MEFs were used for a maximum of passages. Immortalized MEF lines were generated by transfecting primary MEFs with pSG5-SV40-Large-T-Antigen using Lipofectamine 3000 (Invitrogen) according to the manufacturer’s instructions. Starting at 48 hr post transfection, cells were serially passaged 1:10 to select for SV40-immortalized MEFs. After 6 passages, all SV40 large T antigen-transfected MEF lines were regarded as immortalized and this passage was designated as immortalized MEF P1. Of note, the genders of the MEF lines used in this study are not known. All MEF lines were tested and confirmed to be mycoplasma free.–/–NORAD clones via transcription activator-like effector nuclease (TALEN)-mediated insertion of a Lox-Stop-Lox cassette as well as HCT116 PUM1 and PUM2 overexpression clones using lentiviral transduction was described previously  were obtained from ATCC and cultured in DMEM with 4.5 g/L glucose, L-glutamine and sodium pyruvate (Gibco) supplemented with 1X AA (Gibco), and 10% FBS (Sigma-Aldrich). The male colon cancer cell line HCT116 was obtained from ATCC (CCL-247) and cultured in McCoy’s 5a medium (Gibco) supplemented with 1X AA (Gibco) and 10% FBS . The cell line was authenticated by ATCC using short tandem repeat (STR) analysis in November 2017. The generation of HCT116 eviously . Cell liNorad expression in mouse was quantified using the Applied Biosystems TaqMan assay for 2900097C17Rik (Norad) (Mm04242407_s1) and the TaqMan Universal II Master Mix (Applied Biosystems). Human NORAD was quantified using a custom TaqMan assay described previously (Gapdh mRNA (in vivo studies) using either standard curves of each gene or the comparative ΔCt method. The number of biological replicates is stated in the figure legends, each biological replicate was run with three technical replicates.Total RNA was isolated from cells or tissues using the miRNeasy Mini Kit (Qiagen) following the manufacturer’s instructions including an on-column DNAse I digest to remove genomic DNA contamination. Complementary DNA (cDNA) was generated from 1 μg of total RNA using the SuperScript III First-Strand Synthesis SuperMix for qRT-PCR (Invitrogen) according to the manufacturer’s protocol. Relative eviously . For all+/+Norad and –/–Norad mice (three mice per genotype) as well as from spleens of 16-week-old Pum2; rtTA3 and control (Pum2 and wild-type) mice after 4 weeks of dox treatment (four mice per group) using the miRNeasy Mini Kit (Qiagen) including a DNAse I digestion step to remove genomic DNA. RNA integrity was determined with the Agilent 2100 Bioanalyzer, and only RNA samples with an RNA integrity number (RIN) of greater than eight were used for subsequent analysis. Sequencing libraries were prepared with the TruSeq Stranded mRNA Library Prep Kit (Illumina) and sequenced using the 75 base pair (bp) single-read protocol on a NextSeq 500 platform (Illumina). Library prep and RNA-seq were performed by the UTSW McDermott Center Next-Generation Sequencing Core.Total RNA was isolated from brains and spleens of 10-week-old male +/+Norad and –/–Norad mice, quality assessment of the sequencing data was performed with NGS QC Toolkit (v2.3.3)  using HISAT2 (v2.1.0) (For RNA-seq in (v2.3.3) . Reads w(v2.3.3) . Only re(v2.3.3) . Differe(v2.3.3) . For RNA(v2.1.0) . Only re(v2.1.0) , respect–/–Norad spleens using the Database for Annotation, Visualization and Integrated Discovery (DAVID)   was perf (DAVID) .+/+Norad and –/–Norad females  were resected and cut in halves. Per sample, one half of a brain was minced with razor blades in 1X ice-cold diethyl pyrocarbonate (DEPC)-treated PBS. The resulting tissue suspensions were UV crosslinked on ice in a Spectrolinker XL-1500 (Spectronics) at 254 nm three times at 400mJ/cm2. The UV crosslinked tissues were centrifuged, snap-frozen in ethanol/dry ice, and stored at −80°C until needed. For the immunoprecipitation of PUM2, Protein G Dynabeads (Invitrogen) were used together with the same polyclonal goat anti-PUM2 antibody  used previously for PUM2 CLIP analysis in human cells  before all remaining reads, which did not align to Norad, were mapped to GRCm38 (mm10) using Tophat2 (v2.0.12) with default settings  for PCR duplication identification were trimmed and recorded using in-house scripts. Because of the high number of us study . Reads wsettings . Only un+/+Norad CLIP replicates. To calculate normalized CLIP signal of PUM2 targets in +/+Norad and –/–Norad brains /total_usable_reads[IP])/((reads_in_bin[input]+1)/total_usable_reads[input]). Bins with fold-changes greater than or equal to four were considered as clusters. Finally, we filtered clusters for those that were detected in both biological replicates of either genotype. Clusters that overlapped with at least 30% of their length were merged. Bedgraph files of each sample were generated with BEDtools  using re– brains , a methowas used . Genes wwas used , non-CLIwas used , brain CTotal DNA was isolated using the DNeasy Blood and Tissue Kit (Qiagen) according to the manufacturer’s instructions. Mitochondrial (mtDNA) and nuclear (nDNA) DNA were quantified by qPCR using either human or mouse specific primers  and the 2, 0.5% NP-40, RNAse inhibitor), incubated on ice for 5 min, and centrifuged. The supernatant contained the cytoplasmic fraction, while the pellet contained the nuclear fraction. Both fractions were then subjected to RNA isolation and equal cell equivalents of nuclear and cytoplasmic RNA were used in subsequent qRT-PCR reactions. All samples were tested for Norad as well as for Neat1 (nuclear control) and Actb (cytoplasmic control). Because equal cell equivalents of nuclear and cytoplasmic RNA were used in each reaction, the sum of the expression level of each transcript in the nucleus plus cytoplasm can be set to 100%, thereby allowing determination of the percentage of each transcript localized to each compartment. Neat1 and Actb, respectively, showed the expected nuclear and cytoplasmic localization in each experiment, confirming successful subcellular fractionation.Mouse cell lines  were seeded in triplicate and harvested the next day for subcellular fractionation, which was performed as previously described . BrieflyNorad was synthesized by in vitro transcription using a DIG-labeling mix (Roche). Primers used for amplification of the DNA template are provided in RNA FISH was performed as described previously . A DIG-lCell and tissue lysates were prepared in RIPA buffer  supplemented with cOmplete Protease Inhibitor Cocktail (Roche). Western blots were probed with monoclonal rabbit anti-PUM2 antibody , monoclonal rabbit anti-PUM1 antibody , polyclonal rabbit anti-FLAG , or monoclonal rabbit anti-GAPDH antibody . Bands were visualized using an IRDye 800CW donkey anti-rabbit IgG secondary antibody  and an Odyssey CLx Imager (Licor).FLAG-Pum2 transgene, immunohistochemistry (IHC) was performed by the UTSW Tissue Management Shared Resource using the monoclonal anti-FLAG M2 antibody . Images were acquired on an AxioObserver Z1 microscope (Zeiss).Tissues were harvested from mice that had been treated with dox for 3.5–6.5 weeks. All tissues were fixed in 10% neutral buffered formalin (NBF) for 24–48 hr. Fixed samples were processed, paraffin embedded, and sectioned using standard procedures. To detect the expression of the +/+Norad and –/–Norad mice were used for semi-thin and ultrastructural analysis. For these studies, one group of mice were given xylazine/ketamine anesthesia and euthanized by cardiac perfusion with 4% paraformaldehyde followed by 5% glutaraldehyde (both in 0.1 M phosphate buffer). Tissue samples from brain and spinal cord were removed under a dissecting microscope. A second group of mice were perfused with 4% paraformaldehyde and their muscles were removed and further fixed in situ in 5% glutaraldehyde. These tissues were dissected into small blocks and processed for plastic embedding using standard methods  and –/–Norad (n = 5) mice were analyzed. Motor neuron pools in the anterior horn cell areas of both hemicords from each section were included. For each mouse, sections from 1-2 levels were photographed at 20X magnification. Only cell bodies clearly showing a nucleolus on the plane of the section were considered. Equal numbers of spinal cord levels were analyzed in each group. Mean neuronal densities of anterior horn areas per lumbar cord level were calculated.To analyze neuronal cell loss in the ventral horn neuron pools, 5 µm thick, paraffin embedded and hematoxylin and eosin (H and E) stained lumbar spinal cord sections from +/+Norad and –/–Norad mice  were stained for SDH activity. Three images, each representing a distinct zone of the gastrocnemius muscle , were taken along the midline axis at 20X magnification using an Olympus BX41 microscope. Muscle fiber types were determined and counted based on their dark (STO), intermediate (FTO), or light (FTG) SDH staining as previously reported (Succinate dehydrogenase (SDH) enzyme histochemistry was used to assess metabolic fiber type changes in the aging muscle. For this purpose, muscle fiber types were grouped into three categories: slow twitch oxidative (STO), fast twitch oxidative (FTO), and fast twitch glycolytic (FTG). Twelve-µm-thick cross-sections from the gastrocnemius muscles of 12-month-old reported .+/+Norad and –/–Norad mice using cytochrome c oxidase (COX) histochemistry. Twelve µm thick frozen lumbar spinal cord tissue sections were mounted onto superfrost glass slides (Thermo Fisher) and dried at room temperature for 1 hr before the COX enzymatic activity assay was performed. Two sections, 100 µm apart, were analyzed from each mouse. The anterior horn areas of each section were photographed at 10X magnification and the number of neurons with normal COX activity were determined. Counts from the hemicord anterior horn cell area with the higher number of neurons with normal COX activity was included in the analysis. Only cell bodies with an obviously visible nucleolus on the plane of the section and with a COX activity clearly above the background of gray matter were considered. The number of neurons with normal COX activity was counted per each section analyzed. COX activity was also qualitatively analyzed in fresh frozen gastrocnemius muscle sections from 12-month-old +/+Norad and –/–Norad mice and 4-6 month-old Pum2; rtTA3 transgenic mice or controls that had been treated with dox for 1.5–2 months.Mitochondrial function was assessed in spinal cord neurons of 12-month-old DNA FISH for two representative chromosomes (chromosomes 2 and 16) was performed in lymphocytes, splenocytes, and primary MEFs. Cells were first incubated in hypotonic KCl solution: lymphocytes were incubated in 75 mM KCl at 37°C for 15 min, splenocytes in 75 mM KCl at room temperature for 30 min, and MEFs in 0.4% KCl at room temperature for 8 min. Subsequently, cells were centrifuged and resuspended in methanol/acetic acid (3:1), washed twice with methanol/acetic acid (3:1), dropped onto Rite-On Micro Slides , air-dried, and either used immediately or stored at −20°C until needed. DNA FISH was performed using the Mouse IDetect Chromosome Point Probes for chromosome 2 (red) and 16 (green)  following the manufacturer’s protocol. Slides were mounted with ProLong Diamond Antifade Mountant with DAPI (Invitrogen) and analyzed on an AxioObserver Z1 microscope (Zeiss) using the 100X oil objective. Lymphocytes and splenocytes whose chromosome count differed from 2n for at least one of the two tested chromosomes were regarded as aneuploid or off mode. MEFs were only considered aneuploid when their chromosome count differed from 2n or a multiple of 2n in order to account for the increased polyploidy in this cell type.+/+Norad and –/–Norad MEFs  were grown on Lab-Tek Chambered Coverglass slides (Thermo Fisher) that were coated with poly-L-lysine (Sigma-Aldrich). Prior to the analysis, DNA was visualized by adding 50 ng/mL Hoechst dye (Invitrogen) to the growth medium. Mitoses were monitored by taking fluorescence images every 5 min for ~48 hr on a Leica inverted microscope equipped with a temperature and CO2-controlled chamber, a 63X oil objective, an Evolve 512 Delta EMCCD camera, and the MetaMorph Microscopy Automation and Image Analysis Software . Videos were generated from the acquired time-lapse images and analyzed for the occurrence of mitotic defects including anaphase bridges and lagging chromosomes.Primary +/+Norad, +/–Norad, and –/–Norad mice were continuously monitored over a period of 12 months for the onset and progression of kyphosis as well as alopecia and graying of fur. The kyphosis scoring system was adopted from a previously reported strategy  adipose thickness of 12-month-old +/+Norad and –/–Norad mice was determined using standard H and E-stained skin histology. For every skin sample, images were acquired at 5X magnification across the entire length of the section on an AxioObserver Z1 microscope (Zeiss) using the AxioVision 4.8 software (Zeiss). In these images, the thickness of the s.c. adipose tissue was measured at 15 different points using the AxioVision 4.8 software. The average of the 15 measurements was then calculated to obtain the adipose thickness of each mouse.Whole-body fat, as a percentage of body weight, was measured by nuclear magnetic resonance (NMR) in 3-month and 12-month-old +/+Norad and –/–Norad MEFs. 80 × 103 cells were seeded in triplicate in 6-well plates in regular growth medium and incubated for 16–18 hr until 70–80% confluent. Cells were then trypsinized with 0.25% trypsin/EDTA (Gibco), pelleted at 300 g, and resuspended in 500 µL of fresh growth medium containing 50 nM TMRE. Samples were then incubated at 37°C in the dark for 30 min and analyzed by flow cytometry using a BD Accuri C6 Cytometer (BD Biosciences). The average and standard deviation of the mean fluorescence intensities of the three replicates was calculated for each sample, and each –/–Norad MEF line was compared to its +/+Norad littermate control line.Tetramethylrhodamine ethyl ester (TMRE)  was used for measuring mitochondrial membrane potential (MMP) in immortalized 3 MEFs were plated in triplicate in -well plates in regular growth medium and incubated for 16–18 hr. 60 × 103 HCT116 cells were seeded in triplicate in 24-well plates in regular growth medium and also incubated for 16–18 hr. All cells were 70–80% confluent at the time of the assay. ROS levels were measured according to the manufacturer’s protocol. Briefly, cells were trypsinized with 0.25% trypsin/EDTA (Gibco), pelleted at 300 g, washed once with Enzo 1X Wash Buffer, and resuspended in 300 µL of freshly prepared Enzo ROS Detection Solution (1 µL ROS dye in 5 mL Wash Buffer). Samples were incubated in the dark for 30 min and analyzed by flow cytometry using a BD Accuri C6 Cytometer (BD Biosciences). The average and standard deviation of the mean fluorescence intensities of the three replicates was calculated for each sample. Each –/–Norad MEF line was compared to its +/+Norad littermate control line.The Enzo Total ROS Detection Kit  was used for detection of ROS levels in immortalized MEFs and human HCT116 cells. 80 × 10+/+Norad and –/–Norad mice using IHC and immunofluorescence. Lipid peroxidation was examined in formalin-fixed paraffin embedded spinal cord sections by IHC using the polyclonal rabbit anti-4-Hydroxynonenal (4-HNE) antibody . Nucleic acid oxidation was assessed in fresh frozen brain sections by immunofluorescence using the mouse monoclonal anti-DNA/RNA Damage antibody , which detects 8-OHdG/8-OHG. In addition, protein oxidation was assessed in spinal cord and brain using an enzyme-linked immunosorbent assay (ELISA) for 3-nitrotyrosine (3-NT)  according to the manufacturer’s instructions.Oxidative damage was examined in the CNS of 12-month-old 3 cells per well were plated in Seahorse XF96 cell culture microplates (Agilent) in regular growth medium and incubated for 14–16 hr. For HCT116 cells, Seahorse XF96 Cell Culture Microplates were coated with poly-L-lysine (Sigma-Aldrich) to improve cell attachment. Prior to measurement, cells were equilibrated for 1 hr in Seahorse assay medium  supplemented with 10 mM glucose (Sigma-Aldrich), 1 mM sodium pyruvate (Gibco), and 2 mM L-glutamine (Sigma-Aldrich). OCRs were monitored before and after adding the following mitochondrial inhibitors: 2 µM oligomycin (complex V inhibitor), 1 µM carbonyl cyanide 3-chlorophenylhydrazone , and 1 µM antimycin A (complex III inhibitor) . OCRs were normalized to the amount of protein in each sample using a bicinchoninic acid (BCA) assay (Thermo Fisher) according to the manufacturer’s instructions. For HCT116 cells, OCRs were further normalized to the mtDNA content to account for differences in mitochondrial content.Oxygen consumption rates (OCRs) were analyzed in immortalized MEFs and human HCT116 cells using a Seahorse Bioscience XF96 Extracellular Flux Analyzer (Seahorse Bioscience/Agilent). 10 × 10FLAG-Pum2 cDNA that was used for generating the transgenic mouse was cloned into a pBROAD3 vector (Invivogen), in which the Rosa26 promoter was replaced by a strong CAG promoter (pCAG-FLAG-Pum2). A pcDNA3-EGFP vector was used for control transfections. FLAG-PUM2 and EGFP were transfected into immortalized +/+Norad MEFs using 2.5 µg of plasmid DNA and Lipofectamine 3000 (Invitrogen) according to the manufacturer’s protocol. In brief, 100 × 103 cells were seeded into -well plates and incubated overnight. The next day, cells were transfected with either pCAG-FLAG-Pum2 or pcDNA3-EGFP and incubated again overnight. The following day, cells were collected and re-plated for a second transfection, performed identically. After overnight incubation, cells were seeded for Seahorse analysis, as described above. PUM2 overexpression was assessed after the second transfection using western blot.To overexpress PUM2 in MEFs, the same A comprehensive description of the RNA-seq and eCLIP analysis including the use of software is provided in the respective sections. The significance of the cumulative distribution functions was calculated using the Kolmogorov-Smirnov test and plotted in R. For all other analyses, statistical significance was analyzed using Prism 7 (GraphPad Software). Student’s t tests or log-rank tests  were used to determine statistical significance, which is indicated as *p≤0.05, **p≤0.01, ***p≤0.001, ****p≤0.0001. Data are presented as mean ± SD in all figures except RNA-seq and eCLIP data has been deposited in the Gene Expression Omnibus (GEO) at NCBI . Data is available for download via the following links:https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121684https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121688https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE125539 In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.Norad-deficient mice" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Chris P Ponting as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by James Manley as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Igor Ulitsky (Reviewer #2). A further reviewer remains anonymous.Thank you for submitting your article "PUMILIO hyperactivity drives premature aging of The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.In this manuscript, Mendell and colleagues describe the generation and characterization of Norad-/- mice. NORAD was previously described as an abundant and conserved lncRNA, whose disruption in human cells results in genome instability through either increase in activity of Pumilio proteins in the cytoplasm or disruption of protein complex formation involving RBMX in the nucleus. In this new work, Norad-/- mice were generated, and shown to have a severe aging phenotype, which is accompanied by genome instability and mitochondrial defects, along with some changes in gene expression, which can be attributed in part to hyperactivity of the Pumilio proteins. Convincingly, mice with mild inducible transgenic over-expression of Pum2 are shown to recapitulate the Norad-/- phenotype, strongly arguing that Pumilio hyperreactivity is indeed responsible for the Norad loss-of-function phenotype.Essential Revisions:1) CLIP experiments.A) Arguably the weakest part of the results is the changes observed in gene expression in the mouse brain, particularly Figure 3E, which shows deviations in both up-regulated and down-regulated genes, contrary to the expectation for mostly up-regulation of Pumilio targets. This is possibly because the authors select Pum targets based on CLIP data, which has the advantage of yielding "experimentally defined" targets, but biases towards more abundantly expressed genes (as those are likely to have more reads/clusters in CLIP data). What does Figure 3E look like if the authors define the targets based on presence of PREs in 3'UTRs ? An alternative would be to sample non-targets that have the same expression level distribution (and 3' UTR lengths) as the CLIP-defined targets and repeat the analysis. In any case, it is worth discussing how the changes observed in gene expression, at least in bulk tissue are very small (~5%) and occur in both directions. This is consistent with the studies in human cells, but still worth dwelling on in the Discussion section.B) Another molecular phenotype the authors observe is increase in Pumilio target clusters based on eCLIP data. This analysis has two caveats – first, differences in quality of eCLIP libraries can result in differences in numbers of clusters etc. Can the authors show that the replicates of eCLIP from the same genotype (if performed) are similar to each other more than Norad+/+ samples to Norad-/- samples? Alternatively – is the PRE motif recovered equally strongly from Norad+/+ and Norad-/- libraries?C) The other caveat with the eCLIP analysis is that the authors normalize the total number of eCLIP reads, and then compare the eCLIP/FPKM ratios between genes. This makes it difficult to interpret Figure 3D. To me, it shows that the reads become redistributed to clusters in the more abundant genes, rather than that target occupancy globally is indeed increased . First, why don't both curves reach 1 at the y-axis? Also, since the total number of reads is normalized, is this evidence sufficient to say that "target occupancy was significantly increased"? The authors should explain more thoroughly why this analysis is valid (I could be missing something), and how it supports the conclusion in the main text.D) Related to (A) above. Is a PRE-dependent signature of Pum target repression evident in the MEF data ? If it is – it's worth showing in supplement. If not – worth discussing.2) NORAD/RBMX.Analysis of two other protein binding partners have been published for Norad/NORAD: SAM68 and RBMX. In particular, the interaction of NORAD with RBMX and its influence on the topoisomerase complex  is very convincing and accounts for the genome instability resulting from NORAD loss. Moreover, the topoisomerase complex is also implicated in mtDNA maintenance, similar to what the authors show in their work here. The authors briefly mention the NORAD/RBMX study in the Introduction, but curiously state that the role of this interaction for genome stability is unknown. The role of topoisomerases in genome stability is thus known and the authors need to discuss the NORAD/RBMX study in light of their own findings. There are, however, some discrepancies between these studies. Lee et al., and Tichon et al., found NORAD to be predominantly localized in the cytoplasm, while the NORAD/RBMX study find that NORAD is evenly distributed (smFISH and fractionation) between the nucleus and the cytoplasm. RNA-FISH data, if available, would resolve this issue.3) PUM2 level in FLAG-tag experiment.Norad loss of function”) The authors need to define how PUM2 is deregulated, if it is not overtly overexpressed. Otherwise, the models cannot be defined as such. One alternative hypothesis is that deregulated PUM2 RNA, but not protein, is required for the phenotype to be manifested. Do the authors have evidence that this is, or is not, the case? If not then they need to discuss alternative explanations, remove the word "accordingly" (Abstract), and insert further caveats to their conclusion that (subsection “Enforced PUM2 expression phenocopies Norad loss of function” and elsewhere) "PUMILIO hyperactivity in Norad-deficient animals results in.…" etc. The rare cell specificity of effect (Discussion section) is an attractive one but is not supported by evidence in the submission. Subsection “Enforced PUM2 expression phenocopies Norad loss of function”: The lack of alteration of PUM2 protein levels in these 7 tissues (even with the negative feedback mechanism) cannot be reconciled with the PUMILIO "hyperactivity" of the title.There is a surprising lack of PUM2 protein level change for the FLAG-tag PUM2 mouse experiment. This means that the authors' model is not definitive and they need to be more circumspect in their conclusions.  CLIP experiments.A) Arguably the weakest part of the results is the changes observed in gene expression in the mouse brain, particularly Figure 3E, which shows deviations in both up-regulated and down-regulated genes, contrary to the expectation for mostly up-regulation of Pumilio targets. This is possibly because the authors select Pum targets based on CLIP data, which has the advantage of yielding "experimentally defined" targets, but biases towards more abundantly expressed genes (as those are likely to have more reads/clusters in CLIP data). What does Figure 3E look like if the authors define the targets based on presence of PREs in 3'UTRs ? An alternative would be to sample non-targets that have the same expression level distribution (and 3' UTR lengths) as the CLIP-defined targets and repeat the analysis. In any case, it is worth discussing how the changes observed in gene expression, at least in bulk tissue are very small (~5%) and occur in both directions. This is consistent with the studies in human cells, but still worth dwelling on in the Discussion section.+/+Norad and –/–Norad brains simply compared all CLIP targets to all non-targets. To incorporate the reviewers’ suggestions, we first filtered the CLIP targets for those with at least 2 PREs in the 3’ UTR. However, this only marginally increased the apparent repression of PUM2 targets in –/–Norad brain (not shown). On the other hand, the reviewers’ suggestion to sample non-targets with the same expression level distribution and 3’ UTR length dramatically improved the results. As shown in the new Figure 3D-E, when non-targets are filtered for those whose expression level and 3’ UTR length are similar to these parameters in the set of PUM2 CLIP targets, the results much more clearly show repression of PUM2 targets in –/–Norad brain. In addition, we have expanded the discussion in the manuscript to introduce the concept that modest repression of a large set of PUMILIO targets can, in aggregate, produce a dramatic phenotype.We appreciate these excellent suggestions, which have greatly improved the gene expression analyses in the paper. As noted by the reviewers, our original analysis of expression of PUM2 CLIP targets vs. non-targets in B) Another molecular phenotype the authors observe is increase in Pumilio target clusters based on eCLIP data. This analysis has two caveats – first, differences in quality of eCLIP libraries can result in differences in numbers of clusters etc. Can the authors show that the replicates of eCLIP from the same genotype (if performed) are similar to each other more than Norad+/+ samples to Norad-/- samples? Alternatively – is the PRE motif recovered equally strongly from Norad+/+ and Norad-/- libraries?+/+Norad and –/–Norad samples could be easily confounded by experimental variation. Therefore, we did not intend to make the point that there were significantly more clusters in –/–Norad brain. We did not put forth this conclusion in the original text, although the original Figure 3C which showed the number of CLIP clusters in each genotype may have inadvertently implied that we were interpreting any apparent differences as meaningful. To avoid any confusion, we have removed this figure panel from the revised manuscript. The numbers of CLIP clusters in each genotype can be easily determined from Supplementary file 1 should a reader be interested in this information.We agree that comparing the absolute numbers of PUMILIO CLIP clusters between C) The other caveat with the eCLIP analysis is that the authors normalize the total number of eCLIP reads, and then compare the eCLIP/FPKM ratios between genes. This makes it difficult to interpret Figure 3D. To me, it shows that the reads become redistributed to clusters in the more abundant genes, rather than that target occupancy globally is indeed increased . First, why don't both curves reach 1 at the y-axis? Also, since the total number of reads is normalized, is this evidence sufficient to say that "target occupancy was significantly increased"? The authors should explain more thoroughly why this analysis is valid (I could be missing something), and how it supports the conclusion in the main text.–/–Norad condition) will likely result in reduced steady-state mRNA abundance, it is critical to normalize the apparent CLIP signal to the expression of the transcript. Therefore, we divided the normalized CLIP reads per target by FPKM (note that we are only considering targets expressed at or above 1 FPKM). These data were then used to generate the CDF plot in Figure 3C . Importantly, this method of analysis has been used previously to estimate relative binding in CLIP data  and we believe that it can provide a reasonable estimate of this parameter. To be as careful as possible, we have expanded our discussion of this result in the text to better describe how the analysis was done and to emphasize that it provided an estimate of relative CLIP signal that suggested increased PUM2 target occupancy in –/–Norad brain. We have also added the Bosson et al., reference to the main text and Materials and methods section.We apologize for any confusion regarding this analysis. To generate this figure, we first normalized the number of CLIP reads in each library based on the total number of usable reads . We next calculated the average number of normalized CLIP reads in each 3’ UTR cluster between the two replicates that were performed for each genotype. For each PUM2 CLIP target, we determined the total number of clusters in the 3’ UTR and then summed the average number of normalized CLIP reads in each cluster to determine the total number of normalized CLIP reads per CLIP target. Because increased PUM2 binding  Related to (A) above. Is a PRE-dependent signature of Pum target repression evident in the MEF data ? If it is – it's worth showing in supplement. If not – worth discussing.+/+Norad and –/–Norad spleens . Regardless, whether there is evidence of enhanced PUM2 target repression in these data is a valid question which we have addressed in the revised manuscript. Starting with the set of PUM2 brain CLIP targets identified in this study, we first determined which of these were expressed in spleen (FPKM ≥ 1). Because we were attempting to translate the targets identified in brain to a different tissue, we further filtered the CLIP targets for those that have at least two PREs (building off of a suggestion from the reviewers above). The expression of this set of PUM2 targets was compared to the set of non-targets with similar expression level and 3’ UTR length. The results, presented in Figure 4E of the revised manuscript, demonstrate statistically-significant repression of PUM2 targets in –/–Norad spleen.To clarify, the gene expression data in Figure 4 is derived from RNA-seq of 2) NORAD/RBMX.Analysis of two other protein binding partners have been published for Norad/NORAD: SAM68 and RBMX. In particular, the interaction of NORAD with RBMX and its influence on the topoisomerase complex  is very convincing and accounts for the genome instability resulting from NORAD loss. Moreover, the topoisomerase complex is also implicated in mtDNA maintenance, similar to what the authors show in their work here. The authors briefly mention the NORAD/RBMX study in the Introduction, but curiously state that the role of this interaction for genome stability is unknown. The role of topoisomerases in genome stability is thus known and the authors need to discuss the NORAD/RBMX study in light of their own findings. There are, however, some discrepancies between these studies. Lee et al., and Tichon et al., found NORAD to be predominantly localized in the cytoplasm, while the NORAD/RBMX study find that NORAD is evenly distributed (smFISH and fractionation) between the nucleus and the cytoplasm. RNA-FISH data, if available, would resolve this issue.NORAD:RBMX interaction is very interesting and we did not mean to diminish those findings. It is important to point out that the regulation of RBMX and PUMILIO by NORAD are not mutually exclusive and it is possible that this lncRNA performs both functions. We have therefore revised this section of the Introduction by describing the role of SAM68 in PUMILIO antagonism by NORAD and by emphasizing that the intriguing RBMX interaction suggests other functions for NORAD beyond regulating PUMILIO activity.We agree that the NORAD:RBMX interaction is essential to maintain genomic stability. The basis for this statement is the fact that no experimental evidence exists showing that loss of the RBMX binding site in NORAD impairs NORAD’s ability to maintain genome stability. Alternatively, one could show a critical role for RBMX in the NORAD loss-of-function phenotype by performing genetic epistasis experiments, in which modulation of RBMX activity is shown to rescue the NORAD deletion phenotype. These types of experiments have not yet been performed. It is important to contrast this with the NORAD:PUMILIO interaction, for which it has been shown that PUMILIO depletion reverses the NORAD loss-of-function phenotype , providing strong genetic evidence for PUMILIO’s role in the NORAD pathway. Our purpose in making this statement is to provide as accurate a description of the state of the field as possible. Nevertheless, given that our findings reported here do not in any way address the role of RBMX in the NORAD pathway, we do not think it is appropriate to further address that work in this manuscript.Despite these revisions, we still feel that it is appropriate and important to state that it has not yet been demonstrated that the Norad in mouse cells. RNA FISH in MEFs clearly shows a predominantly cytoplasmic localization , in agreement with the localization of the human transcript reported in Lee et al. and Tichon et al. We also performed additional fractionation experiments in a total of four mouse cell lines , which yielded results that perfectly agree with our RNA FISH data and show ~80% cytoplasmic localization of Norad  in all cell lines tested.In addition, as suggested by the reviewers, we have performed further experiments to investigate the localization of 3) PUM2 level in FLAG-tag experiment.There is a surprising lack of PUM2 protein level change for the FLAG-tag PUM2 mouse experiment. This means that the authors' model is not definitive and they need to be more circumspect in their conclusions. (subsection “Enforced PUM2 expression phenocopies Norad loss of function”) The authors need to define how PUM2 is deregulated, if it is not overtly overexpressed. Otherwise, the models cannot be defined as such. One alternative hypothesis is that deregulated PUM2 RNA, but not protein, is required for the phenotype to be manifested. Do the authors have evidence that this is, or is not, the case? If not then they need to discuss alternative explanations, remove the word "accordingly" (Abstract), and insert further caveats to their conclusion that (subsection “Enforced PUM2 expression phenocopies Norad loss of function” and elsewhere) "PUMILIO hyperactivity in Norad-deficient animals results in.…" etc. The rare cell specificity of effect (Discussion section) is an attractive one but is not supported by evidence in the submission. Subsection “Enforced PUM2 expression phenocopies Norad loss of function”: The lack of alteration of PUM2 protein levels in these 7 tissues (even with the negative feedback mechanism) cannot be reconciled with the PUMILIO "hyperactivity" of the title.Pum2; rtTA3 MEFs from both transgenic lines. Thus, transgene activation leads to PUM2 overexpression in specific cell types. To demonstrate that transgene activation leads to PUM2 hyperactivity in vivo, we went a step further by performing RNA-seq in spleens of Pum2; rtTA3 animals. This revealed robust repression of PUM2 targets in mice with transgene activation . These results provide clear evidence of PUM2 hyperactivity in dox-treated Pum2; rtTA3 mice.We have now performed a number of new experiments that have further documented the expression and activity of FLAG-PUM2 in our transgenic mouse lines and provide additional evidence that these animals indeed represent a model of PUMILIO hyperactivity. First, as noted by the reviewers, we posited that the inability to detect an increase in PUM2 levels in bulk tissue in transgenic mice may be due to the masking of transgene induction in specific key cell types due to the complex mixture of cell types in a tissue. To address this model, we isolated MEF lines from transgenic animals to interrogate whether transgene expression is detectable in a more uniform cell population. These results, provided in the new Figure 7—figure supplement 1B, clearly demonstrate robust induction of FLAG-PUM2 protein and a corresponding increase in total PUM2 levels in In addition to the new data demonstrating PUM2 hyperactivity in cells and tissues from transgenic mice, we would also like to emphasize that these in vivo studies are complemented by our data derived from overexpressing PUM1 and PUM2 in human and mouse cell lines . These studies show that overexpression of PUM1 or PUM2 protein impairs mitochondrial function, providing further evidence to support the conclusion that PUMILIO hyperactivity drives the observed phenotypes. These new data are also consistent with our previously reported finding that overexpression of PUM2 protein is sufficient to drive genomic instability in human cell lines . On the basis of this body of evidence, we feel that it is justifiable to refer to “PUMILIO hyperactivity” in the title.


The unprecedented amount of resources dedicated to humanitarian aid has led many stakeholders to demand the use of reliable evidence in humanitarian aid decisions to ensure that desired impacts are achieved at acceptable costs. However, little is known about the factors that influence the use of research evidence in the policy development in humanitarian crises. We examined how research evidence was used to inform two humanitarian policies made in response to the Syrian refugee crisis.We identified two policies as rich potential case studies to examine the use of evidence in humanitarian aid policy decision-making: Lebanon’s 2016 Health Response Strategy and Ontario’s 2016 Phase 2: Health System Action Plan, Syrian Refugees. To study each, we used an embedded qualitative case study methodology and recruited senior decision-makers, policy advisors, and healthcare providers who were involved with the development of each policy. We reviewed publicly available documents and media articles that spoke to the factors that influence the process. We used the analytic technique of explanation building to understand the factors that influence the use of research evidence in the policy-development process in crisis zones.We interviewed eight informants working in government and six in international agencies in Lebanon, and two informants working in healthcare provider organizations and two in non-governmental organizations in Ontario, for a total of 18 key informants. Based on our interviews and documentary analysis, we identified that there was limited use of research evidence and that four broad categories of factors helped to explain the policy-development process for Syrian refugees – development of health policies without significant chance for derailment from other government bodies (Lebanon) or opposition parties (Ontario) , government’s engagement with key societal actors to inform the policy-development process, the values underpinning the process, and external factors significantly influencing the policy-development process.This study suggests that use of research evidence in the policy-development process for Syrian refugees was subordinate to key political factors, resulting in limited influence of research evidence in the development of both the Lebanese and Ontarian policy. Globally, there has been an unprecedented amount of resources dedicated to humanitarian aid . During The Syrian conflict started in the spring of 2011 as a result of a civil war and has caused an estimated 6.6 million people to be displaced within Syria and over 5.6 million refugees seeking safety in Turkey, Lebanon, Jordan, Canada, and beyond. This large exodus of people has placed a strain on host countries’ health systems . The manWhat makes decision-making in humanitarian settings unique is the high levels of stress, often in intense and sometimes dangerous situations. Research evidence can help decision-makers respond in a timely manner in such situations. However, the use of research evidence to help respond to crises is not always straightforward. A culture built on immediate action with a traditionally heavy reliance on professional judgement may not be conducive to using evidence to inform decision-making in crisis zones . For exaPolicymaking is a highly complex process that requires multiple inputs  and can be dependent on the social, political, and historical context in which it occurs –25. An aThis study focuses on examining the factors that influence the use of research evidence in the governmental health policy-development processes in Lebanon and Ontario for two main reasons. First, the role of research evidence in policymaking is often limited . Even whWe used a qualitative embedded case study design. The context of this case study is the host countries response to the Syrian refugee crisis. The case study is bounded by the following timeframe: the 2011 civil war outbreak in Syria, which prompted the refugee crisis, and the release of the key policies in October 2016. In Lebanon, the refugee crisis overlaps with the 2011 conflict in Syria and is still ongoing; however, in Ontario, it was primarily perceived as a crisis in 2015 when the decision was made to accept Syrian refugees. The case studied is the factors that influenced the use of research evidence in health policy-development processes to prepare health systems to respond to an influx of Syrian refugees. The embedded cases consist of recent key policies that host countries have developed to deal with the health of Syrian refugees. Both policies represent the most significant strategy dealing with the health system’s response made in the host country in the last 4 years.We included Lebanon and Canada’s province of Ontario because they have been among the top jurisdictions taking in registered Syrian refugees since the beginning of the crisis. This ensured that we are examining a country where the crisis has had a significant impact. Additionally, both countries have in place explicit mechanisms to support the use of research evidence in policymaking  center in Lebanon and McMaster Health Forum in Ontario), and where access to the key individuals and the documents required to conduct a robust case study could be facilitated through our contacts involved with these explicit mechanisms.Our first embedded case is Lebanon’s 2016 Health Response Strategy (HRS). In response to the influx of Syrian refugees in Lebanon and to address the pressures placed on the Lebanese health system, the Ministry of Public Health (MoPH) made the decision to develop a Health Response Strategy (HRS), which was released in 2015 and subsequently updated in October 2016. It served two interdependent strategic objectives: 1) to harness primary, secondary, and tertiary care to address the essential health needs of the displaced Syrians and host community; and 2) to strengthen national institutions and capacities and thereby enhance the resilience of the health system.The second embedded case is Ontario’s Ministry of Health and Long -Term Care (MOHLTC) 2016 Phase 2: Health System Action Plan, Syrian Refugees, which included a set of policies to prepare the province of Ontario for managing the current and future health status of Syrian refugees who moved to the province. It provided guidance on: roles and responsibilities of specific health system partners, general guidance and considerations for Syrian refugee healthcare, and resources available to support continuing Syrian refugee healthcare delivery.We identified and recruited key informants based on their involvement in the development of both embedded cases. These key informants included: 1) senior decision-makers, staff employed by Lebanon’s and Ontario’s Ministries of Health, international agencies , and at non-governmental organizations ; 2) policy advisors who helped inform the health policy-development process; and 3) healthcare providers who were involved with the development of the policies. The second stage involved snowball sampling by which research participants in the first stage were asked to identify any additional informants. We reviewed the policy documents to obtain the names of these key informants and communicated with key individuals familiar with the policy-development process to ensure that we found the most appropriate individuals to interview. Given the limited pool of potential participants with knowledge of the policy-development process, we aimed to complete 10 interviews for each country. A total of 18 informants were willing to participate in the interviews across the two settings , media articles using LexisNexis, transcripts of legislative debates , and published literature using PubMed. Google was searched for other document types  as was the Internet Archive for documents that are no longer available on internet websites  Center and by the Center for Systematic Reviews on Health Policy and Systems Research in 2014. First, a priority-setting exercise resulted in the production of systematic reviews to address the health needs of Syrian refugees , 34. Sec“there were clear gaps in available research evidence. When we found a gap in the evidence we did not commission a study because of concern of funds going to completing studies instead of services when services were severely underfunded. In the context of this emergency, we had to get the data ourselves without having flawless research paper with a perfect methodological method”There was limited use of research evidence to inform the policy-development process in Lebanon and Ontario. In Lebanon, key informants discussed the scarcity of available research evidence to inform the policy-development process. A senior decision-maker working at MoPH stated:“We had no available literature on what the health conditions of Syrian refugees were. What was really helpful for us on the ground was having a better understanding of the epidemiological conditions in those Syrian refugee populations in Turkey, Lebanon, and Jordan. That required us to look at whatever available evidence we had at the time which included some cross-sectional disease prevalence studies and make policy decisions based on that”In Ontario, a healthcare provider discussed the scarcity of available research evidence and highlighted that research evidence was mainly used to clarify the health issues of Syrian refugees and not necessarily to inform the policy options and implementation considerations stating:“the MoPH looked at what data was available in their hospital observatory system which was providing some quite reliable data and they had some data from network of primary healthcare and then there was other data from other studies being done. For example, there was the VASyR survey led by UNCHR and others that came up with interesting data on health aspects . There was the Hopkins survey looking at access and disease profile among refugees and host communities”A common finding across our two settings was that both governments drew on inputs from a variety of sources. The scarcity of research evidence led the Lebanese MoPH to rely primarily on data obtained from two main sources: MoPH databases  and reports from other organizations ). The data were used in specific and direct ways to learn about the health needs of, and how to support, Syrian refugees . This highlights the importance for up-to-date and accessible databases during a crisis. A senior decision-maker commented on the use of data by the MoPH to inform the policy-development process stating:“the information provided is not necessarily evidence-based reports. It is the expertise and knowledge of the wonderful staff and volunteers that we are constantly engaged with and interact with and that is the advantage of being a global organization that we can reach out to someone that is an Arabic speaking person who can help us better understand the context”A senior decision-maker highlighted how other sources of information  were used by the Ontario government to inform the policy-development process stating:“in a crisis situation, you do not have the time to search for systematic reviews. Systematic reviews were not always available. The approach to commission systematic reviews was costly and timely. This is why data, experience and practice of key stakeholders was more important”There was a degree of consensus across the two settings that some of the main barriers that resulted in the limited use of research evidence included: short time-frame to develop the policies, and accessibility to and availability of relevant systematic reviews. These practical constraints were reported by a senior decision-maker in Lebanon stating:Overall, our results identified that research evidence was not the main determining factor influencing the policy-development process for Syrian refugees in Lebanon and Ontario.Table “this strategy is produced by the MoPH with the endorsements of all stakeholders but not from higher authority. This is in comparison to Lebanon Crisis Response Plan (LCRP) where every single chapter in that plan had to get approval from every named ministry, in particular ministry of social affairs who were mandated to coordinate between all ministries to develop the plan. This leads to a lot of political factors playing a role in hindering the policy-development process”In Lebanon, the Health Response Strategy effectively faced no opposition because it fell under the sole jurisdiction of the MoPH National Health Steering committee: a technical committee headed by the MoPH General Director and comprised of major international and local partners and focused on informing the policy-development process . Senior decision-makers commented:“The fact that the [Federal] government was favorable got the ball rolling in a very significant way. When the Trudeau government was elected, the policy of the Liberal government, which was to open doors and receive people in need, meant that for us  we can start the dialogue about how to best help the federal government get there”Similarly, Ontario’s first-past-the-post electoral system meant that a majority Liberal government, elected in 2014, was able to act on the Liberal Federal government priorities of accepting Syrian refugees without significant chance for derailment from opposition parties . A senior decision-maker highlighted the Liberal Federal government decisions about welcoming Syrian refugees by stating:It is significant to highlight that the Ontario policy was supported by federally legislated funding attached to demarcated priorities for refugee health and the Lebanese policy was a bureaucratic exercise without elected official oversight. This meant that gathering support for the policies was not particularly challenging.“the National Health Steering committee was designed to get all the buy-ins, everyone on board, to make sure we did not miss on anything in the field. The point of the committee was to get all the perspectives on the table. It was mostly made up of the EU delegation, the World Bank, the big UN agencies all at the same table. The point was to have an open and honest conversation about the whole process, how to get money, and where it should go”Second, both jurisdictions relied on policy networks to inform the policy-development process. In Lebanon, the MoPH convened the National Health Steering committee for the purpose of informing the policy-development process. A senior decision-maker at the MoPH described the role of the National Health Steering committee stating:“one of the biggest takeaways was having key stakeholders from all different areas (not just the Ministry) but the key external stakeholders like Canadian Centre for Refugee and Immigrant Healthcare, Crossroads Clinic for Refugees, Canadian Red Cross (CRC) present at the same table was very important and very influential because what that brought to the entire planning exercise was the fact that because CRC was so involved in the community sector we were able to create those linkages”In Ontario, the Liberal government convened round-table discussions with key societal actors . These round-table discussions played a crucial role in the policy-development process as stated by a senior decision-maker:For Lebanon and Ontario, having direct contact and interactions with key stakeholders from various organizational affiliations played an instrumental role in the policy-development process.“what propelled the policy-development process was the safety of the vulnerable people”Similarly, the Syrian refugee crisis spoke to Ontarians values, such as inclusion and fairness, which underpinned the policy-development process. A healthcare provider, involved in informing the policy-development process, reaffirmed this by stating:“this [Ontario’s Health System Action Plan] was made for a host of reasons, one of which to restore the soul of the nation. A nation defined where everybody has recently been touched by the refugee and immigrant experience, where by definition everybody is from somewhere else”The Lebanese values of providing safety for displaced people underpinned the Health Response Strategy with one senior decision-maker stating:“everybody had the best intent of ensuring that the refugees are welcomed and received and healthy and are able to be supported in the best possible way”A senior decision-maker highlighted the willingness of Ontarians to help displaced Syrians by stating:“the National Health Steering committee, included major donors particularly the European Union, that was very engaged. The strategy worked both ways: mobilize more funding from international donors but also some of the donors committed to certain projects, like maternal and child health, which ended up shaping the strategy”In Lebanon, international donors played an important role in influencing the policy-development process. Given the protracted nature of the Syrian refugee crisis in Lebanon, the policy-development process was seen as a way to mobilize increased funding from international donors , and to align funding to targeted priorities . A senior decision-maker highlighted the influence of international donors on the policy-development process, stating:Our document analysis indicated that media coverage played a significant role in the policy-development process for Syrian refugees in Ontario. The photo of Alan Kurdi, a 3-year-old Syrian boy who drowned on 2 September 2015 in the Mediterranean Sea when his family was escaping Syria into Europe, became a focusing event among Ontarians and the governing Ontario Liberal party that accelerated the policy-development process , 36.Our study provided a deeper understanding of the factors that influence the policy-development process in crisis zones and the role of research evidence in the process. This study identified four broad factors that help to explain the overall policy-development for Syrian refugees in Lebanon and Ontario: development of health policies without significant chance for derailment from other government bodies (Lebanon) or opposition parties (Ontario) , government’s engagement with key societal actors to inform the policy-development process, the embedded values underpinning the process, and external factors significantly influencing the policy-development process. These different factors provide insight into the influence of political factors in policy processes in crisis zones, which could help inform future policy-development processes.Our study found that policymakers in Lebanon and Ontario voiced similar challenges with navigating the gaps in available research evidence to inform the policy options and implementation considerations relevant to the policy-development process . Policymakers raised the issue of inadequate access to systematic reviews in a short-time frame. It is a surprising finding that although evidence websites are available to support this particular issue, they were not brought forward to policymakers as a source for accessing information and/or perceived as helpful  .In Lebanon, our documentary analysis revealed that there were multiple efforts to support the use of research evidence in the policy-development process , with none of our participants referencing such efforts in our interviews for two possible reasons. First, this could be a result of our informants’ recollection of activities that occurred in 2014, during interviews conducted in 2018. Second, a case study has shown that although the use of knowledge translation strategies in Lebanon helps to generate evidence-informed policymaking, there is still a need to better link those knowledge translation strategies to specific policy-development processes . For exaOur finding show that Lebanon’s and Ontario’s governments shared similar challenges in the perceived scarcity of available research evidence to inform the policy development for Syrian refugees aligns with previous studies identifying that there are perceived gaps in the research evidence to inform policy development about humanitarian crises , 39–43. There are four strengths to this study. First, this is the first study to address the knowledge gap in our understanding of the policy-development process for Syrian refugees on two health systems – those of Lebanon and Ontario – operating in very different contexts. Second, the embedded case study design allowed for cross-case analysis thereby providing an in-depth analysis of the factors that influence the policy-development process for Syrian refugees in Lebanon and Ontario and the role of research evidence in the process. Third, we interviewed diverse types of decision-makers encompassing various organizational affiliations to arrive at a comprehensive story of how the policy-development process for Syrian refugees unfolded. Finally, the use of an existing theoretical framework, the only one that provides a comprehensive inventory of the full range of likely political factors, was used to explain the policy-development process for Syrian refugees in Lebanon and Ontario.One significant challenge to this study related to recruitment of key informants from Ontario’s MOHLTC. Initially there was interest by decision-makers at the MOHLTC to participate in our study with the caveat of waiting until the newly elected government took office. However, after the change of government, we were informed that our request for interviews was declined. We took the following step to address this challenge: we interviewed other key informants who were identified through documentary analysis and by the MOHLTC as actors who were directly involved in the policy-development process.Another limitation to this study is that the retrieval of media articles in Lebanon was challenging as many of the articles were archived and difficult to access. We addressed this limitation by soliciting the help of a specialized communication officer, who provided us with newspapers articles retrieved from a press tracing exercise conducted on Syrian refugees.The results of our study carry with them some implications. First, this study suggests that other types of information — if not always research evidence — can play an instrumental role in answering specific and direct questions in policy development , however, when it comes to a large-scale decision  other factors are more salient such as the lack of institutional constraints such as veto points. Second, given policymakers’ perception of the scarcity of available research evidence to inform policy development, policymakers not utilizing available evidence websites, and policymakers’ reliance on key stakeholders to share their knowledge and expertise reaffirms the importance for networks to be in place to coordinate and share quality and timely evidence with all stakeholders. For example, EVIPNet is a network established by the World Health Organization to promote the systematic use of data and research evidence in health policymaking by providing key outputs, such as national clearing houses, that aim at providing an opportunity for sharing evidence among all stakeholders . EVIPNetOur methodology of using an existing theoretical framework, the diversity in our types of decision-makers and organizational affiliation, and key findings on supporting the use of evidence in policy development can be used by researchers studying the policy-development process for Syrian refugees in other host communities  and to design and evaluate an intervention to support evidence use in these communities.This study suggests that use of research evidence in the policy-development process for Syrian refugees was subordinate to key political factors, resulting in limited influence of research evidence in the development of both the Lebanese and Ontarian policy. This study highlights the need for interested and committed policymakers who value the role of research evidence in informing policymaking.Additional file 1. Appendix 1. Data collection and sampling for media, published literature and policy documents.Additional file 2. Appendix 2. Interview guide.


In evidence-informed policy-making (EIP), major knowledge gaps remain in understanding the context and possibilities for institutionalisation of knowledge translation. In 2014, the WHO Evidence-informed Policy Network (EVIPNet) Europe initiated a number of pilot countries, with Hungary among them, to engage in a ‘situation analysis’ (SA) in order to fill some of those gaps. This contribution discusses the results of the SA in Hungary on research–policy interactions, facilitating factors and potential barriers to establish a knowledge translation platform (KTP).In line with the EVIPNet Europe SA Manual, a document analysis, 13 interviews, 3 focus group discussions with 21 participants, and an online survey with 31 respondents were carried out from April to October, 2015. A SA aims to assess the context in which EIP takes form and seeks opportunities to establish a KTP, so information was gathered on the current practice of EIP and knowledge translation, its relevant actors, enablers and barriers for EIP, and opinions on a future KTP. Methodological and researcher triangulation resulted in a narrative synthesis of data, including a comparison with literature. A stakeholder consultation was organised to validate findings.This study reveals that stakeholders show commitment to produce and use research evidence in Hungarian health policy-making. All stakeholders endorsed the idea of strengthening the systematic use of evidence in decision-making and favoured the idea of establishing a KTP. In line with literature on other countries, some good practices exist on the uptake of evidence in policy-making; however, a systematic approach of developing, translating and using research evidence in health policy processes is lacking. EIP is currently hampered by scattered capacity, coordination problems, high fluctuation in government, an often legalistic and a more ‘symbolic’ rather than practical support for knowledge translation and EIP. The article summarises recommendations on a Hungarian KTP.Pragmatic adaptation of the SA Manual to local needs proved to be a useful mechanism to provide insight into the Hungarian EIP field and the establishment of a potential KTP. Despite the success of a KTP pilot, it remains unclear how a KTP in Hungary will be institutionalised in a sustainable way.The online version of this article (10.1186/s12961-018-0331-z) contains supplementary material, which is available to authorized users. The procedure [for decision preparation] is described in detail, the decree is kept, for everybody to see, yet the final decision has nothing to do with the whole thing.” (Representative of a government agency 2)“Nevertheless, some participants judged the compulsory deliberative process as an ineffective way to incorporate evidence as these consultations generally did not bring tangible outcomes.Throughout the past 25 years, various initiatives were taken to institutionalise a dialog between research and policy-making , but none of these proved to be long lasting or influential. Many conferences, roundtables, discussion forums, networks and short courses are organised to disseminate research and raise policy-maker. Participants stressed that, despite the frequent attendance by high-level public sector representatives at these events, policy recommendations and opinions provided were rarely taken into consideration. However, some participants reckoned that a number of individual researchers, universities and certain research institutes were sufficiently influential to impact on policy decisions on a personal and more informal level. In any case, the professional advisory boards of the Ministry can provide a solid base to discuss evidence and advise the policy issues for requests of the Ministry . Furthermore, these advisory boards play a role in the clinical governance of medical professions, for instance, by updating clinical guidelines.Hungary has always been eminent in putting in place things that look good: this is how decision-making should be done. But the problem that these changes stop on the legislative level. So it is not true that we have no frames to fill with content, the problem rather is that there is not too much ambition to really fill it with content.” (Researcher 2)“Every legislative act is accompanied by a so called impact assessment sheet, it’s ridiculous, a mere Excel table that need not even be filled entirely, [it’s there] just to have something. No one ever opens it, looks at it, but still looks as if an impact assessment was made about the measure.” (Representative of a government agency 3)“In recent years, the government introduced regulations that can foster the use of evidence in policy-making . However, some participants described these regulatory obligations as mainly theoretical or symbolic in nature.but what I currently see, that decision-preparation, at least what is done by us, is aiming at a line of conduct based on impact assessment.” (Representative of a government agency 4)“[…] I think that, for instance, talking about the deliberations on Healthy Hungary strategy, or how much dialog we had about smoking regulations or about the tasks of health promotion offices, so I believe that beside particular sectoral interests, there is a will for cooperation too.” (Policy-maker 1)“At the same time, other government officials were more satisfied and did not express a need to reinforce the implementation of the regulations.Some participants referred to mass media having the role to raise public awareness regarding health policy problems, proposed policy solutions and the rationale for these solutions, thereby putting pressure on policy-makers to enhance the uptake of scientific findings in decision-making. Others thought that the only way to persuade politicians and thus influence politics was through the constituency.The national health research system is shaped mostly by priorities of funders and researchers rather than a central government strategy. In Hungary, the ‘Investment in the Future’ national research and development and innovation strategy 2013–2020, published in 2013 , delineaHungarian experts have a good reputation, and if we are talking about health economists, Hungary has a great expert lineup on a regional level. There are established research groups, but experts within the government are hurtfully needed. Even those [research oriented working groups] that do exist [within the government] suffer from considerable fluctuation that makes the situation really difficult.” (Researcher 3)“Hungary does not have a government institute whose primary task is to provide policy-relevant research and KT in health system topics like health financing, health system governance, human resources and quality of care. Participants of interviews and focus groups stated that there was potential to enhance EIP as Hungary had sufficient research capacity in academic institutions, as well as research and consultancy companies to support health policy-making. However, public administration lacked the capacity (human resources and knowledge) to access the evidence, synthetise it in the form of high quality decision support tools and apply it in the policy-making process, and was characterised by frequent organisational changes, as participants noted. The Ministry of Human Capacities is particularly overloaded and has high staff-turn-over, inducing lack of time and capacity to engage in preparing strategic decision-making.The majority of both academic and governmental respondents of online questionnaires mentioned that they have financial resource problems, difficulties in accessing health databases, limited opportunities to participate in international conferences and trainings, and limited access to literature databases.EIP examples were identified in two fields, namely (1) in health technology assessment in the decision-making process of pharmaceuticals , and (2)Based on all reflections and commentaries made by stakeholders, a SWOT framework was developed Table . StrengtParticipants endorsed the idea of strengthening EIP and establishing a KTP under the conditions that activities were policy relevant, the KTP maintained its independence and stood for scientific quality. There was general agreement that establishing a KTP would be useful if it aligned with existing activities and capacity in the country. Its role was not to control decision-makers but to supplement the work of the State Secretariat, government agencies and academic institutions, avoiding duplication of activities and unnecessary competition. Its core task would be identifying and channelling of evidence into the policy process. Coordination, synthesis, evidence appraisal and creating platforms of cooperation and discussions should be the core part of the KTP’s tasks, while also providing training in EIP and KT.Numerous participants considered that the State Secretariat for Healthcare could be host of a KTP. At the same time, remarks were made that the daily working culture of the State Secretariat could not provide a protected working time, staff and stable organisational structure for EIP, as its daily routine was mostly guided by quickly changing working priorities. Alternative hosts mentioned were the Professional College , the Hungarian Academy of Science, one of the universities, NGOs or WHO.what does not have legislative and institutional background does not exist” (Researcher 4).Many participants stressed that the KTP should be framed in legislation; as a researcher put it: “Participants considered an independent steering body as part of the organisation’s governance as a necessity. It would consist of representatives of academics, NGOs, advocacy groups and the government. A strong, committed leader who can advocate for the activities of the KTP to politicians and opinion leaders would be vital.The main features of a Hungarian KTP were summarised by the national EVIPNet Europe team. The team adapted the Lavis et al.  criteriaThe Hungarian KTP should be embedded in an existing structure and a network approach should be developed whilst using and channelling existing capacity in the country.The KTP should operate sufficiently separate from the ‘regular’ public administration , while being close enough to understand policy needs. The KTP should enable policy-makers to express their information needs and these needs should to be handled as priorities by the KTP, with subsequent resource allocation.The KTP governance structure should include (1) KTP Board of Trustees, laying down the strategic framework and performing subsequent monitoring of activities; (2) KTP Office with its salaried staff being responsible for the day-to-day operation and coordination; (3) KTP Network of Scientific Advisors as the scientific reflection chamber and which allows access to collaboration with universities, expert groups, NGOs, etc.; (4) the leader of the KTP Office is the main person responsible for carrying out KT tasks and ensuring that the necessary staff is available; (5) ideally, an informal ‘KTP ambassador’  who represents the KTP in the media, political happenings and high-level professional forums would be useful, but could be very hard to find.Considering the current Hungarian situation, the novelty of the idea of a KTP and the limited resources , a gradual implementation strategy would increase the chances of sustainable institutionalisation.The Hungarian KTP should cooperate with EVIPNet.The core features of a Hungarian KTPSupport the identification of health policy research priorities and topics in close collaboration with policy-makers and in line with health policy strategies.Synthetise and package evidence and communicate context-sensitive policy recommendations to policy-makers in a user-friendly way (evidence briefs).Organise policy dialogs and catalyse collaboration among researchers, policy-makers and representatives of health providers and patients.Support capacity-building for KT both within and outside the KTP.The following basic tasks would be key:This summary of a SA in 2015 in Hungary, one of EVIPNet Europe’s pilot countries, provides a yet unprecedented, systematic insight into the interaction between health research and health policy in Hungary. Emerging insights were developed in dialog with experts from the Hungarian research and policy field. The exercise enables the discussion of opportunities to enhance EIP and KT in Hungary.This is not a formal research project; rather, the SA tool aims to develop a more systematic rather than just intuitive expert opinion-based approach. An EVIPNet SA aims to gather background information that supports a systematic and comprehensive reflection on the most important local factors that will either support or act as barriers to the establishment and operationalisation of future KTPs. Conducting an SA is an important process to (1) understand the EIP landscape and obtain a baseline; (2) obtain evidence guiding the next steps of the countries KT activities; and (3) to bring various stakeholders together and increasing their awareness on the need to further engage and invest in EIP. While several tools and approaches are meant to assist users in tackling the SA, the Manual is not a rigid protocol. The data collection methods should be, as required, adapted to the local context. The tool aims to create transparency in the sources and data used to assess the local situation. It is clear, as experiences in other EVIPNet pilot countries show, that the use of the SA toolkit is not a ‘formalised protocol’, but requires flexibility, interpretation and continuous coaching, which is also a key feature of qualitative and realist health system research. Moreover, the SA toolkit urges the comparison of the findings with existing literature on the country for their validation. The approaches of other EVIPNet Europe pilot countries in conducting an SA are varied, as every country chose different methodologies from the menu of options presented in the SA Manual.The findings of this SA are in line with the limited published knowledge on EIP in Hungary , 31, papOrosz  explaineThe lack of appropriate planning, consequent implementation and proper monitoring and evaluation of policies, as was mentioned in our SA, was also found by Gajduschek , who obsgap between intentions and actual practices” as well as providing several ad hoc examples of successful knowledge translation; however, similarly to us, it found that a systematic, institutionalised approach is lacking. Santoro et al. [Slovenia is another EVIPNet Europe country that conducted an SA towards the institutionalisation of a KTP . Their ao et al.  found tho et al. , reportio et al.  reportedo et al.  noted tho et al. , coverino et al.  on counto et al. , yet surIn summary, we conclude that the Hungarian processes of evidence use are not unique, in the sense that the influence of research evidence on decisions is, at the minimum, uneven in most countries. There are commonalities to be found internationally in specific issues of the policy process and the uptake of evidence.The SA identified a number of opportunities and barriers for enhancing EIP in the country, and more specifically to create a KTP. Hungarian stakeholders are committed to advance the systematic use of evidence in the policy process and endorsed the idea of establishing a KTP. While, theoretically, six commonly established organisational forms can be distinguished , the twoAn eventual standardised comparison of the KTP approaches that different countries took would yield information about the patterns emerging in the EVIPNet pilots. This would require a systematic approach and will be taken up in a future publication.This SA exercise has strengths and weaknesses. The EVIPNet Europe draft SA Manual proved to be a supportive tool, but very complex in hindsight. Using it required a balance between (1) the complexity of issues and level of detail needed and (2) the available resources for performing the SA . Specific issues regarding language and concepts when using an international (English written) Manual needed to be tackled when entering the local field. A considerable amount of time was necessary to customise and contextualise content and methods to local conditions. Additionally, no validation of the Hungarian translations took place, which could distort the distribution of answers and thus the conclusions drawn.Findings could be subject to a bias due to the method for inclusion of the stakeholders. On the one hand, purposive sampling can ensure, in a time-effective manner, that those who can serve as important primary data sources are included; on the other hand, it can be subject to researcher bias due to errors in researchers’ judgment. As a consequence, this sampling technique can be used to provide insights into the EIP landscape in Hungary and offer various views of a wide range of actors, but is not fit for generalisation of findings to all possible stakeholders. Although the national EVIPNet Europe team reached out to all major stakeholders to the best of their knowledge, researchers were overrepresented compared to other groups, which probably influenced the commentaries. Methodological precautions  were performed to reduce interpretation bias. The inductive coding method applied to focus group transcripts allowed for themes to emerge, which contributed to the robustness of results.PM and IB were public servants at the time of the study, thus having a potential conflict of interest regarding revealing unfavourable data on government processes. We managed this potential conflict by applying rigorous scientific methods  and disclosure.Readers should be aware that, within the pragmatic constraints of resources and time, the methods used do not claim to have made the ultimate assessment of the current Hungarian situation. Future follow-up measurements and in-depth studies will certainly strengthen the future EIP and KT field in Hungary.The activities of the SA had a wider impact outside the scope of the SA itself – since the submission of the EVIPNet Europe SA to the State Secretariat for Healthcare, the pilot development of an evidence brief for policy (EBP) on antimicrobial resistance had been initiated jointly by WHO Europe and the State Secretariat. This first Hungarian EBP was finalised during 2016–2017 and presented at a policy dialog at the end of 2017. During the course of the EBP development and introduction to stakeholders, one of the KTP options was tested.In Hungary, a multi-stakeholder group was formed . This combination proved fruitful because (1) the team was well familiarised with EVIPNet Europe’s goals, tools and activities, including the experience of other EVIPNet Europe member countries conducting SAs, and (2) a co-production approach was implemented; both proved beneficial in view of the team understanding the task and moving on swiftly in its implementation. The SA team in Hungary set precedence in applying a diversity of data collection methods to understand the EIP context, exceeding what has been performed in other countries and providing a more refined insight into the existing EIP field.The SA toolkit proved to be a supportive tool to assess the local health system and health system research context as well as to reflect and engage a discussion on the relevance of EIP and on the potential to establish a KTP. The dialog with stakeholders is a first step to enhance awareness with a wide range of stakeholders at country level. Owing to piloting the SA Manual and process, EVIPNet Europe member countries that are now embarking on conducting a SA benefit from a more user-friendly version of the SA Manual, as well as the possibility of mentoring and peer support provided by the experienced pilot countries.While one of the proposed KTP options was tested as a pilot, no institutionalised KTP has been created as of 2018. We can conclude that, while a SA can be considered as a useful, or even necessary, step for establishing a KTP, it is not a sufficient one. This Hungarian experience is in line with development in other countries – while a number of them have undertaken SA , thus far, no KTP has been established and country teams are still operating in a more informal manner. This is partially due to high government turnovers in the countries that we are operating, institutional changes in the heath policy landscape, the lack of resources to commit to longer-term investment into a KTP and, in some countries, the turnover of experts with whom the EVIPNet Secretariat has worked and trained.Additional file 1:a Online questionnaire for academic institutes (researchers). b Online questionnaires for government agencies and State Secretariat for Healthcare. (DOCX 169 kb)Additional file 2:Guide for focus group discussions in ‘Evidence-Informed Policy-making’. (DOCX 29 kb)Additional file 3:Semi-structured interview questions and information given on the KTP concept. (DOCX 25 kb)Additional file 4:Document analysis questions. (DOCX 24 kb)Additional file 5:Stakeholder mapping. Institutions, actors interested in the development of evidence-informed health policy practice and the current experiences – summary of group work at the EVIPNet Hungary launch event. (DOCX 33 kb)Additional file 6:Enablers and barriers of evidence-informed health policy practice – summary of group work at the EVIPNet Hungary launch event. (DOCX 33 kb)Additional file 7:Questions supporting group work. Questions supporting group work during EVIPNet Hungary launch event. (DOCX 23 kb)Additional file 8:Information given on the KTP concept to participants at the EVIPNet Hungary launch event. (DOCX 24 kb)Additional file 9:Two options for a KTP in Hungary. (DOCX 13 kb)