Biology
Computer Science-2021.gz:143418
Computer Science Biology
5d451f803eea97327d279bfcd9dc3d74c244b331
FILER: large-scale, harmonized FunctIonaL gEnomics Repository

Motivation Querying massive collections of functional genomic and annotation data, linking and summarizing the query results across data sources and data types are important steps in high-throughput genomic and genetic analytical workflows. However, accomplishing these steps is difficult because of the heterogeneity and breadth of data sources, experimental assays, biological conditions (e.g., tissues, cell types), data types, and file formats. Results FunctIonaL gEnomics Repository (FILER) is a large-scale, harmonized functional genomics data catalog uniquely providing: 1) streamlined access to >50,000 harmonized, annotated functional genomic and annotation datasets across >20 integrated data sources, >1,100 biological conditions/tissues/cell types, and >20 experimental assays; 2) a scalable, indexing-based genomic querying interface; 3) ability for users to analyze and annotate their own experimental data against reference datasets. This rich resource spans >17 Billion genomic records for both GRCh37/hg19 and GRCh38/hg38 genome builds. FILER scales well with the experimental (query) data size and the number of reference datasets and data sources. When evaluated on large-scale analysis tasks, FILER demonstrated great efficiency as the observed running time for querying 1000x more genomic intervals (106 vs. 103) against all 7×109 hg19 FILER records increased sub-linearly by only a factor of 15x. Together, these features facilitate reproducible research and streamline querying, integrating, and utilizing large-scale functional genomics and annotation data. Availability and implementation FILER can be 1) freely accessed at https://lisanwanglab.org/FILER, 2) deployed on cloud or local servers (https://bitbucket.org/wanglab-upenn/FILER), and 3) integrated with other pipelines using provided scripts. Contact lswang@pennmedicine.upenn.edu

The primary functional genomic experimental data is made available by many major consortia such as ENCODE (Davis et al., 2018;Dunham et al., 2012), GTEx (Aguet et al., 2020, 2017, FANTOM5 (Andersson et al., 2014), and NIH Roadmap Epigenomics (Kundaje et al., 2015). So far, these four projects generated datasets spanning over >50,000 experiments across >1,000 tissues, cell types, biological conditions, with each dataset containing millions to billions of annotation or assay readout records across the genome. In order to pair these functional annotations with high-throughput analytical workflows, e.g., for processing current populationlevel studies such as UK Biobank (Bycroft et al., 2018) (500,000 individuals with >2,500 phenotypes), we need a scalable, unified, high-throughput and robust access to these massive, heterogeneous genomic data collections.
To address these issues, we developed FunctIonaL gEnomics Repository (FILER), a large-scale, curated, integrated catalog of harmonized functional genomic and annotation data coupled with a scalable genomic search and querying interface to these data. The latest FILER release provides seamless integration of >58,000 harmonized genomic datasets (data tracks) across diverse (>20) primary data sources organized into >140 data collections, wide biological context (>1,100 cell types) and various genomic and biological features (>30 experimental assays and data types).
FILER provides a unified access to this rich functional and annotation data resource spanning >17 Billion records across genome with >2,700x total genomic coverage for both GRCh37/hg19 and GRCh38/hg38. FILER can be used to perform flexible querying, staging and consolidation of these data for analyses. FILER is accessible via a web server (https://lisanwanglab.org/FILER) or can be deployed on users' own cloud computing instances, local servers, high-performance computing clusters (https://bitbucket.org/wanglab-upenn/FILER) and integrated with genomic and genetic analysis workflows.

2.1.
FILER structure and data organization FILER has been implemented using an easily updatable, extensible, and modular architecture.

Figure 1
gives an overview of the FILER architecture and data organization. FILER contents are derived from the integration of many data and annotation resources as well as curation and processing of publicly available functional genomics datasets ( Figure 1A). Supplementary 2) file schema table (Figure 1B), and 3) external, indexed data collections holding the actual genomic data (right-hand side of Figure 1B) referenced in the meta-information table.
We describe each of these tables in the next sections below.

FILER meta-information table
The FILER metadata table stores detailed information for each data track including the data source (e.g., ENCODE, Roadmap, FANTOM5), experimental assay (e.g., ChIP-seq, DNase-seq, ATAC-seq), type of genomic records (e.g., called peaks, transcription start sites, gene models), biological source (e.g., cell type, tissue, cell line), data provenance (e.g., the data source version (if applicable), the date data was accessed/downloaded, download URL). Both the original (as obtained from the data source) data files and the processed files are stored in FILER in separate folder locations. The metadata table store the references to the original file including the download URL and the local file location. Each data track is uniquely identified by a FILER identifier assigned when the data is added to FILER (see, e.g., Figure 1B and Supplementary

FILER file schema information table
The data tracks in FILER are stored in formats specific to particular data sources and/or experimental assays. The file schema table stores the schemas (see Figure 1B bottom) for each type of data tracks. These file schemas can be used to extract additional information from the genomic records contained in the track files. Each record in the file schema table defines the number of standard BED fields, the number of extra fields, as well as the names of all the fields in the genomic record (see Figure 1B (bottom left) for an example; for more detailed information on the FILER database schema table please refer to Supplementary Table S6).

FILER indexed data collections
The external indexed data collections (see Figure 1B, right panel) are referenced in the FILER metadata

FILER data harmonization and annotation pipeline
The main steps in the FILER data harmonization and annotation pipeline (Figure 2) include 1) data annotation (metadata extraction and generation), 2) data pre-processing and normalization, 3) data classification and organization into data collections, and 4) genomic interval-based data collection indexing (Layer et al., 2018).

Data annotation
All genomic tracks were annotated with the standard set of attributes (meta-information) including data type (ChIP-seq peaks, transcription factor binding sites (TFBS), gene models, etc), assay type (ChIP-seq, DNAse-seq, ATAC-seq, etc), cell/tissue type, data source, data source version, and other relevant meta-information (see Supplementary Table S4 describing FILER metadata schema). To extract/infer the necessary meta-information for each data track, the information provided in the original data source was parsed and matched to the standardized schema employed by FILER (see Supplementary Table S5 for details).

Figure 2.
FILER data annotation and harmonization pipeline. All functional genomic datasets from the curated data sources are annotated with a standardized, consistent meta-information to allow for integration across diverse data sources (see Sections 2.1, 2.2). All individual genomic datasets are systematically pre-processed into standard, uniform BED-like file formats, and grouped into data collections based on the data source, experimental data type, data format (see Section 2.1). Each of the resulting data collections is indexed to allow efficient access using genomic-interval based indexing and query engine (see Section 2.1 for details).

Data classification and organization into data collections
To further organize data and enable efficient search and retrieval of the functional genomics and annotation tracks, datasets from the same data source, same assay, same type of genomic records, same data format were physically grouped into data collections by data source, assay type, data type, and genome build (see Supplementary Table S3  Additionally, all genomic and annotation data tracks included in FILER were categorized based on the biological source, tissue/cell type into broader tissue/cell type categories to further enable cross-data source integration of the data tracks (see Supplementary Methods, Tissue categorization section for details of tissue/cell type-based categorization employed by FILER).
All FILER data tracks were annotated with a major tissue and organ systems category they belong to. FILER tracks were also systematically classified into biologically meaningful data categories (Section 2.1.1) to allow the users to more easily access datasets of interest.

Genomic-interval based data indexing
Each data collection in FILER (Supplementary with the coordinate range covered by each leaf. The generated index for each data collection allows for efficient access to the genomic records overlapping a query interval across all the tracks stored in the data collection. Using these data indexes, the data collection tracks can then be efficiently accessed using the FILER Giggle-based query engine (see Supplementary Methods).

Data access and availability
Data is added to the FILER periodically, with public versioned releases (data freezes) every six months. All data is physically stored in Amazon AWS (Amazon Web Services (AWS)https://aws.amazon.com/) cloud. Access to the data is provided using FILER web server (https://lisanwanglab.org/FILER). Importantly, a local copy of FILER can be deployed on userown servers or cloud computing instances (https://bitbucket.org/wanglab-upenn/FILER) to enable integration with existing or new analysis pipelines using the provided installation and data scripts.

FILER contents
FILER contains functional genomics and annotation data characterizing various biological features of the human genome. Figure 3 shows the distribution of functional genomic datasets integrated into FILER by data source, experimental data types, tissue/cell type categories, and biological sample types. FILER integrates functional genomic and annotation data across many primary data sources (Figure 3a) with the genomic/annotation records spanning a wide range of experimental assays/data types (Figure 3b). The data tracks in FILER are generated from a variety of biological sources (Figure 3c) and span a variety of tissue/cell type categories ( Figure   3d) corresponding to >1,000 tissues/cell types. Supplementary Figure S2 shows the distribution   In particular, each individual FILER dataset is described by a consistent, standardized set of information attributes (metadata, >30 features; Supplementary Table S4)   (iii) analyze and annotate user-provided experimental data using the reference datasets in FILER (Figure 4); (iv) deploy locally or on the cloud the entire FILER or a custom/selected subset of the FILER data.

FILER web-portal features
Supplementary Table S1 shows a comparison of the FILER features with currently existing tools and web servers.

Search
FILER allows users to search for relevant genomic datasets based on their properties (metainformation). The genomic records (genomic features and annotations) across datasets can also be searched using FILER based on a) genomic coordinates of the region of interest or b) a set of coordinates for the genomic loci, e.g., experimentally-derived regions. Next, we describe the search functionality provided by FILER in more detail. Figure 3a) in FILER allows users to locate genomics data tracks and features overlapping with the given genomic coordinates. All the overlapping genomic features, data tracks, and their associated meta-data will be returned (see Supplementary Table S9 for an example of annotated overlap BED file;

Search by genomic interval: Search by genomic interval (Supplementary
descriptions of the file columns are provided in Supplementary Table S4).
Analyze and annotate user's experimental data: As shown in Supplementary Figure S3, the experimental data is provided in BED format. The user BED file can either be uploaded to the FILER web server ('Upload your data' tab), or provided through a web-accessible URL ('Analyze your own data' tab).
Browse and search genomic datasets by data source, tissue/cell type, assay, data type and other properties: FILER allows users to locate relevant data tracks by a given metadata attribute or a combination of attributes (see Supplementary Figure S5; Section 3.4, Use Case 1 for an example). The selected data tracks matching the search parameters can be then downloaded including the processed, as well as the original (raw) datasets along with their associated metadata.

Analysis and annotation of user-provided experimental data
FILER allows users to locate all data tracks and genomic features overlapping user-provided experimental intervals. The experimental intervals in the user-provided BED file are compared with all FILER datasets and all overlapping genomic features, data tracks and meta-data are reported (Supplementary Figure S4).

FILER example use cases
Broad tissue/cell type coverage and a wide range of experimental data types available in FILER enable systematic analyses of genome-wide studies such as experimentally-derived ChIP-seq or RNA-seq genomic intervals or association signals observed in GWASs.
We next describe example use cases for FILER, including 1) using a FILER web server for identification and retrieval of relevant experimental datasets (Use case 1), 2) using custom/user experimental data with FILER webserver (Use case 2), and 3) integration into high-throughput analysis workflows (Use case 3) using FILER installation and data command-line scripts.
FILER can be used to identify and download relevant datasets using various criteria, including a particular human tissue context (tissue category), a particular experimental data type (assay), or a specific data source. As shown in Supplementary Figure S5, via the web-interface, data selectors allow users to choose specific data of interest. All the data tracks matching the selection criteria can be download in bulk or individually, including the associated track meta-information.
Alternatively, the command line interface provided by FILER (see Supplementary Methods, 'Deploying FILER data' section) allows users to deploy the relevant data subset of interest locally or on a cloud using the FILER meta-information table. The metadata will serve as a guide to download individual data tracks, re-create FILER data structure, and generate data indexes (see also Use case 3 for an example of integrating FILER data with a high-throughput workflow; Supplementary Methods, 'Deploying FILER data').
FILER can be used for analysis of users' own experimental data (e.g., experimentallyderived genomic intervals). In particular, FILER can be used to annotate custom genomic data from the user experiment, such as ChIP-seq peaks, RNA-seq small RNA peaks, or ATAC-seq open chromatin regions.
Using the provided BED file with experimental data as input, FILER will generate an annotated BED file containing all genomic records overlapping with the user-provided experimental intervals (available for download under 'Annotated overlaps' subsection).
The annotated overlap file will contain overlap records consisting of the three sections: 1) all the data fields in the user input, 2) overlapping FILER data track information, and 3) the associated FILER track meta-information. Along with the main annotated overlap file in BED format, FILER will generate a report detailing the distribution of overlapping genomic records across tissue categories, data sources, and experimental data types.
Additionally, overlapping genomic records corresponding to a particular tissue category/data source, or tissue category/experimental assay are available for individual download and further analysis.
In Figure 4, we showed the FILER results via the web-portal when analyzed using a sample of ChIP-seq ENCODE data (ENCODE accession ENCFF000AIA; 92,008 ChIP-seq peaks) as an example. This interactive heatmap allows users to download results in BED format for any specific tissue category and data source. for experimental assays (columns) and tissue categories (rows).
3) Use case 3: integration with a genetic and genomic analyses workflow.

FILER deployment
FILER is available in two different ways: 1) FILER data is accessible through the web server (https://lisanwanglab.org/FILER). It allows users to search and retrieve relevant data, retrieve genomic features and annotation for genomic regions of interest, and analyze and annotate user's experimental data with the reference FILER datasets.
2) FILER can be deployed locally on a server or cluster (https://bitbucket.org/wanglabupenn/FILER). This not only allows users to access harmonized functional genomics and annotation data stored in FILER, but also integrate FILER into custom analysis pipelines.
The scalable, genomic indexing-based interface allows to efficiently access FILER largescale functional genomics data collection and use it in custom analyses.

Discussion
Analyses of the results from GWASs and biological experiments require using external functional data as evidence for further interpretation, characterization and discovery.
However, there is currently no single resource that provides unified access to a harmonized collection of such functional genomics and annotation data. This complicates the search and use of relevant functional genomic data, as well as the comparison and aggregation of these heterogeneous datasets for the analyses. More importantly, without a unified, harmonized data resource it is difficult to use these valuable resources in various genomic and genetic pipelines.
We envision that the large-scale, harmonized FILER genomic and annotation data collection will facilitate the downstream genetic and genomic analyses, including but not limited to studying GWAS signals, via system biology, causal gene and other analyses. This allows researchers to focus on the creative analysis tasks, genomic analyses and discoveries rather than data collection and cleaning.
FILER uniquely provides an integrated and extensible repository of harmonized functional genomics and annotation data allowing for efficient and seamless retrieval, analysis and comparison across data sources, biological conditions, tissues/cell types, and experimental data types. Using provided deployment and data interface, FILER allows integration with the existing or new analytical workflows. In addition to the web-based access, FILER can be installed on a local server, high-performance computing (HPC) cluster or cloud computing instances (see

Section 3.5; Supplementary Methods).
Broad tissue/cell and experimental data type coverage in FILER enables systematic analyses of genome-wide experiments such as ChIP-seq or ATAC-seq, or genome-wide analyses of association signals observed in GWASs. Moreover, the modular, data collection-based FILER data architecture allows additional analysis-specific/user data or new data sources to be easily added without affecting other datasets/data collections.
Current limitations of the FILER web server include the usage of a single multi-core server with local data storage. Implementation based on Apache Spark (Zaharia et al., 2016) or similar distributed computing framework and distributed data storage would allow further scalability in terms of both the data size and the search and processing speed.
Future developments for FILER will include 1) broadening of tissue/cell type coverage, 2) adding and expanding experimental data types including chromatin interaction/3D genome organization data, gene and RNA expression, and single-cell experiments, 3) adding diseasespecific datasets, 4) variant-level annotations, 5) visualization, interactive display and exploration of functional genomic and annotation datasets.

Data availability
All harmonized genomic datasets and the corresponding annotation meta-data are freely available from the FILER website https://lisanwanglab.org/FILER. An entire FILER database or a selected data subset can be deployed on local server, cloud or high-performance computing