<commit_before><html>
<head>
<title>Caches and databases</title>
<meta name="order" content="4" />
</head>

<body>
    
<div>
	
	<div style="float:right"><img src="/img/vep_logo.png"/></div>
	
	<h1 id="top"><span style="color:#006">Variant Effect Predictor</span> <img src="/i/16/database.png"> <span style="color:#666">Caches and databases</span></h1>
	<hr/>	
	
	<p> The VEP script can use a variety of data sources to retrieve transcript
	information that is used to predict consequence types. </p>
    
    <p> Using a local cache is the most efficient way to use the VEP; we would
    encourage users to use the cache wherever possible. Caches are easy to
    download and set up using the <a
    href="vep_download.html#installer">installer</a>. Follow the <a
    href="vep_tutorial.html">tutorial</a> for a simple guide.</p>
	
	
	<h2 id="cache">Using the cache</h2>
	
    <p> Using the cache (<a href="vep_options.html#opt_cache">--cache</a>) is
    the fastest and most efficient way to use the VEP script, as in most cases
    only a single initial network connection is made and most data is read from
    local disk. Use <a href="#offline">offline</a> mode eliminate all network
    connections.</p>
	
    <p> Cache files are compressed using the gzip utility. By default zcat is
    used to decompress the files, although gzcat or gzip itself can be used to
    decompress also - you must have one of these utilities installed in your
    path to use the cache. Use <a
    href="vep_options.html#opt_compress">--compress [command]</a> to change the
    default.</p>
	
	<hr/>
	<h2 id="pre">Pre-built caches</h2>
	
	<p> The easiest solution is to download a pre-built cache for your species;
	this eliminates the need to connect to the database while the script is
	running (except when using <a href="#limitations">certain options</a>).
	Cache files can either be downloaded and unpacked as described here, or
	automatically downloaded and configured using the <a
	href="vep_download.html#installer">installer script</a>. <p>
    
    <p> Cache files for popular species: </p>
    
    <ul>
        <li><a href="ftp://ftp.ensembl.org/pub/release-[[SPECIESDEFS::ENSEMBL_VERSION]]/variation/VEP/homo_sapiens_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz">Human <i>(Homo sapiens)</i></a></li>
        <li><a href="ftp://ftp.ensembl.org/pub/release-[[SPECIESDEFS::ENSEMBL_VERSION]]/variation/VEP/mus_musculus_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz">Mouse <i>(Mus musculus)</i></a></li>
        <li><a href="ftp://ftp.ensembl.org/pub/release-[[SPECIESDEFS::ENSEMBL_VERSION]]/variation/VEP/danio_rerio_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz">Zebrafish <i>(Danio rerio)</i></a></li>
    </ul>
    
    <p> Other species: </p>
    
    <ul>
        <li><a href="ftp://ftp.ensembl.org/pub/current_variation/VEP/">Vertebrates</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/bacteria/current/vep/">Bacteria</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/fungi/current/vep/">Fungi</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/metazoa/current/vep/">Metazoa</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/plants/current/vep/">Plants</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/protists/current/vep/">Protists</a></li>
    </ul>
    
    <h4>Instructions for use</h4>
    
	<ol>
		<li>
			Download the archive file for your species</a>
		</li>
		<li>
			Extract the archive in your cache directory. By default the VEP uses
			$HOME/.vep/ as the cache directory, where $HOME is your UNIX home
			directory.
			<pre class="code">mv homo_sapiens_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz ~/.vep/
cd ~/.vep/
tar xfz homo_sapiens_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz</pre>
		</li>
		<li>
			Run the VEP with the <a href="vep_options.html#opt_cache">--cache</a> option
		</li>
	</ol>
	
	<p> Caches for several species, and indeed different Ensembl releases of the
	same species, can be stored in the same cache base directory. The files are
	stored in the following directory hierarchy: $HOME -> .vep -> species ->
	version -> chromosome </p>
	
	<p> If a pre-built cache does not exist for your species, please contact the
	<a href="http://lists.ensembl.org/mailman/listinfo/dev" rel="external">ensembl-dev</a> mailing list and we will
	endeavour to add your species to the list of downloads. </p>
	
	<p> It is also possible to build your own <a href="#gtf">cache from a GTF
	file</a> and a FASTA file. </p>
	
	<p> It is possible to use any combination of cache and database; when using
	the cache, the cache will take preference, with the database being used when
	the relevant data is not found in the cache. </p>
	
	
	<hr/>
	<h2 id="gtf">Building a cache from a GTF file</h2>
	
	<p> For species that don't have a publicly available cache, or even an
	Ensembl core database, it is possible to build a VEP cache using the
	gtf2vep.pl script included alongside the main script. This requires a GTF
	file and a FASTA file containing the reference sequence for the same
	species. The GTF file have its features sorted in chromosomal order, and
	must contain the following entry types and data: </p>
	
	<ul>
		<li>"exon" feature lines with have "transcript_id", "gene_id" and<commit_msg>fix a couple of typos<commit_after><html>
<head>
<title>Caches and databases</title>
<meta name="order" content="4" />
</head>

<body>
    
<div>
	
	<div style="float:right"><img src="/img/vep_logo.png"/></div>
	
	<h1 id="top"><span style="color:#006">Variant Effect Predictor</span> <img src="/i/16/database.png"> <span style="color:#666">Caches and databases</span></h1>
	<hr/>	
	
	<p> The VEP script can use a variety of data sources to retrieve transcript
	information that is used to predict consequence types. </p>
    
    <p> Using a local cache is the most efficient way to use the VEP; we would
    encourage users to use the cache wherever possible. Caches are easy to
    download and set up using the <a
    href="vep_download.html#installer">installer</a>. Follow the <a
    href="vep_tutorial.html">tutorial</a> for a simple guide.</p>
	
	
	<h2 id="cache">Using the cache</h2>
	
    <p> Using the cache (<a href="vep_options.html#opt_cache">--cache</a>) is
    the fastest and most efficient way to use the VEP script, as in most cases
    only a single initial network connection is made and most data is read from
    local disk. Use <a href="#offline">offline</a> mode eliminate all network
    connections.</p>
	
    <p> Cache files are compressed using the gzip utility. By default zcat is
    used to decompress the files, although gzcat or gzip itself can be used to
    decompress also - you must have one of these utilities installed in your
    path to use the cache. Use <a
    href="vep_options.html#opt_compress">--compress [command]</a> to change the
    default.</p>
	
	<hr/>
	<h2 id="pre">Pre-built caches</h2>
	
	<p> The easiest solution is to download a pre-built cache for your species;
	this eliminates the need to connect to the database while the script is
	running (except when using <a href="#limitations">certain options</a>).
	Cache files can either be downloaded and unpacked as described here, or
	automatically downloaded and configured using the <a
	href="vep_download.html#installer">installer script</a>. <p>
    
    <p> Cache files for popular species: </p>
    
    <ul>
        <li><a href="ftp://ftp.ensembl.org/pub/release-[[SPECIESDEFS::ENSEMBL_VERSION]]/variation/VEP/homo_sapiens_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz">Human <i>(Homo sapiens)</i></a></li>
        <li><a href="ftp://ftp.ensembl.org/pub/release-[[SPECIESDEFS::ENSEMBL_VERSION]]/variation/VEP/mus_musculus_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz">Mouse <i>(Mus musculus)</i></a></li>
        <li><a href="ftp://ftp.ensembl.org/pub/release-[[SPECIESDEFS::ENSEMBL_VERSION]]/variation/VEP/danio_rerio_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz">Zebrafish <i>(Danio rerio)</i></a></li>
    </ul>
    
    <p> Other species: </p>
    
    <ul>
        <li><a href="ftp://ftp.ensembl.org/pub/current_variation/VEP/">Vertebrates</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/bacteria/current/vep/">Bacteria</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/fungi/current/vep/">Fungi</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/metazoa/current/vep/">Metazoa</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/plants/current/vep/">Plants</a></li>
        <li><a href="ftp://ftp.ensemblgenomes.org/pub/protists/current/vep/">Protists</a></li>
    </ul>
    
    <h4>Instructions for use</h4>
    
	<ol>
		<li>
			Download the archive file for your species</a>
		</li>
		<li>
			Extract the archive in your cache directory. By default the VEP uses
			$HOME/.vep/ as the cache directory, where $HOME is your UNIX home
			directory.
			<pre class="code">mv homo_sapiens_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz ~/.vep/
cd ~/.vep/
tar xfz homo_sapiens_vep_[[SPECIESDEFS::ENSEMBL_VERSION]].tar.gz</pre>
		</li>
		<li>
			Run the VEP with the <a href="vep_options.html#opt_cache">--cache</a> option
		</li>
	</ol>
	
	<p> Caches for several species, and indeed different Ensembl releases of the
	same species, can be stored in the same cache base directory. The files are
	stored in the following directory hierarchy: $HOME -> .vep -> species ->
	version -> chromosome </p>
	
	<p> If a pre-built cache does not exist for your species, please contact the
	<a href="http://lists.ensembl.org/mailman/listinfo/dev" rel="external">ensembl-dev</a> mailing list and we will
	endeavour to add your species to the list of downloads. </p>
	
	<p> It is also possible to build your own <a href="#gtf">cache from a GTF
	file</a> and a FASTA file. </p>
	
	<p> It is possible to use any combination of cache and database; when using
	the cache, the cache will take preference, with the database being used when
	the relevant data is not found in the cache. </p>
	
	
	<hr/>
	<h2 id="gtf">Building a cache from a GTF file</h2>
	
	<p> For species that don't have a publicly available cache, or even an
	Ensembl core database, it is possible to build a VEP cache using the
	gtf2vep.pl script included alongside the main script. This requires a GTF
	file and a FASTA file containing the reference sequence for the same
	species. The GTF file have its features sorted in chromosomal order, and
	must contain the following entry types and data: </p>
	
	<ul>
		<li>"exon" feature lines must have "transcript_id", "gene_id" and
{'fraction_non_alphanumeric': 0.13357332322849563, 'fraction_numerical': 0.0037893141341417205, 'mean_word_length': 3.7661399548532732, 'pattern_counts': {'":': 0, '<': 267, '<?xml version=': 0, '>': 275, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '411351', 'n_tokens_mistral': 3748, 'n_tokens_neox': 3509, 'n_words': 1139}