<commit_before>---
title: Command Line Interface
teaser: Download, train and package models, and debug spaCy
source: spacy/cli
menu:
  - ['Download', 'download']
  - ['Link', 'link']
  - ['Info', 'info']
  - ['Validate', 'validate']
  - ['Convert', 'convert']
  - ['Debug data', 'debug-data']
  - ['Train', 'train']
  - ['Pretrain', 'pretrain']
  - ['Init Model', 'init-model']
  - ['Evaluate', 'evaluate']
  - ['Package', 'package']
---

As of v1.7.0, spaCy comes with new command line helpers to download and link
models and show useful debugging information. For a list of available commands,
type `spacy --help`.

## Download {#download}

Download [models](/usage/models) for spaCy. The downloader finds the
best-matching compatible version, uses `pip install` to download the model as a
package and creates a [shortcut link](/usage/models#usage) if the model was
downloaded via a shortcut. Direct downloads don't perform any compatibility
checks and require the model name to be specified with its version (e.g.
`en_core_web_sm-2.2.0`).

> #### Downloading best practices
>
> The `download` command is mostly intended as a convenient, interactive wrapper
> – it performs compatibility checks and prints detailed messages in case things
> go wrong. It's **not recommended** to use this command as part of an automated
> process. If you know which model your project needs, you should consider a
> [direct download via pip](/usage/models#download-pip), or uploading the model
> to a local PyPi installation and fetching it straight from there. This will
> also allow you to add it as a versioned package dependency to your project.

```bash
$ python -m spacy download [model] [--direct] [pip args]
```

| Argument                              | Type               | Description                                                                                                                                                                                                    |
| ------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model`                               | positional         | Model name or shortcut (`en`, `de`, `en_core_web_sm`).                                                                                                                                                         |
| `--direct`, `-d`                      | flag               | Force direct download of exact model version.                                                                                                                                                                  |
| pip args <Tag variant="new">2.1</Tag> | -                  | Additional installation options to be passed to `pip install` when installing the model package. For example, `--user` to install to the user home directory or `--no-deps` to not install model dependencies. |
| `--help`, `-h`                        | flag               | Show help message and available arguments.                                                                                                                                                                     |
| **CREATES**                           | directory, symlink | The installed model package in your `site-packages` directory and a shortcut link as a symlink in `spacy/data` if installed via shortcut.                                                                      |

## Link {#link}

Create a [shortcut link](/usage/models#usage) for a model, either a Python
package or a local directory. This will let you load models from any location
using a custom name via [`spacy.load()`](/api/top-level#spacy.load).

<Infobox title="Important note" variant="warning">

In spaCy v1.x, you had to use the model data directory to set up a shortcut link
for a local path. As of v2.0, spaCy expects all shortcut links to be **loadable
model packages**. If you want to load a data directory, call
[`spacy.load()`](/api/top-level#spacy.load) or
[`Language.from_disk()`](/api/language#from_disk) with the path, or use the
[`package`](/api/cli#package) command to create a model package.

</Infobox>

```bash
$ python -m spacy link [origin] [link_name] [--force]
```

| Argument        | Type       | Description                                                     |
| --------------- | ---------- | --------------------------------------------------------------- |
| `origin`        | positional | Model name if package, or path to local directory.              |
| `link_name`     | positional | Name of the shortcut link to create.                            |
| `--force`, `-f` | flag       | Force overwriting of existing link.                             |
| `--help`, `-h`  | flag       | Show help message and available arguments.                      |
| **CREATES**     | symlink    | A shortcut link of the given name as a symlink in `spacy/data`. |

## Info {#info}

Print information about your spaCy installation, models and local setup, and
generate [Markdown](https://en.wikipedia.org/wiki/Markdown)-formatted markup to
copy-paste into [GitHub issues](https://github.com/explosion/spaCy/issues).

```bash
$ python -m spacy info [--markdown] [--silent]
```

```bash
$ python -m spacy info [model] [--markdown] [--silent]
```

| Argument                                         | Type       | Description                                                   |
| ------------------------------------------------ | ---------- | ------------------------------------------------------------- |
| `model`                                          | positional | A model, i.e. shortcut link, package name or path (optional). |
| `--markdown`, `-md`                              | flag       | Print information as Markdown.                                |
| `--silent`, `-s` <Tag variant="new">2.0.12</Tag> | flag       | Don't print anything, just return the values.                 |
| `--help`, `-h`                                   | flag       | Show help message and available arguments.                    |
| **PRINTS**                                       | `stdout`   | Information about your spaCy installation.                    |

## Validate {#validate new="2"}

Find all models installed in the current environment (both packages and shortcut
links) and check whether they are compatible with the currently installed
version of spaCy. Should be run after upgrading spaCy via `pip install -U spacy`
to ensure that all installed models are can be used with the new version. The
command is also useful to detect out-of-sync model links resulting from links
created in different virtual environments. It will a list of models, the
installed versions, the latest compatible version (if out of date) and the
commands for updating.

> #### Automated validation
>
> You can also use the `validate` command as part of your build process or test
> suite, to ensure all models are up to date before proceeding. If incompatible
> models or shortcut links are found, it will return `1`.

```bash
$ python -m spacy validate
```

| Argument   | Type     | Description                                               |
| ---------- | -------- | --------------------------------------------------------- |
| **PRINTS** | `stdout` | Details about the compatibility of your installed models. |

## Convert {#convert}

Convert files into spaCy's [JSON format](/api/annotation#json-input) for use
with the `train` command and other experiment management functions. The
converter can be specified on the command line, or chosen based on the file
extension of the input file.

```bash
$ python -m spacy convert [input_file] [output_dir] [--file-type] [--converter]
[--n-sents] [--morphology] [--lang]
```

| Argument                                         | Type       | Description                                                                                       |
| ------------------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------- |
| `input_file`                                     | positional | Input file.                                                                                       |
| `output_dir`                                     | positional | Output directory for converted file. Defaults to `"-"`, meaning data will be written to `stdout`. |
| `--file-type`, `-t` <Tag variant="new">2.1</Tag> | option     | Type of file to create (see below).                                                               |
| `--converter`, `-c` <Tag variant="new">2</Tag>   | option     | Name of converter to use (see below).                                                             |
| `--n-sents`, `-n`                                | option     | Number of sentences per document.                                                                 |
| `--seg-sents`, `-s` <Tag variant="new">2.2</Tag> | flag       | Segment sentences (for `-c ner`)                                                                  |
| `--model`, `-b` <Tag variant="new">2.2</Tag>     | option     | Model for parser-based sentence segmentation (for `-s`)                                           |
| `--morphology`, `-m`                             | option     | Enable appending morphology to tags.                                                              |
| `--lang`, `-l` <Tag variant="new">2.1</Tag>      | option     | Language code (if tokenizer required).                                                            |
| `--help`, `-h`                                   | flag       | Show help message and available arguments.                                                        |
| **CREATES**                                      | JSON       | Data in spaCy's [JSON format](/api/annotation#json-input).                                        |

### Output file types {new="2.1"}

All output files generated by this command are compatible with
[`spacy train`](/api/cli#train).

| ID      | Description                |
| ------- | -------------------------- |
| `json`  | Regular JSON (default).    |
| `jsonl` | Newline-delimited JSON.    |
| `msg`   | Binary MessagePack format. |

### Converter options

<!-- TODO: document jsonl option – maybe update it? -->

| ID                             | Description                                                                                                                                                                                                                                                                                                                                                                                    |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `auto`                         | Automatically pick converter based on file extension and file content (default).                                                                                                                                                                                                                                                                                                               |
| `conll`, `conllu`, `conllubio` | Universal Dependencies `.conllu` or `.conll` format.                                                                                                                                                                                                                                                                                                                                           |
| `ner`                          | NER with IOB/IOB2 tags, one token per line with columns separated by whitespace. The first column is the token and the final column is the IOB tag. Sentences are separated by blank lines and documents are separated by the line `-DOCSTART- -X- O O`. Supports CoNLL 2003 NER format. See [sample data](https://github.com/explosion/spaCy/tree/master/examples/training/ner_example_data). |
| `iob`                          | NER with IOB/IOB2 tags, one sentence per line with tokens separated by whitespace and annotation separated by `|`, either `word|B-ENT` or `word|POS|B-ENT`. See [sample data](https://github.com/explosion/spaCy/tree/master/examples/training/ner_example_data).                                                                                                                              |

## Debug data {#debug-data new="2.2"}<commit_msg>Document jsonl option on converter [ci skip]<commit_after>---
title: Command Line Interface
teaser: Download, train and package models, and debug spaCy
source: spacy/cli
menu:
  - ['Download', 'download']
  - ['Link', 'link']
  - ['Info', 'info']
  - ['Validate', 'validate']
  - ['Convert', 'convert']
  - ['Debug data', 'debug-data']
  - ['Train', 'train']
  - ['Pretrain', 'pretrain']
  - ['Init Model', 'init-model']
  - ['Evaluate', 'evaluate']
  - ['Package', 'package']
---

As of v1.7.0, spaCy comes with new command line helpers to download and link
models and show useful debugging information. For a list of available commands,
type `spacy --help`.

## Download {#download}

Download [models](/usage/models) for spaCy. The downloader finds the
best-matching compatible version, uses `pip install` to download the model as a
package and creates a [shortcut link](/usage/models#usage) if the model was
downloaded via a shortcut. Direct downloads don't perform any compatibility
checks and require the model name to be specified with its version (e.g.
`en_core_web_sm-2.2.0`).

> #### Downloading best practices
>
> The `download` command is mostly intended as a convenient, interactive wrapper
> – it performs compatibility checks and prints detailed messages in case things
> go wrong. It's **not recommended** to use this command as part of an automated
> process. If you know which model your project needs, you should consider a
> [direct download via pip](/usage/models#download-pip), or uploading the model
> to a local PyPi installation and fetching it straight from there. This will
> also allow you to add it as a versioned package dependency to your project.

```bash
$ python -m spacy download [model] [--direct] [pip args]
```

| Argument                              | Type               | Description                                                                                                                                                                                                    |
| ------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model`                               | positional         | Model name or shortcut (`en`, `de`, `en_core_web_sm`).                                                                                                                                                         |
| `--direct`, `-d`                      | flag               | Force direct download of exact model version.                                                                                                                                                                  |
| pip args <Tag variant="new">2.1</Tag> | -                  | Additional installation options to be passed to `pip install` when installing the model package. For example, `--user` to install to the user home directory or `--no-deps` to not install model dependencies. |
| `--help`, `-h`                        | flag               | Show help message and available arguments.                                                                                                                                                                     |
| **CREATES**                           | directory, symlink | The installed model package in your `site-packages` directory and a shortcut link as a symlink in `spacy/data` if installed via shortcut.                                                                      |

## Link {#link}

Create a [shortcut link](/usage/models#usage) for a model, either a Python
package or a local directory. This will let you load models from any location
using a custom name via [`spacy.load()`](/api/top-level#spacy.load).

<Infobox title="Important note" variant="warning">

In spaCy v1.x, you had to use the model data directory to set up a shortcut link
for a local path. As of v2.0, spaCy expects all shortcut links to be **loadable
model packages**. If you want to load a data directory, call
[`spacy.load()`](/api/top-level#spacy.load) or
[`Language.from_disk()`](/api/language#from_disk) with the path, or use the
[`package`](/api/cli#package) command to create a model package.

</Infobox>

```bash
$ python -m spacy link [origin] [link_name] [--force]
```

| Argument        | Type       | Description                                                     |
| --------------- | ---------- | --------------------------------------------------------------- |
| `origin`        | positional | Model name if package, or path to local directory.              |
| `link_name`     | positional | Name of the shortcut link to create.                            |
| `--force`, `-f` | flag       | Force overwriting of existing link.                             |
| `--help`, `-h`  | flag       | Show help message and available arguments.                      |
| **CREATES**     | symlink    | A shortcut link of the given name as a symlink in `spacy/data`. |

## Info {#info}

Print information about your spaCy installation, models and local setup, and
generate [Markdown](https://en.wikipedia.org/wiki/Markdown)-formatted markup to
copy-paste into [GitHub issues](https://github.com/explosion/spaCy/issues).

```bash
$ python -m spacy info [--markdown] [--silent]
```

```bash
$ python -m spacy info [model] [--markdown] [--silent]
```

| Argument                                         | Type       | Description                                                   |
| ------------------------------------------------ | ---------- | ------------------------------------------------------------- |
| `model`                                          | positional | A model, i.e. shortcut link, package name or path (optional). |
| `--markdown`, `-md`                              | flag       | Print information as Markdown.                                |
| `--silent`, `-s` <Tag variant="new">2.0.12</Tag> | flag       | Don't print anything, just return the values.                 |
| `--help`, `-h`                                   | flag       | Show help message and available arguments.                    |
| **PRINTS**                                       | `stdout`   | Information about your spaCy installation.                    |

## Validate {#validate new="2"}

Find all models installed in the current environment (both packages and shortcut
links) and check whether they are compatible with the currently installed
version of spaCy. Should be run after upgrading spaCy via `pip install -U spacy`
to ensure that all installed models are can be used with the new version. The
command is also useful to detect out-of-sync model links resulting from links
created in different virtual environments. It will a list of models, the
installed versions, the latest compatible version (if out of date) and the
commands for updating.

> #### Automated validation
>
> You can also use the `validate` command as part of your build process or test
> suite, to ensure all models are up to date before proceeding. If incompatible
> models or shortcut links are found, it will return `1`.

```bash
$ python -m spacy validate
```

| Argument   | Type     | Description                                               |
| ---------- | -------- | --------------------------------------------------------- |
| **PRINTS** | `stdout` | Details about the compatibility of your installed models. |

## Convert {#convert}

Convert files into spaCy's [JSON format](/api/annotation#json-input) for use
with the `train` command and other experiment management functions. The
converter can be specified on the command line, or chosen based on the file
extension of the input file.

```bash
$ python -m spacy convert [input_file] [output_dir] [--file-type] [--converter]
[--n-sents] [--morphology] [--lang]
```

| Argument                                         | Type       | Description                                                                                       |
| ------------------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------- |
| `input_file`                                     | positional | Input file.                                                                                       |
| `output_dir`                                     | positional | Output directory for converted file. Defaults to `"-"`, meaning data will be written to `stdout`. |
| `--file-type`, `-t` <Tag variant="new">2.1</Tag> | option     | Type of file to create (see below).                                                               |
| `--converter`, `-c` <Tag variant="new">2</Tag>   | option     | Name of converter to use (see below).                                                             |
| `--n-sents`, `-n`                                | option     | Number of sentences per document.                                                                 |
| `--seg-sents`, `-s` <Tag variant="new">2.2</Tag> | flag       | Segment sentences (for `-c ner`)                                                                  |
| `--model`, `-b` <Tag variant="new">2.2</Tag>     | option     | Model for parser-based sentence segmentation (for `-s`)                                           |
| `--morphology`, `-m`                             | option     | Enable appending morphology to tags.                                                              |
| `--lang`, `-l` <Tag variant="new">2.1</Tag>      | option     | Language code (if tokenizer required).                                                            |
| `--help`, `-h`                                   | flag       | Show help message and available arguments.                                                        |
| **CREATES**                                      | JSON       | Data in spaCy's [JSON format](/api/annotation#json-input).                                        |

### Output file types {new="2.1"}

All output files generated by this command are compatible with
[`spacy train`](/api/cli#train).

| ID      | Description                |
| ------- | -------------------------- |
| `json`  | Regular JSON (default).    |
| `jsonl` | Newline-delimited JSON.    |
| `msg`   | Binary MessagePack format. |

### Converter options

| ID                             | Description                                                                                                                                                                                                                                                                                                                                                                                    |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `auto`                         | Automatically pick converter based on file extension and file content (default).                                                                                                                                                                                                                                                                                                               |
| `conll`, `conllu`, `conllubio` | Universal Dependencies `.conllu` or `.conll` format.                                                                                                                                                                                                                                                                                                                                           |
| `ner`                          | NER with IOB/IOB2 tags, one token per line with columns separated by whitespace. The first column is the token and the final column is the IOB tag. Sentences are separated by blank lines and documents are separated by the line `-DOCSTART- -X- O O`. Supports CoNLL 2003 NER format. See [sample data](https://github.com/explosion/spaCy/tree/master/examples/training/ner_example_data). |
| `iob`                          | NER with IOB/IOB2 tags, one sentence per line with tokens separated by whitespace and annotation separated by `|`, either `word|B-ENT` or `word|POS|B-ENT`. See [sample data](https://github.com/explosion/spaCy/tree/master/examples/training/ner_example_data).                                                                                                                              |
| `jsonl`                        | NER data formatted as JSONL with one dict per line and a `"text"` and `"spans"` key. This is also the format exported by the [Prodigy](https://prodi.gy) annotation tool. See [sample data](https://raw.githubusercontent.com/explosion/projects/master/ner-fashion-brands/fashion_brands_training.jsonl).                                                                                     |

## Debug data {#debug-data new="2.2"}
{'fraction_non_alphanumeric': 0.17158893995328864, 'fraction_numerical': 0.0027122730354855722, 'mean_word_length': 1.0149525616698292, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 64, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 305, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6683960', 'n_tokens_mistral': 5371, 'n_tokens_neox': 5092, 'n_words': 2058}