<commit_before>![DVC icone](https://s3-us-west-2.amazonaws.com/dvc-share/images/favicon_small.png)

# Data version control or DVC
DVC streamlines your git code, your data (S3 and GCP) and dependencies to a single reproducible environment.

# Introduction

It is hardly possible in real life to develop a good machine learning model in a single pass. ML modeling is an iterative process and it is extremely important to keep track of your steps, dependencies between the steps, dependencies between your code and data files and all code running arguments. This becomes even more important and complicated in a team environment where data scientists’ collaboration takes a serious amount of the team’s effort.

![alt text](https://s3-us-west-2.amazonaws.com/dvc-share/images/iterative_ML_small.png)

[Data Version Control](https://dataversioncontrol.com) or DVC is an open source tool which is designed to help data scientists keep track of their ML processes and file dependencies in the simple form of git-like commands: `dvc run python train_model.py data/train_matrix.p data/model.p`. Your existing ML processes can be easily transformed into reproducible DVC pipelines regardless of which programming language or tool was used.

This DVC introduction walks you through an iterative process of building a machine learning model with DVC using [stackoverflow posts dataset](https://archive.org/details/stackexchange).
The full pipeline could be built by running the bash code below.

```bash
# Download a file and put to data/ directory.
$ dvc import https://s3-us-west-2.amazonaws.com/dvc-share/so/25K/Posts.xml.tgz data/

# Extract XML from the archive.
$ dvc run tar zxf data/Posts.xml.tgz -C data/<commit_msg>Readme rounded image and short descr<commit_after>![DVC icone](https://s3-us-west-2.amazonaws.com/dvc-share/images/favicon_rounded.png)

DVC is an open source tool for data science projects. It orchestrates your Git code, external data files (local drive, S3, GCP Storage) and dependencies to a single reproducibly and shareable environment.

# Introduction

It is hardly possible in real life to develop a good machine learning model in a single pass. ML modeling is an iterative process and it is extremely important to keep track of your steps, dependencies between the steps, dependencies between your code and data files and all code running arguments. This becomes even more important and complicated in a team environment where data scientists’ collaboration takes a serious amount of the team’s effort.

![alt text](https://s3-us-west-2.amazonaws.com/dvc-share/images/iterative_ML_small.png)

[Data Version Control](https://dataversioncontrol.com) or DVC is an open source tool which is designed to help data scientists keep track of their ML processes and file dependencies in the simple form of git-like commands: `dvc run python train_model.py data/train_matrix.p data/model.p`. Your existing ML processes can be easily transformed into reproducible DVC pipelines regardless of which programming language or tool was used.

This DVC introduction walks you through an iterative process of building a machine learning model with DVC using [stackoverflow posts dataset](https://archive.org/details/stackexchange).
The full pipeline could be built by running the bash code below.

```bash
# Download a file and put to data/ directory.
$ dvc import https://s3-us-west-2.amazonaws.com/dvc-share/so/25K/Posts.xml.tgz data/

# Extract XML from the archive.
$ dvc run tar zxf data/Posts.xml.tgz -C data/
{'fraction_non_alphanumeric': 0.06535947712418301, 'fraction_numerical': 0.005115089514066497, 'mean_word_length': 4.6957928802588995, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2289579', 'n_tokens_mistral': 929, 'n_tokens_neox': 895, 'n_words': 452}