<commit_before># FATE Pipeline

Pipeline is a high-level python API that allows user to design, start, and query FATE jobs in a sequential manner. 
FATE Pipeline is designed to be user-friendly and consistent in behavior with FATE command line tools. 
User can customize job workflow by adding components to pipeline and then initiate a job with one call. 
In addition, Pipeline provides functionality to run prediction and query information after fitting a pipeline.
Run the [mini demo](./demo/pipeline-mini-demo.py) to have a taste of how FATE Pipeline works.
Default values of party ids and work mode in [config file](./demo/config.yaml) may need to be modified depending on setting.

```bash
python pipeline-mini-demo.py
```

For more pipeline demo, please refer to [demos](./demo).

## A FATE Job is A Sequence

A FATE job includes a sequence of tasks. FATE pipeline provides easy-to-use tools to configure order and setting of the tasks. 

FATE is written in a modular style. Modules are designed to have input and output data and model. <commit_msg>edit demo<commit_after># FATE Pipeline

Pipeline is a high-level python API that allows user to design, start, and query FATE jobs in a sequential manner. 
FATE Pipeline is designed to be user-friendly and consistent in behavior with FATE command line tools. 
User can customize job workflow by adding components to pipeline and then initiate a job with one call. 
In addition, Pipeline provides functionality to run prediction and query information after fitting a pipeline.
Run the [mini demo](./demo/pipeline-mini-demo.py) to have a taste of how FATE Pipeline works.
Default values of party ids and work mode in [config file](./demo/config.yaml) may need to be modified depending on setting.

```bash
python pipeline-mini-demo.py config.yaml
```

For more pipeline demo, please refer to [demos](./demo).

## A FATE Job is A Sequence

A FATE job includes a sequence of tasks. FATE pipeline provides easy-to-use tools to configure order and setting of the tasks. 

FATE is written in a modular style. Modules are designed to have input and output data and model. 
{'fraction_non_alphanumeric': 0.05518867924528302, 'fraction_numerical': 0.0, 'mean_word_length': 4.198529411764706, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1732152', 'n_tokens_mistral': 558, 'n_tokens_neox': 533, 'n_words': 322}