<commit_before># BrakeValve Dataflow
[![Build Status](https://travis-ci.org/BrakeValve/dataflow.svg?branch=master)](https://travis-ci.org/BrakeValve/dataflow)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](https://en.wikipedia.org/wiki/MIT_License)
> Help you think twice and make wise choices on Steam

BrakeValve is a Chrome extension to help Steam buyers making decision on purchasing games. We use novel feature extraction techniques to gain knowledge and insight from the Steam historical data and combined with a robust predictor to achieve over 80% accuracy and F-score.

Click [here](https://brakevalve.github.io/) to see more implementation detail.

## Project Preview

``` markdown
dataflow/
│ 
├── data-crawler/                        
│   ├── historical-price-crawler.py      # Steam price crawler using SteamDB API
│   └── metadata-crawler.py              # Steam metadata crawler using SteamDB API
├── data-preprocessor/                   
│   ├── Game.py                          # Steam Game Object
│   ├── listFile.py                      # file utils
│   └── preprocessor.py                  # preprocessing main fuction
└─── model/
    └── random-forest-model.py           # training demo using random forest model from sklearn
```
## Dependencies

- beautifulsoup4
- requests<commit_msg>Fix typo in README.md (#11)<commit_after># BrakeValve Dataflow
[![Build Status](https://travis-ci.org/BrakeValve/dataflow.svg?branch=master)](https://travis-ci.org/BrakeValve/dataflow)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](https://en.wikipedia.org/wiki/MIT_License)
> Help you think twice and make wise choices on Steam

BrakeValve is a Chrome extension to help Steam buyers making decision on purchasing games. We use novel feature extraction techniques to gain knowledge and insight from the Steam historical data and combined with a robust predictor to achieve over 80% accuracy and F-score.

Click [here](https://brakevalve.github.io/) to see more implementation detail.

## Project Preview

``` markdown
dataflow/
│ 
├── data-crawler/                        
│   ├── historical-price-crawler.py      # Steam price crawler using SteamDB API
│   └── metadata-crawler.py              # Steam metadata crawler using SteamDB API
├── data-preprocessor/                   
│   ├── Game.py                          # Steam Game Object
│   ├── listFile.py                      # file utils
│   └── preprocessor.py                  # preprocessing main function
└─── model/
    └── random-forest-model.py           # training demo using random forest model from `sklearn`
```
## Dependencies

- beautifulsoup4
- requests
{'fraction_non_alphanumeric': 0.10774410774410774, 'fraction_numerical': 0.0029928918817807705, 'mean_word_length': 2.809116809116809, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5138311', 'n_tokens_mistral': 788, 'n_tokens_neox': 764, 'n_words': 255}