<commit_before>## UC Berkeley's Statistics 159/259
### Project Group Beta, Fall Term 2015 

[![Build Status](https://travis-ci.org/berkeley-stat159/project-beta.svg?branch=master)](https://travis-ci.org/berkeley-stat159/project-beta) 
[![Coverage Status](https://coveralls.io/repos/berkeley-stat159/project-beta/badge.svg?branch=master&service=github)](https://coveralls.io/github/berkeley-stat159/project-beta?branch=master)

_**Topic:**_ [Modeling of Semantic Representation in the Brain Using fMRI Response] (https://openfmri.org/dataset/ds000113)
_**Group members:**_ <NAME>, <NAME>, <NAME>, <NAME> & <NAME>


## Roadmap

#### Recommended Steps
1. make data -  downloads the data for analysis
2. make validate - ensure the data is not corrupted
3. make coverage - runs coverage tests and generates the Travis coverage report
4. make test - runs all of the tests for scripts 
5. make eda - generate figures from exploratory analysis
6. make analysis - generate figures and results
7. make report - build final report

#### Data
- `make data_process`: Download a 5GB ds013-subject12 data for our analysis from Googledrive. It is a compressed file that contains 8 bold_dico.nii.gz from 8runs. We then rename it with the correct name. Data is then uncompressed.

#### Validate
- `make validate`: Validates the ds013-subject12 data by checking the right named file is present and has the correct hash. 

#### Tests 
- `make test`: Runs all the tests for this project repository. The test functions are located in code/utils/tests

#### Coverage 
- `make coverage`: Runs the coverage tests and then generates a coverage report.

#### EDA
- `make eda`: Runs all the scripts that are used for exploratory data analysis.

#### Analysis
- `make analysis`: Runs all the scripts that are used for Analysis including Ridge Regression Analysis and Neural Network Analysis

#### Report
- `make report`: Build final report<commit_msg>file discussions added<commit_after># Project Beta
## UC Berkeley's Statistics 159/259
### Project Group Beta, Fall Term 2015 

[![Build Status](https://travis-ci.org/berkeley-stat159/project-beta.svg?branch=master)](https://travis-ci.org/berkeley-stat159/project-beta) 
[![Coverage Status](https://coveralls.io/repos/berkeley-stat159/project-beta/badge.svg?branch=master&service=github)](https://coveralls.io/github/berkeley-stat159/project-beta?branch=master)

_**Topic:**_ [Modeling of Semantic Representation in the Brain Using fMRI Response] (https://openfmri.org/dataset/ds000113)
_**Group members:**_ <NAME>, <NAME>, <NAME>, <NAME> & <NAME>


## Roadmap

#### Repository Navigation Overview 
1. make data -  downloads the data for analysis
2. make validate - ensure the data is not corrupted
3. make coverage - runs coverage tests and generates the Travis coverage report
4. make test - runs all of the tests for scripts 
5. make eda - generate figures from exploratory analysis
6. make analysis - generate figures and results
7. make report - build final report

#### Data
- `make data_process`: Download a 5GB ds013-subject12 data for our analysis from Googledrive. It is a compressed file that contains 8 bold_dico.nii.gz from 8runs. We then rename it with the correct name. Data is then uncompressed.

#### Validate
- `make validate`: Validates the ds013-subject12 data by checking the right named file is present and has the correct hash. 

#### Tests 
- `make test`: Runs all the tests for this project repository. The test functions are located in code/utils/tests

#### Coverage 
- `make coverage`: Runs the coverage tests and then generates a coverage report.

#### EDA
- `make eda`: Runs all the scripts that are used for exploratory data analysis.

#### Analysis
- `make analysis`: Runs all the scripts that are used for Analysis including Ridge Regression Analysis and Neural Network Analysis

#### Report
- `make report`: Build final report

## Discussion on Reproducibility 
All files mentioned below are located in the /code/ directory. 
The files are grouped by their functionalities. 

### 1. Download Data 
- data_loading_script.py 

### 2. Data Preprocessing 
- dataprep_script.py 
- mask_generating.py 
- filter_script.py 
- mask_generating.py 

### 3. Preprocess Description
- dataclean.py 
- gen_design_matrix.py 

### 4. Analysis 
- ridge regression 
	- description_modling_ridge_regression.py 
- k-nearest neighbors 
	- scenes_pred.py 
- neural network 
	- nn.py 

## Team Members  
- <NAME> ([([`raj4`](https://github.com/raj4))
- <NAME> ([([`yuchengdong`](https://github.com/yuchengdong))
- <NAME> ([([`cxmo`](https://github.com/cxmo))
- <NAME> ([([`rishizsinha`](https://github.com/rishizsinha))
- <NAME> ([([`ariaaay`](https://github.com/ariaaay))

{'fraction_non_alphanumeric': 0.11455897980871413, 'fraction_numerical': 0.021679064824654624, 'mean_word_length': 4.440462427745665, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 15, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6998337', 'n_tokens_mistral': 1488, 'n_tokens_neox': 1342, 'n_words': 511}