<commit_before>Network Analysis on the Big Cancer Genome Data
==============================================
@author <NAME> (<EMAIL>) <NAME> (<EMAIL>) 

The package we provided here contains the toolkit for performing two types of MapReduce-based concordance index evaluation in Hadoop environment and the HBase tools for data importing and exporting, as well as the R script files for preprocessing, feature analysis and Bayesian inference. 

For the treatment engine, we include the source code of the engine and the implementation of the entire web service. The detailed instruction and screenshots of those five parts of our package is available as the following.

We provide the final report on the algorithms and programs we created for this project [here](https://github.com/Sapphirine/Network-Analysis-on-the-Big-Cancer-Genome-Data/blob/master/BigData_final_report_v6_201412-4.pdf "final report"). and the demo video at [YouTube](https://www.youtube.com/watch?v=kcP1PLSFb9o "Demo video").

##1	The Hadoop-based concordance index (CI) toolkit
The source code and the executable jar is located at ./CI/

  1.1	Compute the CI of the genes in TCGA data set 

    1.1.1	Download the pancan12 gene expression matrix from http://192.168.3.11/html/pancan12.ge.rda
and make sure Hadoop HDFS is properly running.

    1.1.2	Run ci_preprocess.R in R to convert the features into CSV files.

    source("ci_preprocess.R")

    1.1.3	(Optional) Compile the java source code

    ant compile jar

    1.1.4	Convert the CSV files into SeqenceFiles.<commit_msg>link to the website<commit_after>Network Analysis on the Big Cancer Genome Data
==============================================
@author <NAME> (<EMAIL>) <NAME> (<EMAIL>) 

The package we provided here contains the toolkit for performing two types of MapReduce-based concordance index evaluation in Hadoop environment and the HBase tools for data importing and exporting, as well as the R script files for preprocessing, feature analysis and Bayesian inference. 

For the treatment engine, we include the source code of the engine and the implementation of the entire web service ([live demo](http://192.168.127.12/ "demo web site")). The detailed instruction and screenshots of those five parts of our package is available as the following.

We provide the final report on the algorithms and programs we created for this project [here](https://github.com/Sapphirine/Network-Analysis-on-the-Big-Cancer-Genome-Data/blob/master/BigData_final_report_v6_201412-4.pdf "final report"). and the demo video at [YouTube](https://www.youtube.com/watch?v=kcP1PLSFb9o "demo video").

##1	The Hadoop-based concordance index (CI) toolkit
The source code and the executable jar is located at ./CI/

  1.1	Compute the CI of the genes in TCGA data set 

    1.1.1	Download the pancan12 gene expression matrix from http://192.168.3.11/html/pancan12.ge.rda
and make sure Hadoop HDFS is properly running.

    1.1.2	Run ci_preprocess.R in R to convert the features into CSV files.

    source("ci_preprocess.R")

    1.1.3	(Optional) Compile the java source code

    ant compile jar

    1.1.4	Convert the CSV files into SeqenceFiles.
{'fraction_non_alphanumeric': 0.09759949463044852, 'fraction_numerical': 0.027479469361970942, 'mean_word_length': 4.322689075630252, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 3, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4832387', 'n_tokens_mistral': 953, 'n_tokens_neox': 859, 'n_words': 397}