<commit_before>#CDH 5 pseudo-distributed cluster Docker image

Do you develop Hadoop mapreduce applications on top of Cloudera distribution? This docker image can help you. It contains basic CDH 5 setup with YARN. You can use it for developmeent and verification of your code in local environment without messing up your system with Hadoop instalation.

Docker image was prepared according to [Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode](http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Quick-Start/cdh5qs_yarn_pseudo.html) with a few adjustments for Docker environment.

#####Installed services
* HDFS
* YARN
* JobHistoryServer
* Oozie
* Hue

###Execution
Get docker image

    docker pull chalimartines/cdh5-pseudo-distributed

Run image with specified port mapping

    docker run --name cdh -d -p 8020:8020 -p 50070:50070 -p 50010:50010 -p 50020:50020 -p 50075:50075 -p 8030:8030 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8088:8088 -p 8040:8040 -p 8042:8042 -p 10020:10020 -p 19888:19888 -p 11000:11000 -p 8888:8888 chalimartines/cdh5-pseudo-distributed

 Or you can use docker-compose configuration from [here](https://github.com/chali/cdh5-pseudo-distributed-cluster-docker-compose)
  
If you are Mac OS user with boot2docker and you would like to get from your local system to a cdh container add these port forwardings

	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8020,tcp,,8020,,8020"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50070,tcp,,50070,,50070"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50010,tcp,,50010,,50010"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50020,tcp,,50020,,50020"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50075,tcp,,50075,,50075"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8030,tcp,,8030,,8030"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8031,tcp,,8031,,8031"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8032,tcp,,8032,,8032"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8033,tcp,,8033,,8033"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8088,tcp,,8088,,8088"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8040,tcp,,8040,,8040"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8042,tcp,,8042,,8042"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port10020,tcp,,10020,,10020"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port19888,tcp,,19888,,19888"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port11000,tcp,,11000,,11000"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8888,tcp,,8888,,8888"
    VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port9999,tcp,,9999,,9999"

###UI entry points
Those urls consider port forwarding from localhost.<commit_msg>Add port publishing doc for custom port<commit_after>#CDH 5 pseudo-distributed cluster Docker image

Do you develop Hadoop mapreduce applications on top of Cloudera distribution? This docker image can help you. It contains basic CDH 5 setup with YARN. You can use it for developmeent and verification of your code in local environment without messing up your system with Hadoop instalation.

Docker image was prepared according to [Installing CDH 5 with YARN on a Single Linux Node in Pseudo-distributed mode](http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Quick-Start/cdh5qs_yarn_pseudo.html) with a few adjustments for Docker environment.

#####Installed services
* HDFS
* YARN
* JobHistoryServer
* Oozie
* Hue

###Execution
Get docker image

    docker pull chalimartines/cdh5-pseudo-distributed

Run image with specified port mapping

    docker run --name cdh -d -p 8020:8020 -p 50070:50070 -p 50010:50010 -p 50020:50020 -p 50075:50075 -p 8030:8030 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8088:8088 -p 8040:8040 -p 8042:8042 -p 10020:10020 -p 19888:19888 -p 11000:11000 -p 8888:8888 -p 9999:9999 chalimartines/cdh5-pseudo-distributed

 Or you can use docker-compose configuration from [here](https://github.com/chali/cdh5-pseudo-distributed-cluster-docker-compose)
  
If you are Mac OS user with boot2docker and you would like to get from your local system to a cdh container add these port forwardings

	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8020,tcp,,8020,,8020"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50070,tcp,,50070,,50070"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50010,tcp,,50010,,50010"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50020,tcp,,50020,,50020"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port50075,tcp,,50075,,50075"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8030,tcp,,8030,,8030"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8031,tcp,,8031,,8031"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8032,tcp,,8032,,8032"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8033,tcp,,8033,,8033"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8088,tcp,,8088,,8088"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8040,tcp,,8040,,8040"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8042,tcp,,8042,,8042"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port10020,tcp,,10020,,10020"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port19888,tcp,,19888,,19888"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port11000,tcp,,11000,,11000"
	VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port8888,tcp,,8888,,8888"
    VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port9999,tcp,,9999,,9999"

###UI entry points
Those urls consider port forwarding from localhost.
{'fraction_non_alphanumeric': 0.11526972412581268, 'fraction_numerical': 0.145844315586013, 'mean_word_length': 5.40990990990991, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2665823', 'n_tokens_mistral': 2632, 'n_tokens_neox': 2085, 'n_words': 539}