<jupyter_start><jupyter_text>Step 1: Sign up / get assigned EC2 instance Step 2: SSH into EC2 instance Step 3: Start up the cluster by running the following commands<jupyter_code>cd ~/nexus/esip-workshop/docker/infrastructure
docker-compose up -d cassandra1
docker logs -f cassandra1<jupyter_output><empty_output><jupyter_text>Step 4: Wait to see "INFO  01:57:02 Starting listening for CQL clients on /0.0.0.0:9042…" and then run<jupyter_code>docker-compose up -d<jupyter_output><empty_output><jupyter_text>Step 5: Verify services are running using<jupyter_code>docker ps<jupyter_output><empty_output><jupyter_text>There should be 9 containers running: 3 solr, 3 zookeeper, and 3 cassandra. If they are not all running, run docker-compose up -d again. Step 6: Start the analysis cluster<jupyter_code>cd ~/nexus/esip-workshop/docker/analysis
docker-compose up -d<jupyter_output><empty_output><jupyter_text>Step 7: Verify services are running using<jupyter_code>docker ps<jupyter_output><empty_output><jupyter_text>There should be an additional 5 containers running: 3 mesos-agent, 1 mesos-master, 1 nexus-webapp Step 8: Start the ingestion cluster<jupyter_code>cd ~/nexus/esip-workshop/docker/ingest
docker-compose up -d<jupyter_output><empty_output><jupyter_text>Step 9: Verify services are running using<jupyter_code>docker ps<jupyter_output><empty_output><jupyter_text>There should be an additional 9 containers running: 3 kafka, 3 xd-container, 1 redis, 1 mysqldb, 1 xd-admin Ingest 2017 AVHRR data that has been staged under /home/ndeploy/ingest/data/avhrr/2017 Step 10: Deploy the stream to Spring XD using nx-deploy-stream.sh available inside xd-admin container<jupyter_code>docker exec -it xd-admin /usr/local/nx-deploy-stream.sh --datasetName AVHRR_OI_L4_GHRSST_NCEI --dataDirectory /usr/local/data/nexus/avhrr/2017 --variableName analysed_sst --tilesDesired 1296<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08784850078905838, 'fraction_numerical': 0.03156233561283535, 'mean_word_length': 6.488188976377953, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '9649', 'license': 'Apache-2.0', 'n_tokens_mistral': 720, 'n_tokens_neox': 653, 'n_words': 182, 'path': 'esip-workshop/other-notebooks/Nexus Deployment and Ingestion.ipynb', 'repo_name': 'dataplumber/nexus', 'size': '1901'}