<jupyter_start><jupyter_text>First Steps  Test 0<jupyter_code>print("If this does not work, maybe we should start few steps earlier...")
!pip install findspark<jupyter_output>Collecting findspark
  Downloading findspark-1.4.2-py2.py3-none-any.whl (4.2 kB)
Installing collected packages: findspark
Successfully installed findspark-1.4.2<jupyter_text>Install everything If you like windows...https://bigdata-madesimple.com/guide-to-install-spark-and-use-pyspark-from-jupyter-in-windows/If you don't :'( ...Option A:https://medium.com/@GalarnykMichael/install-spark-on-ubuntu-pyspark-231c45677de0Option B:https://www.roseindia.net/bigdata/pyspark/install-pyspark-on-ubuntu.shtmlLet's make sure we all have the same vesion: **spark-3.0.0   hadoop2.7**  Test 1<jupyter_code>import findspark

findspark.init()<jupyter_output><empty_output><jupyter_text>Test 2<jupyter_code>import pyspark

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

df = spark.sql("select 'spark' as hello ")

df.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.125, 'fraction_numerical': 0.025763358778625955, 'mean_word_length': 5.947019867549669, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '387400', 'license': 'MIT', 'n_tokens_mistral': 415, 'n_tokens_neox': 376, 'n_words': 68, 'path': 'RealTimeDataAnalysis/Lecture1/First Steps.ipynb', 'repo_name': 'robretoarenal/BTS2', 'size': '1048'}