<jupyter_start><jupyter_text>Installing JupyterTo install Jupyter please run the following commands:``` Ensure pip is up to date$ pip3 install --upgrade pip Install jupyter$ pip3 install jupyter Run jupyter$ jupyter-notebook```That will dump out some server status stuff similar to this:```[I 09:04:18.118 NotebookApp] Writing notebook server cookie secret to /run/user/1000/jupyter/notebook_cookie_secret[I 09:04:18.316 NotebookApp] Serving notebooks from local directory: /home/riley/Documents/DataScience[I 09:04:18.316 NotebookApp] 0 active kernels[I 09:04:18.316 NotebookApp] The Jupyter Notebook is running at:[I 09:04:18.316 NotebookApp] http://localhost:8888/?token=60cff<PASSWORD>[I 09:04:18.316 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).```At the same time, it should have opened a browser tab with what looks like a file browser. In the top right corner, click `New` this will create a new Jupyter doc. (To run a premade one just select the file in the jupyter file browser.Congratulations! You have now installed Jupyter NoteBooks  Basic PythonNow that we have a solid workbench, lets get some tools. Back in the command line, run the following:```$ pip3 install numpy       Super Efficient Math$ pip3 install scipy       More complicated Super Efficient Math$ pip3 install pandas      You probably already have this, but super handy for manupulating data$ pip3 install matplotlib  Plots data```Sweet, now that we have some tools lets start coding...  Variable Declarations<jupyter_code># Python doesn't technically need variables to run, but to do anything useful on our data we do. 
2+2

# To create a varaible, like in most languages, use the following syntax
variable_name = 2 + 2<jupyter_output><empty_output><jupyter_text>A note on variables: A variable can have a short name (like x and y) or a more descriptive name (age, carname, total_volume). Rules for Python variables:- A variable name must start with a letter or the underscore character- A variable name cannot start with a number- A variable name can only contain alpha-numeric characters and underscores (A-z, 0-9, and _ )- Variable names are case-sensitive (age, Age and AGE are three different variables)<jupyter_code># Some other cool things python can do with variables
x, y = 2+2 , 5
print (x, y)
x = y = 3 + 3
print(x, y)<jupyter_output>4 5
6 6<jupyter_text>Data Types.More often than not, we will not be doing simple 2 + 2 operations. Our work will include large tables and more variables than I will ever want to keep track of. So we need a more robust way to store data. Here are a few options:  ListsLists, in their most basic form act as 1D arrays with the ability to add, and remove anything we want<jupyter_code>example_list = [] # Declare and empty list. 
example_list.append("This Orange")
example_list.append(4)
example_list.append("Tastes Weird")

print(example_list[1])

print(example_list)

example_list.remove(4)
print(example_list)<jupyter_output>4
['This Orange', 4, 'Tastes Weird']
['This Orange', 'Tastes Weird']<jupyter_text>DictionariesDictionaries act similarly to lists with one key difference, each item in the dict is stored as a key - value pair.Meaning that if I have a dictionary that looks like this:```{    'data': 1234,    'body': "Hello World"}```We can access the body message by calling `dict['body']`.Below is a more explicit example:<jupyter_code>dictionary = {
    'date': "1/23/2020",
    'code': 404,
    'body': "Could not find my motivation"
}

dictionary['date'] = "2/17/2020"
print("Date:", dictionary['date'])
print("Error Code:", dictionary['code'])
print("Message:", dictionary['body'])<jupyter_output>Date: 2/17/2020
Error Code: 404
Message: Could not find my motivation<jupyter_text>Conditionals<jupyter_code>if x is y:
    print("X is equal to Y")
else:
    print("X is not equal to Y")
if x is not y:
    print("X is not equal to Y")
else:
    print("X is equal to Y, again")
if x is 3:
    print("X equals 3")
elif x is 6:
    print("X equals 6")
else:
    print("X is ", x)
    
# Logical Operators (AND, OR)
if x == 6 and y == 6:
    print("X and Y are 6")<jupyter_output>X and Y are 6<jupyter_text>Basic Packages for Data ScienceSince there are literally thousands of packages that can be used for our applications, I chose a few below that I use frequently. When it comes to the real world, everyone has a different favorite tool and package they like to use so take my recommendations with a grain of salt and go find packages that fit your work flow later.  PandasWe have used this before so this will kind of be a flying overview of the package:<jupyter_code>import pandas as pd
#import matplotlib as plt
# Grabbing the dataset
csv_path = 'housing.csv'
data = pd.read_csv(csv_path)
data.describe()


data.hist(bins=50, figsize=(20,15))
# IDK how clear it is to everyone else but the plot is showing the distribution of values within each column fr
# in the data frame.
# Show correlations between cols
corr_matrix = data.corr()
print(corr_matrix)<jupyter_output>longitude  latitude  housing_median_age  total_rooms  \
longitude            1.000000 -0.924664           -0.108197     0.044568   
latitude            -0.924664  1.000000            0.011173    -0.036100   
housing_median_age  -0.108197  0.011173            1.000000    -0.361262   
total_rooms          0.044568 -0.036100           -0.361262     1.000000   
total_bedrooms       0.069608 -0.066983           -0.320451     0.930380   
population           0.099773 -0.108785           -0.296244     0.857126   
households           0.055310 -0.071035           -0.302916     0.918484   
median_income       -0.015176 -0.079809           -0.119034     0.198050   
median_house_value  -0.045967 -0.144160            0.105623     0.134153   

                    total_bedrooms  population  households  median_income  \
longitude                 0.069608    0.099773    0.055310      -0.015176   
latitude                 -0.066983   -0.108785   -0.071035      -0.079809   
housing_[...]<jupyter_text>NumPyThis package is crazy dense so I will not go too deep into it but I will touch on the big differences. **Google is your friend with NumPy**.NumPy has datatypes called ndArrays. These tend to act very similarly to a tradtional python array but with one key difference. NumPy arrays already have code written for them that make Linear Algebra (kind of an important thing in data science) really easy.<jupyter_code>import numpy as np

X = np.arange(9) + 1
Y = np.arange(9) + 4

print(X, Y)
X = X.reshape((3,3))
Y = Y.reshape((3,3))
print(X)
print(Y)
Z = X * Y # Element wise multiplication
print(Z)
Z = X.dot(Y) # True Matrix Multiplication
print(Z)
U, Sig, V = np.linalg.svd(Z)
print("Eigen Vectors:", U, "\n")
print('Eigen Values:',Sig, "\n")
print("V':", V, "\n")<jupyter_output>Eigen Vectors: [[-0.22407046  0.88494393  0.40824829]
 [-0.5231604   0.24420592 -0.81649658]
 [-0.82225034 -0.3965321   0.40824829]] 

Eigen Values: [4.19119795e+02 7.73048670e-01 1.46710407e-15] 

V': [[-0.505578   -0.57459367 -0.64360935]
 [ 0.76008172  0.05635107 -0.64737959]
 [ 0.40824829 -0.81649658  0.40824829]]
{'fraction_non_alphanumeric': 0.09098520792631873, 'fraction_numerical': 0.09014792073681273, 'mean_word_length': 3.641839378238342, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '580514', 'license': 'MIT', 'n_tokens_mistral': 2695, 'n_tokens_neox': 2278, 'n_words': 898, 'path': 'Installation_And_Basic_Python.ipynb', 'repo_name': 'VintageDesign/DataScienceTutorial', 'size': '7199'}