<jupyter_start><jupyter_text>Introduction to PythonIn this lesson we will be learning the basics of the Python programminglanguage to accomplish simple tasks. We can use programming to solvecomplex problems. Once we've written a program we can make it operate ondifferent inputs. This allows us to take one solution and apply to manycases. Before we can solve complex problems we need to break it down in tosmaller simpler problems or tasks. Once we've written code to solve eachsimple task, we can put everything together and solve our complex problem.This lesson will start with the basics of Python to give you an idea ofwhat functionality Python can provide. By using and combining these techniqueswe can apply Python programming to our own real world problems to make thecomputer solve our problems quickly and repeatedly. Jupyter Notebooks and JupyterLabThis lesson is stored in a Jupyter Notebook (`.ipynb` file). Notebooks are aspecial type of file that bring text (like this) and code in to the same file.When Notebooks are used in tools like JupyterLab we can see the code, executeit, and see the results. Results can be simple text or with the right toolswe can make images or interactive widgets. By default, a saved Notebook willinclude all of the text, code, and code results. This allows us to easilyexplore or analyze a programming problem and then share it with others.Notebooks can be exported as a PDF, an HTML web page, or even a slide showlike PowerPoint. JupyterLabJupyterLab gives us an integrated environment to work with Notebooks and otherfiles all in one browser window. On the top of the window we have the menu barwith typical File, Edit, and View menus, but also special Notebook-specificmenus like Kernel. On the left we have the file browser where wecan double-click files or folders to open them. When a file is opened itappears in the tabbed area to right. If the file is a simple text format it isusually shown in an editor where you can change the contents of the file andsave it for later use. If a notebook file is opened it is presented in aninteractive viewer. CellsNotebooks are made up of cells. There are two main types of cells: Code andMarkdown. Markdown cells hold text, like this one, that is formattedwith special characters to represent section headers, or bold words, or otherspecial styling. Code cells hold code to be executed. In this notebook everycode cell contains Python code.Code cells can be executed by selecting them (clicking on them) and hitting`Shift` + `Enter` on the keyboard. This will execute the current cell andselect the next cell. Using `Ctrl` + `Enter` will execute the current celland keep it selected. When a code cell is executing it will show `*` in thebrackets to the upper-left of the cell. Code cells will execute in the orderyou run them and do not run in parallel. ModesThere are two main modes in Jupyter Notebooks when working with cells: Editmode and Command mode. In Edit mode you can change the contents of a cell bytyping in the cell. You can enter Edit mode by clicking inside a code cell orhitting "Enter" while in Command mode. There will be a green border around acell when in Edit mode.In Command mode you can perform operations on the cell like deleting the cell,creating a new cell above or below the current cell, etc. In Command modecells will have a blue border. You can enter Command mode from Edit mode byhitting "Esc". Keyboard ShortcutsIn command mode you can use various keyboard shortcuts:* a: Create new cell above current cell* b: Create new cell below current cell* dd: Delete current cellSee the "Help" menu for more information. Resources* Software Carpentry Python Lessons: http://swcarpentry.github.io/python-novice-gapminder/* Dive in to Python: https://www.cmi.ac.in/~madhavan/courses/prog2-2012/docs/diveintopython3/index.html Executing CodeLet's try executing some simple python code in the cells below to get a feelfor how Notebooks can be used. The next cell adds two numbers together.Execute it by selecting it and hitting `Shift` + `Enter` simultaneously onyour keyboard. The result should be shown below the cell.**Exercise****Time: 2 minutes**After executing the cell try editing the cell, change the numbers used orperform another calculation (`-`, `*`, `/`), and rerun the cell. Notice howthe cell number (in brackets to the left) increases with every execution.<jupyter_code>1 + 2<jupyter_output><empty_output><jupyter_text>What's displayed as the result if we put multiple equations in one cell?<jupyter_code>1 + 2
4 + 4<jupyter_output><empty_output><jupyter_text>Variables With Python variables we can store values for later use. Variables arepointers or labels for objects in Python. They allow us to give shorthuman-readable names to values. Update the values below to your ownage and name. We'll use these values later on.<jupyter_code>age = 18
first_name = 'Julie'<jupyter_output><empty_output><jupyter_text>PrintWe can use Python's `print` function to print things to the screen. We cancombine text and numbers by passing them as separate arguments to the`print` function. The text we pass can be provided by wrapping it in singleor double quotes.<jupyter_code>print(age)
print(first_name, "is", age, 'years old')
print('age =', age)<jupyter_output><empty_output><jupyter_text>Variables must be defined before they are used.<jupyter_code>print(last_name)
last_name = 'Smith'<jupyter_output><empty_output><jupyter_text>Notice how it is the order of **execution** that matters for Jupyter cells, not the placement in the notebook. If we rerun the `print` cell above after we've declared `last_name`, we see the last name printed.We can use variables anywhere we would have used its value, like our equations from before:<jupyter_code>age + 3<jupyter_output><empty_output><jupyter_text>We can also change the value a variable:<jupyter_code>age = age + 3
print("Age in 3 years:", age)<jupyter_output><empty_output><jupyter_text>IndexingSo far we've been dealing with values as one single element. However, some types of objects in Python are made up of smaller parts. One example are string objects, the text we've been dealing with so far, which is made up of individual characters. Characters can be thought of as single character strings and that's exactly how Python treats them.<jupyter_code>atom_name = 'helium'
atom_name<jupyter_output><empty_output><jupyter_text>If we want to get the first character of our `atom_name` variable we can passthe **index** between brackets (`[]`) to get the item. Python is a 0-basedindex language which basically means we start counting from 0.<jupyter_code>atom_name[0]<jupyter_output><empty_output><jupyter_text>We can ask for a range of characters by providing multiple indexes separatedby a colon (`:`). This is known as slicing. The first number is our startingindex, the index of the first item we want (inclusive). The second number isthe index **after** the last element we want (exclusive).<jupyter_code>atom_name[1:4]<jupyter_output><empty_output><jupyter_text>**Exercise****Time: 2 minutes**Try change the range of the indexes and see how the results change. What happens if you ask for an index past the end of the string? What happens if you use a negative number?  LengthWe can use the builtin `len` function to see how many elements or charactersare in a string.<jupyter_code>len(atom_name)
print("Your name is", len(first_name), "characters long!")<jupyter_output><empty_output><jupyter_text>TypesEverything in Python is an object, a "thing", of a particular **type**. Sometypes of objects can work together, or be combined, in certain ways and otherscan't. We can check the type of an object by using the builtin `type` function.<jupyter_code>type(1)
type(1.5)
type("abc")
type(atom_name)<jupyter_output><empty_output><jupyter_text>We can usually combine objects of the same type. We saw this before whenadding `3` to our age. We can do something similar with strings.<jupyter_code>"abc" + "d"<jupyter_output><empty_output><jupyter_text>We saw above that the numbers `1` and `1.5` have different types; `int` short for integer and`float` short for floating point number. Even though they are different typesof objects, they both represent numbers and can be combined.<jupyter_code>1 + 1.5<jupyter_output><empty_output><jupyter_text>Other types can't be combined if Python doesn't know what the result should be.<jupyter_code>"abc" + 1<jupyter_output><empty_output><jupyter_text>The above cell gives us our first error from Python. When Python isn't able to complete an operation it tells us by presenting an error, this is called raising an exception. There are different types of exceptions depending on what went wrong. In this case the TypeError and its associated message tell us that the **types** of objects in our operation don't work together. We can try changing the types of objects by using other functions provided by Python. Builtin FunctionsPython comes with a lot of builtin functions for doing common operations.Let's say we wanted to add the number `1` to the end of the string `"abc"`.We could convert the number `1` to a string and then add the two stringslike we did before.<jupyter_code>str(1)
"abc" + str(1)<jupyter_output><empty_output><jupyter_text>A lot of the builtin types in python (`int`, `float`, `str`, etc) havefunctions that can help us convert values.<jupyter_code>int(1.5)
5 + int("1")<jupyter_output><empty_output><jupyter_text>There are also functions for rounding a fractional number to the nearestinteger (`round`). Or to compute the minimum value between two objects (`min`)or the maximum (`max).<jupyter_code>round(1.5)
min(2.5, 2.2)
max(0.3, 2.5, 2.2, 1.8)
max("a", "c", "b")
max("a", 2)<jupyter_output><empty_output><jupyter_text>Just like with addition (`+`) there are some operations that Python'sfunctions know how to handle and others that it doesn't. Comparing astring to a number is one of those cases.  Builtin typesSo far we've dealt with integers (`int`), floating point numbers (`float`),and strings (`str`). There are other common types that Python provides usto organize are data. We'll cover lists and dictionaries. Other types like`set` and `tuple` are left as an exercise for the reader. ListsIf we want to have a series of objects in Python, one option we have are lists.We can create a list by using square brackets (`[]`). We can use a list in alot of the same ways we used strings.<jupyter_code>a = [1, 2, 3]
a
type(a)
b = []
type(b)
a[0]
a[1:3]
len(a)
max(a)<jupyter_output><empty_output><jupyter_text>We can add more values to a list too. We can either use `+` to "add" a listto another list.<jupyter_code>a + [5]
a<jupyter_output><empty_output><jupyter_text>We can also use a special method called `append` to add individual elementsto the list.<jupyter_code>a.append(6)
a<jupyter_output><empty_output><jupyter_text>Note how `+` created a **new** list; one where `5` was on the end. When weused `append` the `a` list was modified **in place**, meaning `a` is nowdifferent than it was before. In the `+` case, `a` remained the same afterthe operation.We can modify individual elements in a list too.<jupyter_code>a[1] = 10
a<jupyter_output><empty_output><jupyter_text>DictionariesAnother very useful builtin type are dictionaries (`dict`). They allow us tomap a "key" to a "value". Let's create a dictionary mapping peoples names totheir ages.<jupyter_code>ages = {'Julie': 18, 'David': 29}
ages<jupyter_output><empty_output><jupyter_text>To access the "value" for a particular "key" we can use the bracket syntaxlike we did when indexing in a list or string. Instead of an index numberwe use the "key".<jupyter_code>ages['Julie']<jupyter_output><empty_output><jupyter_text>Similar to lists we can add more key/value **pairs** to our dictionary.<jupyter_code>ages['Anne'] = 17<jupyter_output><empty_output><jupyter_text>CommentsMany programming languages allow developers to provide extra information next to their code to describe what's going on in a more human friendly way. These are typically called **comments**. Comments don't *do* anything to the execution of the code, only add information for the reader of the code. In Python we can provide them by using the `` character.<jupyter_code>number_of_guests = 1 + 2  # Billy plus his two parents

# Calculate size of the table needed (inches)
length = number_of_guests * 24
length<jupyter_output><empty_output><jupyter_text>Arrays with NumPySo far we've been using the builtin functionality of Python. If there issomething Python can't do out of the box, you can often find third-party**libraries** with extra functionality that can be installed and used.Libraries are collections of python code that can provide additionaltypes of objects or functions to do more than the base Python installation.For example, in scientific programming you often have to perform calculationson millions of numbers and quickly. Due to some low-level details of howPython does things, trying to perform these calculations with numbers in aPython list would not be efficient (memory or processing time). The mostpopular library for detailing with large lists of numbers or arrays is`NumPy`.Below we'll go over the basics of NumPy arrays and how we can perform largecalculations quickly. First, we'll use a new Python concept called `import`to bring NumPy functionality to our Notebook. The below line will importthe `numpy` library and assign it to a variable named `np`.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Once we have access to NumPy we can start using the types and functionsprovided by it. Let's start by creating an array object.<jupyter_code>a = np.array([1.2, 2.5, 3.2])
a<jupyter_output><empty_output><jupyter_text>NumPy arrays can do a lot of mathematical operations just like the integersand floats we used before.<jupyter_code>a + 5
b = a * 2
a + b<jupyter_output><empty_output><jupyter_text>We can also use arrays a lot like lists.<jupyter_code>a[0]<jupyter_output><empty_output><jupyter_text>NumPy arrays have their own set of functions and properties that we caninvestigate. For example `.shape` tells us how many elements are in thearray.<jupyter_code>a.shape<jupyter_output><empty_output><jupyter_text>The `.sum` function will add up all of the numbers in the array and returnthe result.<jupyter_code>a.sum()<jupyter_output><empty_output><jupyter_text>But this is just with three numbers. If we were dealing with real data wewould most likely have millions of values. Let's use the `random` functionfrom NumPy to generate a lot of random numbers.<jupyter_code>r = np.random.random([1000000])
r
r.shape
r.sum()
r2 = r / 2
r2.sum()<jupyter_output><empty_output><jupyter_text>We can also use NumPy's `arange` function to generate a sequence of numbers:<jupyter_code>x = np.arange(1000000)
x<jupyter_output><empty_output><jupyter_text>There are many other operations that NumPy can perform quickly and manyother ways to generate NumPy arrays. Learning about these is leftas an exercise for the reader.  Plotting with MatplotlibSo far we've been dealing with code and print outs. It is often much easierto see what data looks like in a plot or an image. Python doesn't come witha lot of graphing functionality out of the box, but we can use the verypopular `matplotlib` library to plot some data values. Due to NumPy'spopularity and ease of use, matplotlib can plot data stored in NumPy arrays.Let's create some X/Y plots of some basic formulas. First we have to createthe arrays we are going to plot.<jupyter_code>x = np.arange(100)
y = x<jupyter_output><empty_output><jupyter_text>Next we import some code from the `matplotlib` library. We also use a magic`%matplotlib` function to tell JupyterLab how to display our plots insidethe Notebook.<jupyter_code>%matplotlib widget
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Now we can tell matplotlib that we want to plot our array data by firstcreating a figure, a place to plot the data, and then we call `plot` tocreate the plot. The plot is displayed in an interactive widget belowthe cell.<jupyter_code>plt.figure()
plt.plot(x, y)<jupyter_output><empty_output><jupyter_text>In addition to the `x` and `y` arguments, we can also set labels for the X and Y axis by calling `plt.xlabel("X label")` and `plt.ylabel("Y label")` functions after `plt.figure`. We can provide a title for the plot by calling `plt.title("Title")`.We can change some of the properties of the plot like the color of the line by passing a keyword argument `color='red'` to the `plt.plot` function. A keyword argument or named argument is one where we specifically say what argument we are providing. This differs from positional arguments like the `x` and `y` which must be in the correct order when calling the function. Another keyword argument is `linestyle` which can be `'-'` (default), `'--'`, `'-.'`, or `':'`.**Exercise****Time: 5 minutes**Change the equation (the value) for `y` to change what the line looks like in the plot above. Try setting your own labels or changing the style of the plot using some of the keyword arguments mentioned above. 2D ArraysWe've been using 1 dimensional (1D) arrays of data so far, but NumPy arrayscan also be multi-dimensional. The easiest multi-dimensional array we can makeis a 2D array. Let's create a 2D array of random points using NumPy.<jupyter_code>noise = np.random.random([600, 800])
noise.shape<jupyter_output><empty_output><jupyter_text>Notice how the shape of the array is different than the 1D array. The shape istelling us there are 600 rows of data in the first dimension and 800 columnsin the second dimension. We can still perform calculations on 2D arrays justlike 1D arrays.<jupyter_code>noise = noise / 4 + 1<jupyter_output><empty_output><jupyter_text>2D arrays can usually be visualized as images. The`matplotlib` library let's us plot images by using the `imshow` function.First, let's make some data to plot. We can generate asequence of numbers like we did early by calling `arange`. We then use anew method to "reshape" the array. So instead of having `600 * 800` elements,now it will have 600 rows and 800 columns, just like our `noise` variable.<jupyter_code>z = np.arange(600 * 800)
z = z.reshape([600, 800])
plt.figure()
plt.imshow(z * noise)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06948080279232112, 'fraction_numerical': 0.00856239092495637, 'mean_word_length': 5.420518207282913, 'pattern_counts': {'":': 0, '<': 161, '<?xml version=': 0, '>': 161, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '40', 'dir': 'jupyter-structured-clean-dedup', 'id': '518758', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 5292, 'n_tokens_neox': 5021, 'n_words': 2535, 'path': 'notebooks/01_intro_to_python.ipynb', 'repo_name': 'djhoese/tutorial-intro-python-satpy', 'size': '18336'}