<jupyter_start><jupyter_text>Introduction to PythonSo you want to learn Python? Good choice!This notebook will lead you through some of the basics of programming in **Python 3**.It includes many references to the (freely available) online book [Think Python](https://greenteapress.com/wp/think-python-2e/) by <NAME>.  The basics of a computer programPrograms are made up of (1) data and (2) operations on that data.You can think of it like a cooking recipe, which has ingredients and steps that do things to those ingredients.For example, consider this very simple program:(you can run the cell below by clicking the play button next to it, or by clicking within the cell and pressing `Ctrl + Enter`)<jupyter_code>1 + 1<jupyter_output><empty_output><jupyter_text>The cell above adds (the operation) two numbers (the data).There are all kinds of arithmetic that we can do using Python. For example, see if you can figure out what the following cells are doing:<jupyter_code>43 - 1
6 * 7
84 / 2<jupyter_output><empty_output><jupyter_text>You will note that the third expression returns a decimal, even though the original inputs were both integers (whole numbers). Let's take a look at why.We can ask Python to tell us the difference between these two types of data using `type()`:<jupyter_code>type(42)
type(42.0)<jupyter_output><empty_output><jupyter_text>`int` stands for integer, whereas float stands for floating-point number (i.e. a decimal number).(Python distinguishes between them because it takes more memory to store a `float` than an `int`, and because integers can be more precise). There's another type of data that we often want to use: **strings**, which are a series of letters strung together.We indicate that something is a string by putting it between quotation marks:<jupyter_code>"hello"
type("hello")<jupyter_output><empty_output><jupyter_text>**Exercise**What are the types of `"42.0"` and `"42"` ?<jupyter_code># Use this cell for your answer<jupyter_output><empty_output><jupyter_text>VariablesWhen we run a line of code like`2 + 2`the expression is evaluated and displayed, but the result of the evaluation is then discarded.However, we often want to save the result of some code, so that we can use it in the future.To do that, we need to **assign** it to a **variable**.<jupyter_code>a = 1 + 1<jupyter_output><empty_output><jupyter_text>You will note that there is no output when you run the above cell. Assignment does not produce any output in Python - instead, the expression on the right hand side is evaluated, and then the result is stored in the variable of the left-hand side.We can then use the variable `a` again:<jupyter_code>a
a + 1
b = a + 2<jupyter_output><empty_output><jupyter_text>What is the value stored in the variable b? (What type does it have?)  Recommended reading on Variables:* [Think Python, Chapter 2](http://greenteapress.com/thinkpython2/html/thinkpython2003.html)  FunctionsSometimes we want to rerun the same procedure more than once.Instead of having to write each line of code once, we can store the procedure as a **function** and then *call* the function whenever we want to do those operations.We have already used one function: `type()`<jupyter_code>type(42)<jupyter_output><empty_output><jupyter_text>The name of the function is `type`To call the function, we write its name followed by parentheses. Any inputs to the function are placed inside the parentheses.*   The inputs are called **arguments***   The result of the function is called the **return value**We can use functions to convert data from one type to another:<jupyter_code>float(42)
str(42)<jupyter_output><empty_output><jupyter_text>**Exercises**What happens if you convert `42.3` (i.e. a float) to an integer?What happens if you convert the string `"hello"` to an integer?  Creating functionsWe can actually write our own functions!A **function definition** specifies the arguments that the functions accepts, what operations will be done inside the function, and then (optionally) a return value.```pythondef say_hello():    print("hello")```**Exercise**Write this `say_hello` function definition out in the code cell below, and run the cell to define it (optionally, you can also try to call the function *before you have defined it...*)<jupyter_code># Write function definition here
def say_hello():
    print("hello")
# Try calling the function to see what happens
say_hello()<jupyter_output><empty_output><jupyter_text>The `def` keyword indicates the start of the function definition. It is followed by the name of the function, and then parentheses stating the arguments that the function will take.The code inside the function (the **body** of the function) is indented by four spaces. There can be as many lines inside the function as you like, but each one must be indented by four spaces to indicate that it is part of the function body.```pythondef function_name(argument_1, argument_2):    a3 = argument_1 + argument_2   a line of code    a4 = a3 / 10                   another line    return some_value``` **Exercises*** What happens if you call a function name without the parentheses, e.g. just `type` or `say_hello` ?* What is the `type` of a function? E.g. what is `type(say_hello)` ?* If you assign the result of a function without a specified return value (e.g. `say_hello`) to a variable, what will be stored in the variable?  Recommended reading on FunctionsWe have only brushed the surface of functions.For a more detailed version of this tutorial, take a look at:* [Think Python, Chapter 3: Functions](http://greenteapress.com/thinkpython2/html/thinkpython2004.html)* For much more detail, see the official Python tutorial's section on [defining functions](https://docs.python.org/3/tutorial/controlflow.htmldefining-functions) **Intermediate function topics*** Non-fixed numbers of arguments in a function: https://www.linuxtopia.org/online_books/programming_books/python_programming/python_ch15s09.html* Scope of variables inside and outside of functions: https://python-textbok.readthedocs.io/en/1.0/Variables_and_Scope.html**Advanced function topics*** Lambda (i.e. 1-line) functions: https://stackabuse.com/lambda-functions-in-python/* Decorators: https://realpython.com/primer-on-python-decorators/  Control flowPython runs code one line at a time. After is has finished running one line it moves onto the next line.Functions are one way of jumping out of sequence: when we call a function, the program jumps to the first line of code in the function body (which could be earlier in the same file, or in a different file entirely).We can also pick paths for our program to go down at certain points using **conditional statements**.But first we need to learn about boolean data.  Boolean dataThere is another data type that we have not yet enountered, called the **boolean** type, which can have two possible values: `True` or `False`<jupyter_code>True
c = True
type(c)<jupyter_output><empty_output><jupyter_text>We often get a boolean value when we compare two other pieces of data using **relational operators**.For example, we can compare two variables to see if they are equal:```pythona == b```or to see if one is greater than the other:```pythona > b```<jupyter_code>a > b<jupyter_output><empty_output><jupyter_text>**Exercise**Experiment with different values and variables and see if you can figure out what these relational operators do:* `!=`* `>=` There are also 3 **logical operators** which allow us to combine boolean values: * `and` (True if both are True)* `or`  (True if either is True)* `not` (flips the boolean value<jupyter_code>True and False
True or False
not True
not False and True<jupyter_output><empty_output><jupyter_text>We can combine relational and logical operators to test for multiple things to be True or False:<jupyter_code>(42 > 30) and (type("hello") == str)<jupyter_output><empty_output><jupyter_text>Conditional StatementsWe can use the `if` keyword to evaluate certain lines based on the truth-iness of boolean expressions:```pythonif x > 0:   print("x is positive")```Note that the condition (`x > 0` is followed by a colon, `:`), and the body is indented (just like in a function).Try varying the value of x is the following code chunk to see what happens:<jupyter_code>x = 1

if x > 0:
    print("x is positive")<jupyter_output><empty_output><jupyter_text>We can also execute a different line if the condition is False:```pythonif x > 0:    print("x is positive")else:    print("x is negative")```Try varying x again:<jupyter_code>x = 1

if x > 0:
    print("x is positive")
else:
    print("x is negative")<jupyter_output><empty_output><jupyter_text>Finally we can chain multiple conditionals using the `elif` keyword:<jupyter_code>x = 0

if x > 0:
    print("x is positive")
elif x == 0:
    print("x equals 0")
else:
    print("x is negative")<jupyter_output><empty_output><jupyter_text>Collections of data, and iterationPython's `int`, `float`, `str`, and `bool` data types each hold a single piece of data.However there are also data types that hold multiple pieces of data.  ListsA `list` data type is a sequence of items. We create a list using square brackets<jupyter_code>l = [3, "b", True]

print(l)
type(l)<jupyter_output><empty_output><jupyter_text>You can put variables (and even other lists!) inside lists:<jupyter_code>[b, l]<jupyter_output><empty_output><jupyter_text>We retrieve values from within lists using **indexing**, again with square brackets after the variable name that holds the list:<jupyter_code>l[1]<jupyter_output><empty_output><jupyter_text>You will note that the "first" position (index = 1) actually returns the second item in the list `l`.Python is *zero indexed*, which means that the first item has an index of zero:<jupyter_code>l[0]<jupyter_output><empty_output><jupyter_text>We can get the length of a list using the `len` function:<jupyter_code>len(l)<jupyter_output><empty_output><jupyter_text>LoopsSometimes we want to iterate through all the items in a list. In Python there are two main ways to do this:* for loops* while loops<jupyter_code>for thing in l:
    print(thing)
thing
position = 0
while position < len(l):
    print(l[position])
    position = position + 1<jupyter_output><empty_output><jupyter_text>**Exercises**Exercise 1 from Chapter 10 of Think Python:Write a function called `nested_sum` that takes a list of lists of integers and adds up the elements from all of the nested lists. For example:```pythont = [[1, 2], [3], [4, 5, 6]]nested_sum(t)```> 21Hint: you can put loops inside loops, e.g.```pythonfor inner_list in outer_list:    for item in inner_list:        ...```Note that we have to add an extra 4 spaces of indentation each time...  DictionariesAnother data structure that can hold multiple pieces of data is a **dictionary**. Whereas a list is a sequence, a dictionary is a mapping.A physical dictionary maps from words to their definitions. A Python dictionary maps from *keys* to *values*.For example:<jupyter_code>dictionary_1 = {
    "key 1": "some value",
     5: True
}<jupyter_output><empty_output><jupyter_text>This dictionary has 2 key-value pairs:* The key `"key 1"` maps to the value `"some value"`* The key `5` maps to the value `True`Each key can only occur once in a dictionary.However, the value for any key can be a data structure such as a list or even another dictionary, allowing us to store multiple values for any key, e.g.<jupyter_code>dictionary_2 = {
     "dog": {"type": "animal", "noise": "woof"}, 
     "hamster": "animal",
     "cat": ["animal", "meow"]
     }<jupyter_output><empty_output><jupyter_text>To access the value associated with a particular key, we have to index into the dictionary using that key:<jupyter_code>dictionary_2["hamster"]<jupyter_output><empty_output><jupyter_text>If a value is itself a list or a dictionary, we can index multiple times:<jupyter_code>dictionary_2["dog"]["noise"]
dictionary_2["cat"][1]<jupyter_output><empty_output><jupyter_text>Further reading* More about lists in [Think Python Chapter 10](http://greenteapress.com/thinkpython2/html/thinkpython2011.html)* And dictionaries in [Chapter 11](http://greenteapress.com/thinkpython2/html/thinkpython2012.html)More on Iteration in [Think Python Chapter 7](http://greenteapress.com/thinkpython2/html/thinkpython2008.html)  Classes and objectsWe have talked about data and functions separately.However, there is actually a way to combine the two using something called **classes**.Essentially a class is a way to create a new data type.For example, let's say we want to create an Animal data type.We could do something like this:<jupyter_code>class Animal:
    pass

dog = Animal()<jupyter_output><empty_output><jupyter_text>Animal is a class: essentially it is a blueprint for a new type a data.`dog` is an object: it is an actual instance of an Animal.(A useful analogy is to think of architectural blueprints versus actual buildings).<jupyter_code>print(dog)
print(type(dog))<jupyter_output><empty_output><jupyter_text>We can attach functions to classes. When we do this, these functions are called **methods**.<jupyter_code>class Animal:
    
    # note how we have to indent a method by four spaces inside the class
    
    def make_sound(self): # the first argument of any method is "self"
        print("Woof!")

dog = Animal()<jupyter_output><empty_output><jupyter_text>Now we can call the method on the dog instance:<jupyter_code># we use a period to access methods on an object:
dog.make_sound()<jupyter_output><empty_output><jupyter_text>We can also set the **attributes** of an object (i.e. bits of data attached to the object):<jupyter_code>dog.name = "Fido"
print(dog.name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08897316219369895, 'fraction_numerical': 0.00969953325554259, 'mean_word_length': 5.401960784313726, 'pattern_counts': {'":': 6, '<': 131, '<?xml version=': 0, '>': 141, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '32', 'dir': 'jupyter-structured-clean-dedup', 'id': '578988', 'license': 'MIT', 'n_tokens_mistral': 4066, 'n_tokens_neox': 3865, 'n_words': 1733, 'path': 'python/python_basics.ipynb', 'repo_name': 'dominicwhite/kaggle-club', 'size': '13718'}