<jupyter_start><jupyter_text>Python refresher: Part I Hello everyone!The first two labs are going to be just an introduction to python. It's not feasible to become familiar with programming in just two weeks, so this is meant more for people who have had previous experience with programming but might need a refresher. These are just the minimal requirements to follow the rest of the course, and it would be great if you could try to really internalize it and become familiar with it. You do this by going through the lab multiple times, or by looking at other introductions ot python on the internet. The latter might make more sense because the particular way I have to explain this stuff might not click with you; seeing different approaches is always useful.> __NOTE__ Much of what I will say here is not strictly true from a programming point of view, but it is true enough for our purposes. If you want to know more I can give you pointers for more in-depth treatments.Unfortunately, *familiarity with programming takes time* - no way around that!  Colab If you want to run this code on the server (not in your laptop), you should be seeing this document on colab. Colab. Colab is a product from Google that allows anyone to write and run python code through the browser. This way, you don't need to set up and keep track of a python environment, which can be painful, and also we can make sure that we are all working on the same system, which helps with managing packages and avoiding surprises.While colab works pretty well for our purposes, it has some disadvantages. First, it limits the amount of computation you can do. Second, it makes it more complicated to manage files if you make changes to the notebooks I'll provide. If you do, you'll have to save your version of the notebook on your laptop and then re-upload it on colab to keep working on it.  Jupyter notebooks The labs (including the one you are working on right now!) will be written in Jupyter notebooks. Jupyter notebooks are files that contain a combination of text and code. Each notebook consists of a series of _cells_, and each cell is either a text cell or a code cell. Various things to keep in mind:- You can select a cell by clicking on it.- You can edit both code and text cells. To edit a code text, press twice on it. To edit a code cell, just click on it.- You can run a cell by selecting it and then pressing ctrl+enter. If the cell is a text cell, it will render. If the cell is a code cell, it will run the code. - To add new code cells, select a cell, press Esc, and then click `a` to create a new cell above and `b` to create a new cell below the selected one.- You can move between cells also by pressing `k` to move to the cell above and `j` to move to the cell below. > **__Exercises__** > - Add a new cell below this one.  Some programming concepts in python  Spaces and indentation Spaces in python are part of the code, so adding or removing spaces can change the meaning of the code or break it. For instance, this should run fine:<jupyter_code>for i in range(3):
    print(i, end='')<jupyter_output>012<jupyter_text>while this, which is only different in the spaces, should throw an error:<jupyter_code>for i in range(3):
print(i, end='')<jupyter_output><empty_output><jupyter_text>Printing and commands As is traditional when learning a programming language, let's start with how you print the string 'Hello world'. Strings in python are written with single or double quotation marks (they aren't quite the same, but that doesn't matter here). For instance, the string containing the sentence 'Hello world' can be written as `"Hello world"` or as `'Hello world'`. The way we print a string is by passing the string as an argument to the `print` function. Here's the code:<jupyter_code>print('Hello world')<jupyter_output>Hello world<jupyter_text>> **__Exercises__** > - Create a new cell and print your name!  Comments Comments are parts of the code that are not ran when you run the code. You can add them to your code to add explanations about what your code is doing. In python, if we want to only comment one line, we use the symbol ``: everything after `` will be a comment. If we want the comment to span over multiple lines, we use three quotation marks to mark the beginning of the comment and then three more to mark its end (possibly on a different line).For instance, suppose we wanted to comment the code above to explain what we are doing. We could do it so:<jupyter_code># We are printing the sentence 'Hello world'
print('Hello world')<jupyter_output>Hello world<jupyter_text>And suppose we wanted a longer comment over multiple lines:<jupyter_code>"""
Here comes the first line of comment
and here comes the second
"""
print('Hello world')<jupyter_output>Hello world<jupyter_text>Python doesn't see the comments!  Variables in python The topic of variables is huge and we can't do it justice in the little space we have available here. But just to give you an idea, you can think of a variable as a letter or word that has some content.In python, variables are defined with the symbol `=`. On the left of the symbol you specify the name of your variable, and on the right the (initial) value, which you can generally change later. For instance, suppose we want to create a variable called `name` that has as value the string `John`. We could do it as follows:<jupyter_code>name = 'John'<jupyter_output><empty_output><jupyter_text>> __NOTE__ Once you have defined a variable by running a cell in the notebook, the variable is accessible in other cells too!Let's verify this by printing the value of `name` in a different cell:<jupyter_code>print(name)<jupyter_output>John<jupyter_text>> **__Exercises__** > - Define a new variable called `myname` that has your name as its value.> - Print `myname`. We can also print more than one variable in one go by passing multiple arguments to `print`, separated by commas:<jupyter_code>second_name = 'Rawls'

print(name, second_name)<jupyter_output><NAME><jupyter_text>Note that not all sequence of symbols can be used for variable names in python. Here's some criteria for a variable name:- It must start with a letter or the underscore character, but not a number or other symbols.- It can only contain letters, numbers, and underscores (A-z, 0-9, and _ )- It is case-sensitive (name and Name are two different variables)We will follow the convention that when a variable name contains more than one word we want to separate, we use an underscore. For instance, we will write `first_name` rather than for instance `firstName`.> __NOTE__ Always give your variables meaningful names or your code will be very confusing!  Operators We have seen how to define string and variables containing string. But what if you wanted to combine two or more strings into a longer string? This is an example of what an _operator_ can do. You can think of an operator as a python keyword to do something to an object, for example a string.In the case of combining string, we can use the operator `+`. We can do it as follows:<jupyter_code>full_name = 'John' + ' ' + 'Rawls'
print(full_name)<jupyter_output><NAME><jupyter_text>Note that we can use the operator on the variables as well, and not just on the string themselves. The operator then will be applied to the _content_ of the variables. For instance:<jupyter_code>full_name = name + ' ' + second_name
print(full_name)<jupyter_output><NAME><jupyter_text>Equality sign The equality sign in programming in a sense is close to what in English is expressed by the word 'is'. However, note that there is a fundamental difference between two sense of '`a` is 3': 1. We are _giving_ `a` the value 31. We are _claiming_ that the (previously defined) variable `a` has value 3The former expression is not making a claim, it's changing something, and therefore is neither true nor false. The latter expression is true or false depending on whether we gave `a` the value 3. In python, these two senses are written differently. We saw above that we use a single equality sign to attribute a value to a variable, which is the first sense defined above. For the second sense, we use _two_ equality signs. For example:<jupyter_code># here we are giving a value to a with a single equality sign
a = 3
# here we are checking the value of a with two equality signs
print(a == 3)

# Now let's try with a false one!
a = 4
print(a == 3)<jupyter_output>True
False<jupyter_text>Data types and their operators Things in python can be of different _types_, and depending on what type they are you can do different things with them and use different operators. To check the type of any object, use the keyword `type`. For instance, the following tells us that `a` is a string (`'str'`) and that `b` is an integer (`'int'`):<jupyter_code>a = 'Hello Mr. Fodor!'
print('a is of type: ', type(a))

b = 12
print('b is of type: ', type(b))<jupyter_output>a is of type:  <class 'str'>
b is of type:  <class 'int'><jupyter_text>Let's consider some important types one by one (not exhaustive!).   Strings: str We have already briefly seen strings above. Basically, you can use them to encode text. You can join multiple strings with `+`, as we saw above. You can also do loads of other stuff with strings!<jupyter_code>a = 'this is a good string'

print(a.capitalize())
print(a.swapcase())<jupyter_output>This is a good string
THIS IS A GOOD STRING<jupyter_text>Etcetera. If you want to know more about what you can do with strings, have a look [here](https://docs.python.org/3/library/string.html).  Integers: int Integers are just numbers without a fractional part. They can be positive or negative. Note that there are limits to the size of integers in python (and to the size of variables of every other type, too).You can do with ints everything you'd expect to be able to do:<jupyter_code># add them
print(3 + 5)

# subtract them
print(4 - 10)

# check for equality
print(3 == 5)

# multiply them
print(3 * 5)

# divide them 
# NOTE: result is a float rather than an int
print(6 / 2)

# raise them to a power
print(3 ** 2)<jupyter_output>8
-6
False
15
3.0
9<jupyter_text>Floats: float Intuitively, these are numbers with a comma. You can do with them everything you can do with integers, and it behaves as you'd expect. Moreover, note that equality doesn't care about the type:<jupyter_code>a = 3
print(type(a))

b = 3.0
print(type(b))

print(a == b)<jupyter_output><class 'int'>
<class 'float'>
True<jupyter_text>Lists: list You can think of lists as ordered sets of values of any type. You can define them as follows:<jupyter_code># create a list 
# whose first value is 1,
# whose second value is 2,
# and whose third value is 3
a = [1, 2, 3]

print(type(a))<jupyter_output><class 'list'><jupyter_text>You can add a value to the end of a list with the keyword `append`, as follows:<jupyter_code># first define a list with two strings
a = ['first', 'second']
# add a new string to the end of the list
a.append('third')
print(a)<jupyter_output>['first', 'second', 'third']<jupyter_text>You can also access elements of a list as follows:<jupyter_code>a = ['first', 'second', 'third']

# get the first element of the list
# NOTE: the index of the first element is 0!
print(a[0])

# get the second element
print(a[1])

# try to get the fourth element
# (note that the list only has three elements!)
print(a[3])<jupyter_output>first
second<jupyter_text>Boolean: bool Boolean values are truth values, and there's two of them: `True` and `False`. We can manipulate Booleans as you learned in logic 1. For instance, we can use the Boolean operators to calculate a function of multiple Boolean values:<jupyter_code>print(not True)
print(not False)
print(True and True)
print(True and False)
print(True or False)
print(False or False)<jupyter_output>False
True
True
False
True
False<jupyter_text>Some expressions in python (e.g., ones containing equality) _are_ Boolean values. For instance, the whole expression `3==(1+2)` is just the Boolean value `True`:<jupyter_code>a = 3 == (1+2)
print(a)
print(a is True)<jupyter_output>True
True<jupyter_text>Conditionals Suppose you want one thing to happen if a condition obtains and another thing to happen if it doesn't. We can use the 'if-else' construction to achieve this. The way it works is:- The first line of the if-else construct is like:```if :```where `` is some expression with a Boolean value.- Then there is one or more lines of indented code with the stuff you want to happen if the condition obtains.- Then optionally there is a line that says:```else:```followed by one or more lines of indented code with the code you want to run if the condition does _not_ hold.For instance, suppose you want to print `The variable is true` if a variable has Boolean value True and print `The variable is false` otherwise. You can do it as follows:<jupyter_code>a_boolean_variable = True

if a_boolean_variable:
    print('The variable is true')
else:
    print('The variable is false')<jupyter_output>The variable is true<jupyter_text>`for` loops Suppose you want to go through the elements of a list one by one. One way to do it would be to just do it by hand. But for long lists that would become impractical, and the point of programming is automation. A better way is to use a _for_ loop. The idea is that you run some piece of code repeatedly, and a variables you specify takes on each value in the list in turn. The syntax to do this is:1. The first line of the loop says `for  in `, where `` is the name of a (new) variable and `` is a list.1. Then there is one or more lines of indented code, a so-called _block_ of code. The code in the block is called one for every element of the list, and the variable `` takes on each value of the list in turn.To see how this works, suppose we have a list of words and we want to print each word along with its index in the list. We could do it as follows:<jupyter_code>useful_list = ['first', 'second', 'third']
for i in [0,1,2]:
    print(i, useful_list[i])<jupyter_output>0 first
1 second
2 third<jupyter_text>Some other important keywords `range`: Often, we want to loop over successive integers. `range` allows us to do this easily. Specifically, suppose we want to loop from number 0 to number 9. Then we could write `for i in range(10)`. Let's test it:<jupyter_code>for i in range(10):
    print(i)<jupyter_output>0
1
2
3
4
5
6
7
8
9<jupyter_text>Note how `range(n)` loop from 0 to $n-1$ rather than from 0 to $n$!However, suppose we want to start from number $m$ rather than starting from 0. `range` can do this too. We just need to write `range(m, n)` and we can loop from $m$ to $n-1$. Let's try it:<jupyter_code>for i in range(10, 16):
    print(i)<jupyter_output>10
11
12
13
14
15<jupyter_text>Finally, suppose we don't want all the integers, but only every three integers. Then we can write `range(m, n, 3)`. Let's test this:<jupyter_code>for i in range(10, 30, 3):
    print(i)<jupyter_output>10
13
16
19
22
25
28<jupyter_text>`enumerate`: Recall the code above where we printed each word in a list of words along with its index. Wouldn't it be cool if there was a way of getting the word _along with_ the index, without having to take the word from the index? There is! That's what `enumerate` is for. We use it as follows:<jupyter_code>for i, word in enumerate(['first', 'second', 'third']):
    print(i, word)<jupyter_output>0 first
1 second
2 third
{'fraction_non_alphanumeric': 0.06787682333873582, 'fraction_numerical': 0.008298217179902756, 'mean_word_length': 4.611495089123317, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 99, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '233678', 'license': 'MIT', 'n_tokens_mistral': 4440, 'n_tokens_neox': 4272, 'n_words': 2456, 'path': 'book/2_lab.ipynb', 'repo_name': 'thelogicalgrammar/pLoT_course', 'size': '15437'}