<jupyter_start><jupyter_text>**2021/22**  Revisiting Python and Jupyter notebooks There is a growing demand for people to collaborate and present experiment results faster but in a friendly manner. Furthermore, results should be checked and validated collaboratively.In this context, using notebooks and a programming language like Python is an option that has achieved greater relevance in recent years.**Python** is a high level interpreted but object oriented programming language. It is also a strong and dynamic typed language. Strong typing means that variables do have a type and that the type matters when performing operations on a variable whereas dynamic typing means that the type of the variable is determined only during runtime. **Jupyter notebooks** are documents that can contain both code to be run and rich elements mainly for explanatory purposes such as images, links, text, etc.  Jupyter notebooksLet us focus on the most important features  Magic commands There are commands denominated `_magic command_` that starts with character `%`. - To obtain the list of commands available use `%lsmagic`.- For more information in general use `%magic`.<jupyter_code># line (%) and cell (%%) magics
%lsmagic
%magic<jupyter_output><empty_output><jupyter_text>One command _magic_ introduced in one unique line must start with the character `%` (_line magic_). On case of a command with multiple lines use `%%`.Some magic useful commands:- `!`: Executes a `shell` command. E.g.: `! ls -l`.- `%pylab inline`: Load the module `numpy` (as `np`) and the module `matplotlib` "inline".- `%time`: Evaluate the execution time of a command.- `%bash`: To run cell with bash shell.- `%%latex`: Interprets a cell as latex.- `%timeit`: Evaluates the time of execution of a command multiple times and shows the average time.- `%reset`: Delete all _workspace_ the variables.- `%bookmark`: (Allow to save a location in hard drive, i.e. actual directory to use later.- `%hist -o`: (history returns the list of executed commands).Explore some of those commands. For instance, if we want to run a script file, say script.py, we can use command magic `run` to do so:`%run script.py`<jupyter_code># Some magic useful commands
! ls -la
! pwd
! env
%time
# check others ...
%%HTML
<p>This is an HTML paragraph</p>
<p>and another paragraph</p><jupyter_output><empty_output><jupyter_text>PythonAn overview of the main features of the language  Basic instructions Try the follow simple instructions:        1+1    4+2*3    6 / 7    6 // 7 (floor or integer division)    x = 2    y = 3    z = x+y    print(z)    print("Hello world")  Primitive data types  StringThe type `string` allow us to save a character sequence. A character sequence is defined between "" or ''.Special characters (_escape characters_) can be defined using the character `\`  follow the one letter. E.g. `\n` defines a new line.    s = "Hello"    print(s)    t = "hello\neverything fine?"    print(t)<jupyter_code>s = "Hello"<jupyter_output><empty_output><jupyter_text>The command `print` accepts multiple arguments and options:    print(s,t)    print("Hello ", end='')    print("Everything fine?)<jupyter_code>print(s)
# to uppercase 
s = "hello"
s.upper()
type(s) # check the type object x
dir(s) # check the methods of string type x
help(s.upper)
s = "hello\neverything\t fine?"
print(s) # change the string line after "hello" and put a tab between "everything" and "fine"
len(s) # len of a string<jupyter_output><empty_output><jupyter_text>Concatenation and Length    s = "Hello" + "margarida"    print(s)        len(s)        s="-"*1000    print(s)  SlicesAllow the access of an element or set of elements in a variable. `s[begin : end]`. E.g.: be `s = "Hello"`.    s[1:4]    s[1:]    s[:]    s[1:100] **Important:**    s[:n] + s[n:] == s<jupyter_code>s[1:4]<jupyter_output><empty_output><jupyter_text>Operators Arithmetic  OperatorsOperator | Description | Example:-:|-|- `+` | Addiction | x = y+2`-` | Subtraction | x = y-2`*` | Multiplication | x = 3*y`/` | Division (by default integer if operands are integers) | x = y / 2`//` | Integer division (regardless of operands) | x = 3.0 // 2.`**` | Exponentiation | x = 2**4`%` | Rest of the whole division | x = 5%4  Relational Operators Operator | Description:-:|-`<` | Less than`<=` | Less or equal than`>` | Greater than`>=` | Greater or equal than`==` | Equal`!=` or `` | Different   Logical OperatorsOperator | Description :-:|-`or` | Disjunction - "or"`and` | Conjunction - "and"`not` | Negation - "not"  Inclusion Operators `in` e `not in`    character = "s"    word = "operators"    character in word  Identity Operators In `Python` most of the times the operator `=` does not copy the object but return a reference  for the same.  for testing if two objects share the same memory position can be used identity operators.Operador | Description:-:|-`is` | Returns true if two variables points the same object.`is not` | Returns false if two variables do not point to the same object.   Attention!`==` and `!=` vs `is` and `is not`    [2,3] == [3,3]    [2,3] == [2,3]    [2,3] is [2,3]        x=[2,3]    y = x    y is x        x=[2,3]    y = x    y is x    **Note: Check the id of an object:**    id(x)    id(y)    For copy complex structures can be used the module `copy`.    import copy    x = [1,[2]]    y = copy.copy(x)  Integer and DecimalsIf an integer value is set into a variable the type of data is (`int`). If the value has decimal part, it is a `float`.    x=2    type(x)        x=2.    type(x)  BooleanThe type of data `boolean` allow us to save two values: true(`True`) or false (`False`).    x = True    y = False    type(y)  Other data types  ListsLists are **mutable** sequences of objects of any type.    lista = ['machine', 'learning', 2019]    numbers = [1,2,3,4]    lista[2] = 'is the best'    del lista[2]    numbers.append(10)        x = [12, 14, 16]    numbers + x        numbers.reverse()        names = ["Joao", "Antonio", "Pedro"]    names.sort()         len(names)  TuplesTuples are **immutable** sequences typically used to store heterogeneous data. It is a single object that consists of several different parts.Once common use is to return more than one object from a Python function.<jupyter_code>tup1 = ('big', 'data', 2019)
tup2 = (1,2,3,4,5,6)
tup1[1:]<jupyter_output><empty_output><jupyter_text>RangesRanges are immutable sequences of integers (range object not a list)<jupyter_code>range(0,5)
list(range(5))<jupyter_output><empty_output><jupyter_text>Sets`sets` are a collection of elements without repetitions. The objects of type  `set` support math operations such as `union`, `intersection`, `difference` e `symmetric difference`.    colors = {'blue', 'red', 'green', 'orange', 'red'}    colors        'green' in colors    'yellow' in colors        lesscolors = set(['blue', 'green', 'yellow'])        colors - lesscolors    colors | lesscolors    colors & lesscolors    colors ^ lesscolorsAdd or remove elements    colors.add('black')    colors.pop()  DictionariesDictionaries are maps between _key objects_ for _value objects_.The _key objects_ must be unchanging!The dictionaries can be created with `{ }` or using constructor  `dict()`:    phones = {'joao': 123, 'pedro': 124, 'antónio': 125}    rooms = dict([('joao', 'D604'), ('pedro', 'D605')])    months = dict(jan=31, fev=29, mar=31)     Access dictionary elements         The access of a dictionary elements do it using `[ ]`  and indicating the corresponding _key_:        phones['joao']    rooms['pedro'] = 'D606'    Delete a element we use del command `del`:        del rooms['pedro']    Remove the entry with 'pedro' key.     months.clear()       Remove all the dictionary elements.    del months          Delete the dictionary.     MethodsThe following table lists a set of methods available for dictionary objects (note that there are more).Method | Description-|-`dict.clear()` | Remove all dictionary elements `dict`.`dict.copy()` | Returns a copy of the dictionary `dict`.`dict.keys()` | Returns a list with dictionary `keys`. `dict.values()` | Returns a list of dictionary elements `dict`.`dict.items()` | Returns a list of tuples (`key`, `value`) from the dictionary `dict`.`dict.has_key(key)` | Returns `true` if the `key` exists in the dictionary `dict`.  Control structures  Indentation- code block starts with character`:`- use o TAB- select the code block and use TAB or shift TAB  Ifs<jupyter_code>if x>5:
    print("Bigger than 5")
else:
    print("Less or equal to 5")
if x>5:
    print("Bigger than 5")
elif x==5:
    print("Equal to 5")
else:
    print("Less than 5")<jupyter_output><empty_output><jupyter_text>Loops<jupyter_code>x = 10
while x > 0:
    print(x)
    x-=1
new_list = ['Hello', 'Good', 'Morning']
for item in new_list:
    print(item)
for i in range (2,10,2):
    print(i)
coordinates = [(i, i**2) for i in range(10)]
coordinates<jupyter_output><empty_output><jupyter_text>Functions<jupyter_code>def fib(n):    # write Fibonacci series up to n
   """Print a Fibonacci series up to n."""
   a, b = 0, 1
   while a < n:
     print (a,"", end='')
     a, b = b, a+b
fib(5)
# Return more than one value in a function
def squares_till_n(n):
    return [(i,i**2) for i in range(1,n+1)]
squares_till_n(5)<jupyter_output><empty_output><jupyter_text>Anonymous functions<jupyter_code>sqr = lambda x: x**2
sqr(5)
# One function can return a function:
def make_incrementor(n):
        return lambda x: x + n
incrementa4 = make_incrementor(4)
incrementa4(5)<jupyter_output><empty_output><jupyter_text>Classes<jupyter_code># Defining a new class with a function that receives a list and removes the min integer. 

class MyList(list):
    def remove_min(self):
        self.remove(min(self))
x = [10,3, 5, 1, 2, 7, 6, 4, 8]
y = MyList(x)
y

y.remove_min()
y<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12052117263843648, 'fraction_numerical': 0.016490228013029316, 'mean_word_length': 4.0, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 58, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '123052', 'license': 'Apache-2.0', 'n_tokens_mistral': 3171, 'n_tokens_neox': 3004, 'n_words': 1225, 'path': 'Aulas/Aula_01/Python-Notebook.ipynb', 'repo_name': 'JoaoAlmeida-dev/ABD', 'size': '9824'}