<jupyter_start><jupyter_text>Lab 1 - Python According to Wikipedia, [Python] is a widely used interpreted, high-level programming language used for general-purpose programming. It has a design philosophy which emphasizes code readability (notably using whitespace indentation to delimit code blocks rather than curly braces or keywords), and a syntax which allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or Java. Let us first look at the characteristics of the language:-   **General purpose**: not built for a particular usage, it works as well for scientific computing as for web and application development. It features high-level data structures and supports multiple paradigms: procedural, object-oriented and functional.-   **Elegant syntax**: easy-to-read and intuitive code, easy-to-learn minimalistic syntax, quick to write (low boilerplate / verbosity), maintainability scales well with size of projects.-   **Expressive language**: fewer lines of code, fewer bugs, easier to maintain.Some technical details:-   **Dynamically typed**: no need to define the type of variables, function arguments or return types.-   **Automatic memory management** (garbage collector): no need to explicitly allocate and deallocate memory for variables and data arrays. No memory leak bugs.-   **Interpreted**: No need to compile the code. The Python interpreter reads and executes the python code directly. It also means that a single Python source runs anywhere a runtime is available, like on Windows, Mac, Linux and in the Cloud.A couple of advantages:-   **Ease of programming**: it is very intuitive to write Python code, minimizing the time required to develop, debug and maintain the code.-   **Well-designed language**: it encourages many good programming practices, such as modularity, object-oriented programmed, good system for packaging, large open-source community, integrated documentation, among many ofthers.And some disadvantages:-   **Two versions**: there are two versions of Python in general use: 2 and 3. While Python 3 is now well adopted, some libraries or legacy code still only support Python 2 leading sometimes to incompatibility. In this class, we will use Python 3.5.-   **Slow execution**: due to its interpreted and dynamic nature, the execution of Python code can be slow compared to compiled statically typed programming languages, such as C and Fortran.-   **No compiler**: there is no compiler to catch your errors. Solutions include unit / integration tests or the use of a [linter] such as [pyflakes], [Pylint] or [PyChecker]. [Flake8] combines static analysis with style checking.  [Python]: https://en.wikipedia.org/wiki/Python_(programming_language)  [linter]: https://en.wikipedia.org/wiki/Lint_%28software%29  [pyflakes]: https://pypi.python.org/pypi/pyflakes  [Pylint]: https://www.pylint.org  [PyChecker]: http://pychecker.sourceforge.net  [Flake8]: https://pypi.python.org/pypi/flake8  Finally, a general tip: Python is really easy to read, and libraries always provide in-code as well as on-line documentation that you should read. "[*Read the doc*](https://docs.python.org/3.5/)" should be your moto when coding in Python.Now, let us start playing around!  IndentationThe very first thing to know is that Python is an **indent-based** language meaning that blocks are delimited by the indentation of the code (and not by the curly brackets `{}`, as it is the case in Java or C++ for instance). You can run the following cell by focusing it (e.g., click on the cell) and doing `CTRL-Enter` (to leave the focus on the cell) or `SHIFT-Enter` (to focus the next cell). You should get and error. Try to fix it!<jupyter_code>a = 1
    b = 2
a + b<jupyter_output><empty_output><jupyter_text>VariablesVariables are easily set. No need to specify any type. For instance, fill the following cell with your name and your sciper number:<jupyter_code>my_name = "YOURNAME"
my_sciper = 123456<jupyter_output><empty_output><jupyter_text>Weak TypingPython is a weakly typed programming language. If you are used to Java, you know that you have to specify the type of your variables. In Python, you do not need to do that, and types can be mixed. For instance here we can store fruits (`str`), numbers (`float` and `int`) and students in a `list`:<jupyter_code>fruits_and_numbers = ["Apple", 3.141593, "Cherry", 4]<jupyter_output><empty_output><jupyter_text>You can easily `append` elements to the list. Append your sciper to the list.<jupyter_code>fruits_and_numbers.append(my_name)
fruits_and_numbers.COMPLETE<jupyter_output><empty_output><jupyter_text>You can learn more about [built-in types](https://docs.python.org/3.5/library/stdtypes.html).  ForYou can go through a `list` using `for` loops. Remember that they behave like `foreach` by default. In this cell, we use the `print` [built-in function](https://docs.python.org/3.5/library/functions.html).<jupyter_code>for item in fruits_and_numbers:
    print(item)<jupyter_output><empty_output><jupyter_text>IfAn `if` statement is also pretty straight forward. The `elif` and `else` statements are optional.<jupyter_code>for item in fruits_and_numbers:
    if type(item) is int and item == my_sciper:
        print(item, "is your sciper number")
    elif type(item) is int:
        print(item, "is an integer")
    elif type(item) is float:
        print(item, "is a float")
    else:
        print(item, "might be a string")<jupyter_output><empty_output><jupyter_text>FunctionsAs many modern language, Python allows to define functions that can be called later in your code. A function is denoted by the `def` keyword. It can take *positional* (or *required*) arguments as well as *keyword* (or *optional*) arguments. Keyword arguments are optional and a default value is given in the function definition.<jupyter_code>def shout(sentence, louder=False):
    if not louder:
        print(sentence.upper())
    else:
        print(sentence.upper() + "!!!")<jupyter_output><empty_output><jupyter_text>The following three lines are equivalent:<jupyter_code>shout("you shall not pass")
shout("fly you fools", louder=False)
shout(sentence="my precious", louder=False)<jupyter_output><empty_output><jupyter_text>Try to shout your name `louder`.<jupyter_code>shout(FILL HERE)<jupyter_output><empty_output><jupyter_text>Import[Modules](https://docs.python.org/3/tutorial/modules.html) in Python are files where variables, functions and classes are defined. They can then be imported in other modules (or scripts). Python comes with a [standard library](https://docs.python.org/3.5/library/) containing a lot of useful modules that you are likely to make use of. For instance, we use here the `listdir()` function from the [operating system](https://docs.python.org/3.5/library/os.html) library.<jupyter_code>import os

os.listdir()<jupyter_output><empty_output><jupyter_text>You can also define your own modules, and import them in other files or notebooks. This is very useful if you want to modularize your code for clarity, efficiency, readability and bug-proof code. For instance here, we have a file named `internet_analytics.py` in the `notebooks/modules` folder (check it!) which contains some functions (`describe()` in this case).<jupyter_code>from modules import internet_analytics

internet_analytics.describe()<jupyter_output><empty_output><jupyter_text>You can import the whole module or just part of it, such as one function of interest.<jupyter_code>from modules.internet_analytics import solution_to_final_exam

solution_to_final_exam()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08664212463844334, 'fraction_numerical': 0.005390481199053379, 'mean_word_length': 5.204730831973899, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 11, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '137545', 'license': 'MIT', 'n_tokens_mistral': 2144, 'n_tokens_neox': 2056, 'n_words': 920, 'path': 'lab01/ix-lab1/1-python.ipynb', 'repo_name': 'Battleman/InternetAnalyticsW', 'size': '7606'}