<jupyter_start><jupyter_text>Introducton to Python syntax and features  I. Introduction1. All code and examples use Python 3.x2. We will use the following Python packages:  - Pandas   - Matplotlib  - Numpy3. General instructions:  - Some sections may contain a study drill, which is a small exercise to help you review the concepts.   - You should work through each section in sequence - each section builds upon the previous.    - You can run code using the 'play' button that appears when you click in each interactive cell. Additionally, you can use a keyboard shortcut (shift + enter) to run the contents of the code cell.4. Style conventions:  - In-text code snippets will be formatted like this:     `print("Hello world!")`  - Interactive cells will look the like code examples, but will have a set of square brackets in the top left ( `[ ]`).  - Study drills will be formatted like this:  > **Study drill:**  >   > What is the difference between these two blocks of code?   II. Outline* Python syntax* Built-in data types* Operators* Basic data types* Loops* Functions* Importing data  III. SyntaxPython is an interpreted language, so code may be run in two ways:- "Interactive" programming- "Script" programming  3.1 "Interactive mode" programmingInteractive programming allows you to enter code and then immediately see the results. This style of programming is useful for simple operations or "line by line" programming. We will be interactively programming in Python in this workshop.- Type the following code in the cell below and use "Shift+Enter" to run the code.    ```python    print("Hello World!")    ```    __`print()` is a Python built-in function. This means that it is "always there."__<jupyter_code># Type your code below:<jupyter_output><empty_output><jupyter_text>3.2 A brief note about "script mode" programmingFor complex programming tasks, it is common to save the code in plain text files with a ".py" extension. Programming this way allows you to save your code for reuse and automation, and allows you to interact with multiple programs at once.       3.3 Quotations & string dataText or string data have to be wrapped in quotation marks otherwise Python will attempt to evaluate the words as code and will raise an error.- Single quotes (`'`)- Double quotes (`"`)```pythona_word = 'Python'a_sentence = "Today is a beautiful day!"```Single and double quotes may be used interchangeably, but be careful when using single quotes when your text contains apostrophes!```python The following string will cause an error:a_sentence = 'Spam and eggs are my friend's favorite foods.' The same string wrapped in double quotes will be interpreted correctly:a_sentence = "Spam and eggs are my friend's favorite foods."```- Triple quotes (`'''` or `"""`)```python A multi-line string:a_paragraph = '''We are starting to study Python. After we finish this tutorial, we will be on our to programming with Python! That's pretty cool!'''```> **Study drill:**> - Create a multi-line comment and store the comment in a variable.> - Return the contents of the variable using `print(variable)`.> - Now return the contents of the variable by calling the variable directly, `variable`.> - What is different between what each printed to the screen?<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>3.4 Comments- Any line beginning with an octothorpe (or hashtag) `` is treated as a comment in the Python interpreter.- Comments are a great way to leave explanations or instructions in the code.- For example:    ```python     Print Hello world with print() function    print("Hello World!")    ```- Multi-line comments should be wrapped in triple quotes, for example:    ```python    '''    This comment will cover multiple lines.    Anything between the triple quotes will be interpreted as a comment.    Neat!    '''    ```Although multi-line comments are possible with tripe quotes, it is common to only use them as Docstrings for functions. If you are writing comments in your code that spans multiple lines, you should get in the habit of using the single line octothorpe/hash comment style.   3.5 Reserved words in Python- The following reserved words cannot be used as variable names or other identifier names (__Lower case only__). In general, you should avoid using any variation of these words in your variable names.|          ||         ||        ||:---:     ||:---:    ||:---:   |  | assert   || finally || or     || break    || for     || pass   || class    || from    || print  || continue || global  || raise  || def      || if      || return || del      || import  || try    || elif     || in      || while  || else     || is      || with   || except   || lambda  || yield  | | not      || exec    || and    |  3.6 Indentation- The number of spaces in the indentation is variable (typically four of five spaces), but all statements within the block must be indented the same amount.In the examples below, the `if` statement is not indented, but the statement that follows the colon is indented.- For example:  __Copy below code to the below cell and run it:__      ```python    A correct if statement  if 2 > 3:      print(True)  else:      print(False)  ```  __Copy below code to the below cell and run it:__  ```python   An incorrect if statement  if 2 > 3:      print(True)  else:  print(False)  ```  > **Study Drill:**>> What are the differences between these two blocks of code?<jupyter_code># Try each code block here!<jupyter_output><empty_output><jupyter_text>3.7 Built-in data typesAll data in Python has a type associated with it. Data types allow you to perform operations like arthmetic and text formatting. When you create data, Python can often automatically determine what type it is. However, it can be useful to explictly specify the type.Here we only cover some basic built-in data types (Strings, Integers, Floats). Click [here](https://docs.python.org/2/library/stdtypes.htmlnumeric-types-int-float-long-complex) for More.  3.7.1 Numeric data- Integers    ```python     Assign the number 1 to the variable `an_integer` with assignment operator `=`.     Now, the variable `an_integer` has the value `1`.    an_integer = 1     Alternatively, we could explicitly specify 1 is an integer using int()    an_interger = int(1)    ```- Floats     ```python     Assign a floating point number to the variable `a_float` with assignment operator `=`.     Now,the variable `a_float` represents the floating point number.    a_float = 1.4325     Alternatively, we could explicitly specify 1.4325 is a float using float()    a_float = float(1.4325)    ```  3.7.2 Textual data - Strings    ```python     Assign a string `Hello` to the variable `greeting_text` with assignment operator `=`.     Now, the variable `greeting_text` represents the string `Hello`.    greeting_text = "Hello"    ```Anything that is wrapped in quotation marks will be treated as a string. However, it is sometiems useful to convert numeric data into strings without using quotations. We can use `str()` to accomplish this.```python Convert the value of an_integer to a stringstr(an_integer) We can also pass raw numeric data to str() as wellstr(6174)```<jupyter_code># Create some variables and assign some string or character data to them.
# The variable names and content you assign to them are up to you!<jupyter_output><empty_output><jupyter_text>3.7.3 Checking the data type of a variableWhat's the difference between `1` and `"1"`? Python interprets `1` as an integer and `"1"` as character or string data. When you print the value of a variable, the results are displayed as character data, which makes it difficult to confirm what the actual data type stored in a variable is.You can check the data type of any value using the `type()` function.```python Assign the integer `1` to a variable `an_integer` Assign the string `"1"` to a variable `a_string` Confirm the data type or "class' of each using `type()`an_integer = 1a_string = "1"type(an_integer)type(a_string) Just to test that type works as expected,  let's convert a integer to a string and then check the typetype(str(1))```> **Study Drill:** >> 1. Declare three variables and assign an integer, a float, and string to each.> 2. Return the value of each variable by typing the variable name.> 3. Try returing the value of each using the `print()` function.> 4. Confirm that the value stored in each of your variables is what it should be using the `type()` function.>> How do the results of calling the variable, printing the value of the variable, and testing the type of each variable differ?<jupyter_code># Explore built-in data types here!<jupyter_output><empty_output><jupyter_text>3.8 Basic arithmetic- Addition `+`    -     ```python    2 + 2    ```- Subtraction `-`    -     ```python    4 - 2    ```- Division `/`    -     ```python    4 / 2    ```- Multiplication `*`    -     ```python    2 * 2    ```- Modulo `%`    -     ```python    5 % 2    ```    > **Study drill:**>> Find the answers of the above operations by using the cell below.<jupyter_code># Do your arithmetic here!<jupyter_output><empty_output><jupyter_text>3.9 Logical operators The data type returned by following are Boolean i.e., The output will be either True or False.- Less than   <    -     ```python    4 < 2    ```- Greater than  >     -     ```python    3 > 1    ```- Equals ==     -     ```python    True == False    ```- Not equal !=     -     ```python    2 != 2    ```- Less than or equal <=     -     ```python    3 <= 4    ```- Greater than or equal >=     -     ```python    3 >= 2    ```    > **Study drill:** > Evaluate each logical statement above. Use the below cell to run the each operation and get the answer (`True` or `False`). >> Hint: Run one line code each time.<jupyter_code># Evaluate your logical statements here!<jupyter_output><empty_output><jupyter_text>IV. Basic data structuresIn order to get the most value out Python, we will need to organize our data.Python has three basic data structures: 1. Lists (mutable collections of objects)2. Tuples (immutable collection of objects)3. Dictionaries (collections of "key-value" pairs)  4.1 Lists- Lists are comma separated sequnces of object enclosed in square brackets (`[..., ...]`)- Lists can contain different data types(variables, integers, floats, strings, etc.)Copy the following code to the below cell and use __"Shift+Enter"__ to run it:```pythonlist_of_numbers = [1, 2, 3, 4, 5, 6]list_of_strings = ['a', 'b', 'c', 'd', 'e', 'f']list_of_mixed_types = [1, 'a', 2, 'b', 3.14, 'Section 4.1 - Lists']```<jupyter_code># Create the lists here!

# Practice here!
# Call each list variable to see the contents<jupyter_output><empty_output><jupyter_text>4.1.1 List indexes- Lists are unordered sequences of elements, i.e. by default, the value of an element has no impact on its position within the list.- Each element can be accessed using a positional index (__Note: Python indexes start with 0__).- The syntax for accessing list elements by index position is `name_of_list[index]`- Copy the following code to the below cell and use "**Shift+Enter**" to run it:```python Below code will get the first element of `list_of_numbers` listlist_of_numbers[0]```<jupyter_code># Practice here!
# Return the element in postion 0<jupyter_output><empty_output><jupyter_text>> **Study Drill:**>>- What is the __2nd__ element of `list_of_numbers`?>- What is the __5th__ element of `list_of_strings`?>- What are the __2nd and 4th__ elements of `list_of_mixed_types`?<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>4.1.2 Append a list- We can use the `append` method to add a element to the end of a list. - __Methods__ are a way to interact with an object such as a list. - We can use a method using dot notation (`object.method()`). Type the name of the ojbect followed by `.` and then the method name and `()`. Some methods can accept arguments, which are placed in between the parentheses.```python Add `element` to the list `Listname`Listname.append(element)```Copy following code to the below cell and use "Shift+Enter" to run it:```python   Create a list  some_list = [1, 2, 3, 4, 5, 6]       Add the element "7" to the list  some_list.append(7)       Print all elements in the list  print(some_list)```<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>4.1.3 Insert an element into a list- We can use the `insert()` method to insert an element to any part of the list using an index number. - The first argument of the `insert()` method is the element position you want to insert (__Note: Python indexes start with 0__).- Copy following code to the cell below and use "Shift+Enter" to run it:```python Create a listanother_list = [1, 2, 3, 4, 5, 6] Add the element "10" as the first element of the listanother_list.insert(0, 10) Print the elements in the listprint(another_list)```<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>4.1.4 Removing elements from a list- We can use the `remove()` method to remove an element from any part of the list. - In this example, the `remove()` method takes a single argument which is the postion of the element you want to remove (__Note: Python indexes start with 0__).- Copy following code to the below cell and use"Shift+Enter" to run it:```python Create a listspam_numbers = [1, 2, 3, 4, 5, 6] Remove the element at index 2spam_numbers.remove(2) Print the elements in the listprint(spam_numbers)```<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>4.1.5 List slicing- If we only want to print some elements of the list, but not all of them, we can "slice" the list.- List slicing uses two index numbers separated by a colon to specify a range of elements.- The range you use to slice a list will return the first specified element *up to, but not including,* the second element.- Copy following code to the cell below and use "Shift+Enter" to run it:```python Create a listlist_to_slice = [1, 2, 3, 4, 5, 6] Print the 2nd element of the Listprint(list_to_slice[1]) Print the 2nd and 3rd elements of the Listprint(list_to_slice[1:3]) Print the all elements before 3rd element in the List.print(list_to_slice[:3]) Print the all elements after 2nd element in the List.print(list_to_slice[2:])```<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>> **Study Drill:**>> What did you notice about the syntax for slicing the list to return the 2nd and 3rd elements of the list?>> Create a list in the below cell: `practice_list = [1, 2, 4, 5, 6]`>    - Insert a "3" into the third position in the list>    - Remove last element of the list>    - Append a "7" to the list>    - Print all elements>    - What happens if you don't use the `print()` function? Try to return the first element of `practice_list` without using the `print` statement.<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>4.2 Tuples- Tuples are enclosed in parentheses`()` and values are sperated by commas `,`.- Copy the following code to the below cell and use __"Shift+Enter"__ to run it:```pythonspam_tuple = (1, 2, 3, 4, 5, 6)```> **Study drill:**>> How do lists and tuples differ?>> Try using a list method on a tuple. What happens?<jupyter_code># Practice here!<jupyter_output><empty_output><jupyter_text>4.3 Dictionaries- A Dictionary is a structure that stores pairs of objects, e.g. keys and values.- Dictionaries are like lists, except that you index them with keys. - Keys have to be unique within the dictionary and named using something immutable, such as a tuple, string, or integer.- Dictionaries are enclosed in curly braces `{}` and keys are separated from values using a colon.- Copy following code to the below cell and use"Shift+Enter" to run it:```python A Dictionarydays_of_the_week = {1: 'Monday', 2: 'Tuesday', 3: 'Wednesday', 4: 'Thursday', 5: 'Friday', 6: 'Saturday', 7: 'Sunday'} Print the paired value of the key "2"print(days_of_the_week[2])```<jupyter_code># Enter your code here!<jupyter_output><empty_output><jupyter_text>You can assign a new value to a key using the following syntax: `days_of_the_week[1] = 'Sunday'`> **Study drill:**>> Update the values of `days_of_the_week` accordingly so that the week begins on Sunday.<jupyter_code># Update your dictionary here!<jupyter_output><empty_output><jupyter_text>V. Flow control: a.k.a. 'loops'Loops allow you to repeat a block of code or perform an action multiple times until a specified condition terminates the loop. We will cover three different types of flow control:1. The `while` loop2. The `for` loop3. The `if-else` statement  5.1 The `while` loop- Repeats a statement or group of statements while a given condition is `True`. - Useful for when a condition needs to be checked each iteration, or to repeat a block of code numerous times.  ![WhileLoop](https://raw.githubusercontent.com/drdn/intro_python_module/master/assets/python_while_loop.jpg)    > Image credit: Tutorialspoint.com- Copy the following code to the below cell and use __"Shift+Enter"__ to run it:```python  Assign the value `0` to a variable named `eggs`eggs = 0 Write a while loop that continues until the value of `eggs` is less than 10while eggs < 10:   Increment the value of `eggs` by `1`  eggs = eggs + 1   Print the current value of `eggs`  print(eggs)```<jupyter_code># Copy just the code, not the comments here. Remember to preserve the correct indentation!<jupyter_output><empty_output><jupyter_text>5.2 The `for` loop- `for` loops are traditionally used when you have a block of code that you want to repeat a fixed number of times. - In Python, the `for` statement iterates over the members of a sequence in order, executing the block each time.  ![ForLoop](https://raw.githubusercontent.com/drdn/intro_python_module/master/assets/python_for_loop.jpg)  > Image credit: Tutorialspoint.com- Copy the following code to the cell below and use __"Shift+Enter"__ to run it:```python A for loopnumbers = 0for i in range(10):    numbers = numbers + 1    print(numbers)```<jupyter_code># Try out the for loop here!<jupyter_output><empty_output><jupyter_text>> **Study drill:**>> Our example of a `while` loop and a `for` loop are very similar. What's the difference?>> The `for` loop also introduced the `range()` function. Can you guess what it does?  5.3 The `if-else` statementThe `if-else` statement is a powerful way to create multiple outcomes for the same piece of code. Some examples how the `if-else` statement can be used:- Return different text for even and odd values.- Write placeholder text to a spreadsheet in the event of an error or unexpected value.- Test a string for specific formatting, such as an email address, and if the email is incorrectly formatted, prompt a user to re-enter their email address.  ![IfELSE](https://raw.githubusercontent.com/drdn/intro_python_module/master/assets/decision_making.jpg)    > Image credit: Tutorialspoint.com- Copy the following code to the below cell and use __"Shift+Enter"__ to run it.- On the first run, enter a value greater than 50.- Run the code again with __"Shift+Enter"__ and enter a value less than 50```python Define a Variable "Weight" weight = float(input("How many pounds does your suitcase weigh? ")) If Statement: Judge if weight over than 50 lbsif weight > 50:     If the value we input here is bigger 50 would print following statement    print("There is a $25 charge for luggage that heavy.")    else:     If the value we input here is less than 50, would print following statement    print("There is no charge for your luggage. Thanks!")```<jupyter_code># Enter your code here!<jupyter_output><empty_output><jupyter_text>VI. FunctionsA function is a block of organized, reusable code that is used to perform a specific task. Using functions is beneficial and introduce modularity and reusability of the same code. For example, `print()` is a function; it performs the same action everytime, but there is a lot of code that we don't see. We can create our own functions in Python.- Functions have to be defined using the `def` keyword. - The entire body of the function is indented.- It is good practice to include a multi-line comment known as a 'docstring' which explains what the function does and what the requirements are.We're going to create a simple function that prompts a user for two numbers and then returns the sum.<jupyter_code># Shift+Enter to run this code
def interactive_addition():
  '''
  The arguments passed to this function will be summmed 
  and returned."
  ''' 
  # Prompt a user to input the values of a and b
  a = float(input("Please input the first number: "))
  b = float(input("Please input the second number: "))

  # Do the math
  result = a + b
  return result

# Run the cell above and then call the function `interactive_addition()`<jupyter_output><empty_output><jupyter_text>This function is useful if you want to perform interactive addition. Notice that there are no arguments required in function definition.If you wanted to create something more reusable, remove the input prompts and add arguments to the function definition.For example:```pythondef add_two_numbers(a, b):  '''Returns the sum of two numbers.'''  result = float(a) + float(b)  return result```> **Study drill:**>> 1. Add two numbers using the `add_two_numbers` function.> 2. Create a new function called `add_terms` which sums three terms.> 3. Edit the `return` statement to print a message along with the result. (**Hint:** Use the following syntax `print("Message", result)`<jupyter_code># Practice writing functions here<jupyter_output><empty_output><jupyter_text>VII. Importing modules and dataPython only loads a set of basic libraries on startup, but there are a lot more available in both the default Python distribution and from the Python community. Although a specific module/library may be installed on your computer, you cannot access the code until you `import` it. The `import` statement has a simple syntax: `import nameOfModule`.- The following code shows how to import two modules:  1. Pandas, a popular data analysis library, and  2. NumPy, a scientific computing library```python Import Pandasimport pandas Import NumPyimport numpy```- There is nothing wrong with your code if Pandas or Numpy are imported this way. However, if you were to use any of the functions from either one of these modules, you would have to type the full name each time, e.g. `pandas.read_csv()`.- To save a few keystrokes, you can give modules a short alias. Pandas is commonly imported as `pd` and Numpy as `np`. Knowing these conventions is also useful when reusing other people's code.```pythonimport pandas as pdimport numpy as np```  7.1 Import a plain text file (CSV)A comma-separated values (CSV) file stores tabular data as plain text with distinct records on each line and each record containg one or more fields or values. The use of the comma as a field separator is the source of the name for this file format.- There are several ways to read CSV files into Python, but we will be using Pandas here for its simplicity.- Pandas stores tabular data in data structures known as 'data frames.' A common convention is to assign data frames to variables with `df` in the variable name.- Copy the following code to the below cell and use __"Shift+Enter"__ to run it  ```python  import pandas as pd  data = pd.read_csv('https://raw.githubusercontent.com/drdn/intro_python_module/master/data/data.csv')  print(data)  ```  **Note:** The data file is being sourced from a URL. We could also read the file from a local filesystem or even from Google Drive! The file is provided to you in this manner to account for differences in operating systems and varying levels of familiarity with navigating file systems.<jupyter_code># Enter your code here<jupyter_output><empty_output><jupyter_text>7.2 Import Excel files (with multiple worksheets)CSV files are a platform independent spreadsheet format. However, you might create data using Excel or receive data in an Excel format. Pandas has a function specifically for reading Excel data.- Copy the following code to the below cell and use __"Shift+Enter"__ to run it```python Import pandasimport pandas as pd Store the data file name in the `file` variablefile = 'https://github.com/drdn/intro_python_module/blob/master/data/UN_Data.xltx?raw=true' Read the Excel file using the `ExcelFile` function Store the data in the `UN` variableUN = pd.ExcelFile(file) Excel files have a `sheet_names` attribute Find the names of the sheets in the `UN` Excel objectUN.sheet_names Use the `.parse()` method on an Excel file to extract a specific sheetdf_HDI = UN.parse('HDI') Print the contents of df_HDI to the consoleprint(df_HDI)```<jupyter_code># Enter your code here!<jupyter_output><empty_output><jupyter_text>VII. The End!Congratulations! You've completed the first section of the Intro to Python online learning module!Take a break, and when you're ready, continue on to the next section, [Working with Data using Python and Pandas]()!  References:- Data Carpentry : http://www.datacarpentry.org/python-ecology-lesson/00-short-introduction-to-Python/- Tutorials Point: https://www.tutorialspoint.com/python/- Python 3 Document: https://docs.python.org/3/- Matplotlib: https://matplotlib.org/- Data Camp: https://www.datacamp.com/- Plotly: https://plot.ly/- Sentdex Youtube Channel : https://www.youtube.com/user/sentdex- UN Human Developments Data :http://hdr.undp.org/en/data- WIKIPEDIA : https://www.wikipedia.org/**Take your learning further:*** [Learn Python the Hard Way](https://learncodethehardway.org/python/)* [Python Crash Course](https://ehmatthes.github.io/pcc/)<jupyter_code># Enter whatever code you want to here!<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08835637719434519, 'fraction_numerical': 0.009942519807363678, 'mean_word_length': 4.364375, 'pattern_counts': {'":': 0, '<': 110, '<?xml version=': 0, '>': 174, 'https://': 15, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '71940', 'license': 'CC0-1.0', 'n_tokens_mistral': 7309, 'n_tokens_neox': 6893, 'n_words': 3512, 'path': 'notebooks/1_Intro_to_Python.ipynb', 'repo_name': 'drdn/intro_python_module', 'size': '25748'}