<jupyter_start><jupyter_text>Lecture 3: Python Variables and SyntaxCSCI 1360: Foundations for Informatics and Analytics  Overview and ObjectivesIn this lecture, we'll get into more detail on Python variables, as well as language syntax. By the end, you should be able to:  - Define variables of string and numerical types, convert between them, and use them in basic operations - Explain the different variants of typing in programming languages, and what "duck-typing" in Python does - Understand how Python uses whitespace in its syntax - Demonstrate how smart variable-naming and proper use of comments can effectively document your code This week will effectively be a "crash-course" in Python basics; there's a lot of ground to cover!  Part 1: Variables and TypesWe saw in the last lecture how to define variables, as well as a few of the basic variable "types" available in Python. It's important to keep in mind that *each variable you define has a "type"*, and this type will dictate much (if not all) of the operations you can perform on and with that variable. To recap: a *variable* in Python is a sort of placeholder that stores a value. Critically, a variable has both a *name* and a *type*. For example:<jupyter_code>x = 2<jupyter_output><empty_output><jupyter_text>It's easy to determine the name of the variable; in this case, the name is $x$. It can be a bit more complicated to determine the type of the variable, as it depends on the value the variable is storing. In this case, it's storing the number 2. Since there's no decimal point on the number, we call this number an *integer*, or *int* for short.<jupyter_code>Thus, in this case, the name of the variable is `x` and the type is `int`.<jupyter_output><empty_output><jupyter_text>Numerical typesWhat other types of variables are there?<jupyter_code>y = 2.0<jupyter_output><empty_output><jupyter_text>`y` is assigned a value of 2.0: it is referred to as a *floating-point* variable, or *float* for short. Floats do the heavy-lifting of much of the computation in data science. Whenever you're computing probabilities or fractions or normalizations, floats are the types of variables you're using. In general, you tend to use floats for heavy computation, and ints for counting things. There is an explicit connection between ints and floats. Let's illustrate with an example:<jupyter_code>x = 2
y = 3
z = x / y<jupyter_output><empty_output><jupyter_text>In this case, we've defined two variables `x` and `y` and assigned them integer values, so they are both of type `int`. However, we've used them both in a division operation and assigned the result to a variable named `z`. If we were to check the type of `z`, what type do you think it would be? `z` is a float!<jupyter_code>type(z)<jupyter_output><empty_output><jupyter_text>How does that happen? Shouldn't an operation involving two ints produce an int? In general, yes it does. However, in cases where a decimal number is outputted, Python *implicitly* "promotes" the variable storing the result. This is known as **casting**, and it can take two forms: implicit casting (as we just saw), or explicit casting.  Casting Implicit casting is done in such a way as to try to abide by "common sense": if you're dividing two numbers, you would all but expect to receive a fraction, or decimal, on the other end. If you're multiplying two numbers, the type of the output depends on the types of the inputs--two floats multiplied will likely produce a float, while two ints multiplied will produce an int.<jupyter_code>x = 2
y = 3
z = x * y
type(z)
x = 2.5
y = 3.5
z = x * y
type(z)<jupyter_output><empty_output><jupyter_text>Explicit casting, on the other hand, is a little trickier. In this case, it's *you the programmer* who are making explicit (hence the name) what type you want your variables to be. Python has a couple special built-in functions for performing explicit casting on variables, and they're named what you would expect: `int()` for casting a variable as an int, and `float` for casting it as a float.<jupyter_code>x = 2.5
y = 3.5
z = x * y
print("Float z:\t{}\nInteger z:\t{}".format(z, int(z)))<jupyter_output>Float z:	8.75
Integer z:	8<jupyter_text>Any idea what's happening here? With explicit casting, you are telling Python to override its default behavior. In doing so, it has to make some decisions as to how to do so in a way that still makes sense. When you cast a `float` to an `int`, some information is lost; namely, the decimal. So the way Python handles this is by quite literally **discarding the entire decimal portion**. In this way, even if your number was 9.999999999 and you perfomed an explicit cast to `int()`, Python would hand you back a 9.  Language typing mechanisms Python as a language is known as *dynamically typed*. This means you don't have to specify the type of the variable when you define it; rather, Python infers the type based on how you've defined it and how you use it. As we've already seen, Python creates a variable of type `int` when you assign it an integer number like 5, and it automatically converts the type to a `float` whenever the operations produce decimals. Other languages, like C++ and Java, are *statically typed*, meaning in addition to naming a variable when it is declared, the programmer must also explicitly state the type of the variable. Pros and cons of *dynamic* typing (as opposed to *static* typing)? Pros: - Streamlined - Flexible Cons: - Easier to make mistakes - Potential for malicious bugs For type-checking, Python implements what is known as [duck typing](https://en.wikipedia.org/wiki/Duck_typing): if it walks like a duck and quacks like a duck, it's a duck. This brings us to a concept known as **type safety**. This is an important point, especially in dynamically typed languages where the type is not explicitly set by the programmer: there are countless examples of nefarious hacking that has exploited a lack of type safety in certain applications in order to execute malicious code. A particularly fun example is known as a roundoff error, or more specifically to our case, a [representation error](https://en.wikipedia.org/wiki/Round-off_errorRepresentation_error). This occurs when we are attempting to represent a value for which we simply don't have enough precision to accurately store. - When there are too many decimal values to represent (usually because the number we're trying to store is very, very small), we get an *underflow error*.  - When there are too many whole numbers to represent (usually because the number we're trying to store is very, very large), we get an *overflow error*. One of the most popular examples of an overflow error was the [Y2K bug](https://en.wikipedia.org/wiki/Year_2000_problem). In this case, most Windows machines internally stored the year as simply the last two digits. Thus, when the year 2000 rolled around, the two numbers representing the year overflowed and reset to 00. A similar problem is [anticipated for 2038](https://en.wikipedia.org/wiki/Year_2038_problem), when 32-bit Unix machines will also see their internal date representations overflow to 0. In these cases, and especially in dynamically typed languages like Python, it is very important to know what types of variables you're working with and what the limitations of those types are.  String typesStrings, as we've also seen previously, are the variable types used in Python to represent text.<jupyter_code>x = "this is a string"
type(x)<jupyter_output><empty_output><jupyter_text>Unlike numerical types like ints and floats, you can't really perform arithmetic operations on strings, *with one exception*:<jupyter_code>x = "some string"
y = "another string"
z = x + " " + y
print(z)<jupyter_output>some string another string<jupyter_text>The `+` operator, when applied to strings, is called *string concatenation*. This means that it glues or *concatenates* two strings together to create a new string. In this case, we took the string in `x`, concatenated it to an empty space `" "`, and concatenated that again to the string in `y`, storing the whole thing in a final string `z`. Other than the `+` operator, the other arithmetic operations aren't defined for strings, so I wouldn't recommend trying them...<jupyter_code>s = "2"
t = "divisor"
x = s / t<jupyter_output><empty_output><jupyter_text>Casting, however, is alive and well with strings. In particular, if you know the string you're working with is a *string representation of a number*, you can cast it from a string to a numeric type:<jupyter_code>s = "2"
x = int(s)
print("x = {} and has type {}.".format(x, type(x)))<jupyter_output>x = 2 and has type <class 'int'>.<jupyter_text>And back again:<jupyter_code>x = 2
s = str(x)
print("s = {} and has type {}.".format(s, type(s)))<jupyter_output>s = 2 and has type <class 'str'>.<jupyter_text>Strings also have some useful methods that numeric types don't for doing some basic text processing.<jupyter_code>s = "Some string with WORDS"
print(s.upper())  # make all the letters uppercase
print(s.lower())  # make all the letters lowercase<jupyter_output>SOME STRING WITH WORDS
some string with words<jupyter_text>A very useful method that will come in handy later in the course when we do some text processing is `strip()`. Often when you're reading text from a file and splitting it into tokens, you're left with strings that have leading or trailing whitespace:<jupyter_code>s1 = " python "
s2 = "   python"
s3 = "python     "<jupyter_output><empty_output><jupyter_text>Anyone who looked at these three strings would say they're the same, but the whitespace before and after the word `python` in each of them results in Python treating them each as unique. Thankfully, we can use the `strip` method:<jupyter_code>print("|" + s1.strip() + "|")
print("|" + s2.strip() + "|")
print("|" + s3.strip() + "|")<jupyter_output>|python|
|python|
|python|<jupyter_text>You can also delimit strings using *either* single-quotes or double-quotes. Either is fine and largely depends on your preference.<jupyter_code>s = "some string"
t = 'this also works'<jupyter_output><empty_output><jupyter_text>Python also has a built-in method `len()` that can be used to return the length of a string. The length is simply the number of individual characters (including any whitespace) in the string.<jupyter_code>s = "some string"
len(s)<jupyter_output><empty_output><jupyter_text>Variable comparisons and Boolean types We can also compare variables! By comparing variables, we can ask whether two things are equal, or greater than or less than some other value. This sort of true-or-false comparison gives rise to yet another type in Python: the *boolean* type. A variable of this type takes only two possible values: `True` or `False`. Let's say we have two numeric variables, `x` and `y`, and want to check if they're equal. To do this, we use a variation of the assginment operator:<jupyter_code>x = 2
y = 2
x == y<jupyter_output><empty_output><jupyter_text>Hooray! The `==` sign is the equality comparison operator, and it will return `True` or `False` depending on whether or not the two values are exactly equal. This works for strings as well:<jupyter_code>s1 = "a string"
s2 = "a string"
s1 == s2
s3 = "another string"
s1 == s3<jupyter_output><empty_output><jupyter_text>We can also ask if variables are less than or greater than each other, using the `` operators, respectively.<jupyter_code>x = 1
y = 2
x < y
x > y<jupyter_output><empty_output><jupyter_text>In a small twist of relative magnitude comparisons, we can also ask if something is less than *or equal to* or greater than *or equal to* some other value. To do this, in addition to the comparison operators ``, we also add an equal sign:<jupyter_code>x = 2
y = 3
x <= y
x = 3
x <= y
x = 3.00001
x <= y<jupyter_output><empty_output><jupyter_text>Interestingly, these operators also work for strings. Be careful, though: their behavior may be somewhat unexpected until you figure out what actual trick is happening:<jupyter_code>s1 = "some string"
s2 = "another string"
s1 > s2
s1 = "Some string"
s1 > s2<jupyter_output><empty_output><jupyter_text>Part 2: Variable naming conventions and documentation [There are some rules](https://www.youtube.com/watch?v=yUfec8S10MI) regarding what can and cannot be used as a variable name. Beyond those rules, [there are guidelines](https://www.youtube.com/watch?v=jl0hMfqNQ-g).  Variable naming rules  - Names can contain only letters, numbers, and underscores. All the letters a-z (upper and lowercase), the numbers 0-9, and underscores are at your disposal. Anything else is illegal. No special characters like pound signs, dollar signs, or percents are allowed. Hashtag alphanumerics only.   - Variable names can only *start* with letters or underscores. Numbers cannot be the first character of a variable name. `message_1` is a perfectly valid variable name; however, `1_message` is not and will throw an error.  - Spaces are not allowed in variable names. Underscores are how Python programmers tend to "simulate" spaces in variable names, but simply put there's no way to name a variable with multiple words separated by spaces.  - Avoid using Python keywords or function names as variables. This might take some trial-and-error. Basically, if you try to name a variable `print` or `float` or `str`, you'll run into a lot of problems down the road. *Technically* this isn't outlawed in Python, but it will cause a lot of headaches later in your program.  Variable naming conventions These are not hard-and-fast rules, but rather suggestions to help "standardize" code and make it easier to read by people who aren't necessarily familiar with the code you've written.  - Make variable names short, but descriptive. I've been giving a lot of examples using variables named `x`, `s`, and so forth. **This is bad.** Don't do it--unless, for example, you're defining `x` and `y` to be points in a 2D coordinate axis, or as a counter; one-letter variable names for counters are quite common. Outside of those narrow use-cases, the variable names should constitute a pithy description that reflects their function in your program. A variable storing a name, for example, could be `name` or even `student_name`, but don't go as far as to use `the_name_of_the_student`.  - Be careful with the lowercase `l` or uppercase `O`. This is one of those annoying rules that largely only applies to one-letter variables: stay away from using letters that also bear striking resemblance to numbers. Naming your variable `l` or `O` may confuse downstream readers of your code, making them think you're sprinkling 1s and 0s throughout your code.  - Variable names should be all lowercase, using underscores for multiple words. Java programmers may take umbrage with this point: the convention there is to `useCamelCase` for multi-word variable names. Since Python takes quite a bit from the C language (and its back-end is implemented in C), it also borrows a lot of C conventions, one of which is to use underscores and all lowercase letters in variable names. So rather than `multiWordVariable`, we do `multi_word_variable`. The one exception to this rule is when you define variables that are *constant*; that is, their values don't change. In this case, the variable name is usually in all-caps. For example: `PI = 3.14159`.  Self-documenting code The practice of pithy but precise variable naming strategies is known as "self-documenting code."We've learned before that we can insert comments into our code to explain things that might otherwise be confusing:<jupyter_code># Adds two numbers that are initially strings by converting them to an int and a float,
# then converting the final result to an int and storing it in the variable x.
x = int(int("1345") + float("31.5"))
print(x)<jupyter_output>1376<jupyter_text>Comments are important to good coding style and should be used often for clarification. However, even more preferable to the liberal use of comments is a good variable naming convention. For instance, instead of naming a variable "x" or "y" or "c", give it a name that describes its purpose.<jupyter_code>str_length = len("some string")<jupyter_output><empty_output><jupyter_text>I could've used a comment to explain how this variable was storing the length of the string, but by naming the variable itself in terms of what it was doing, I don't even need such a comment. It's self-evident from the name itself what this variable is doing.  Part 3: Whitespace in Python Whitespace (no, not [that Whitespace](https://en.wikipedia.org/wiki/Whitespace_(programming_language))) is important in the Python language. Some languages like C++ and Java use semi-colons to delineate the end of a single statement. Python, however, does not, but still needs some way to identify when we've reached the end of a statement. In Python, it's the **return key** that denotes the end of a statement. Returns, tabs, and spaces are all collectively known as "whitespace", and each can drastically change how your Python program runs. Especially when we get into loops, conditionals, and functions, this will become critical and may be the source of many insidious bugs. For example, the following code won't run:<jupyter_code>x = 5
  x += 10<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07055764229991332, 'fraction_numerical': 0.0076856399884426465, 'mean_word_length': 4.888397414086424, 'pattern_counts': {'":': 1, '<': 100, '<?xml version=': 0, '>': 99, 'https://': 7, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '25', 'dir': 'jupyter-structured-clean-dedup', 'id': '108599', 'license': 'MIT', 'n_tokens_mistral': 4853, 'n_tokens_neox': 4574, 'n_words': 2562, 'path': 'lectures/L3.ipynb', 'repo_name': 'eds-uga/csci1360-fa16', 'size': '17305'}