<jupyter_start><jupyter_text>Lecture 07 - Working with External LibrariesIn this lesson, we will walkthrough **imports** in Python, giving some tips for working with unfamiliar libraries (and the objects they return).  ImportsSo far we've talked about types and functions which are built-in to the language. But one of the best things about Python (especially if you're a data scientist) is the vast number of high-quality custom libraries that have been written for it. Some of these libraries are in the "standard library", meaning you can find them anywhere you run Python. Others libraries can be easily added, even if they aren't always shipped with Python.Either way, we'll access this code with **imports**.We'll start our example by importing **`math`** from the standard library.<jupyter_code>import math

print("It's math! It has type {}".format(type(math)))<jupyter_output>It's math! It has type <class 'module'><jupyter_text>`math` is a **module**. A **module** is just a collection of variables (a *namespace*, if you like) defined by someone else. We can see all the names in `math` using the built-in function `dir()`.<jupyter_code>print(dir(math))<jupyter_output>['__doc__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']<jupyter_text>We can access these variables using dot syntax `.`. Some of them refer to simple values, like **`math.pi`**:<jupyter_code>print("pi to 4 significant digits = {:.4}".format(math.pi))<jupyter_output>pi to 4 significant digits = 3.142<jupyter_text>But most of what we'll find in the module are **functions**, like **`math.log`**:<jupyter_code>math.log(32, 2)<jupyter_output><empty_output><jupyter_text>Of course, if we don't know what `math.log` does, we can call `help()` on it:<jupyter_code>help(math.log)<jupyter_output>Help on built-in function log in module math:

log(...)
    log(x[, base])
    
    Return the logarithm of x to the given base.
    If the base not specified, returns the natural logarithm (base e) of x.<jupyter_text>We can also call ****`help()`** on the module itself. This will give us the combined documentation for *all* the functions and values in the module (as well as a high-level description of the module).<jupyter_code>help(math)<jupyter_output>Help on built-in module math:

NAME
    math

DESCRIPTION
    This module is always available.  It provides access to the
    mathematical functions defined by the C standard.

FUNCTIONS
    acos(...)
        acos(x)
        
        Return the arc cosine (measured in radians) of x.
    
    acosh(...)
        acosh(x)
        
        Return the inverse hyperbolic cosine of x.
    
    asin(...)
        asin(x)
        
        Return the arc sine (measured in radians) of x.
    
    asinh(...)
        asinh(x)
        
        Return the inverse hyperbolic sine of x.
    
    atan(...)
        atan(x)
        
        Return the arc tangent (measured in radians) of x.
    
    atan2(...)
        atan2(y, x)
        
        Return the arc tangent (measured in radians) of y/x.
        Unlike atan(y/x), the signs of both x and y are considered.
    
    atanh(...)
        atanh(x)
        
        Return the inverse hyperbolic tangent of x.
    
    ceil(...)
        ceil(x)
        
 [...]<jupyter_text>Other Import SyntaxIf we know we'll be using functions in `math` frequently we can import it under a shorter **alias** to save some typing (though in this case "math" is already pretty short).<jupyter_code>import math as mt
mt.pi<jupyter_output><empty_output><jupyter_text>> By convention, later on when you work with certain popular libraries like **Pandas**, **Numpy**, **Sklearn**, or **Matplotlib**. For example, it's a common convention to **`import numpy as np`** and **`import pandas as pd`**. The **`as`** simply renames the imported module. It's equivalent to doing something like:<jupyter_code>import math
mt = math<jupyter_output><empty_output><jupyter_text>Wouldn't it be great if we could refer to all the variables in the `math` module by themselves? Ex: Can we just refer to **`pi`** instead of **`math.pi`** or **`mt.pi`**? Good news: we can do that.<jupyter_code>from math import *
print(pi, log(32, 2))<jupyter_output>3.141592653589793 5.0<jupyter_text>**`import *`** imports everything in a specific module. Specifically, it makes all the module's variables directly accessible to you (without any dotted prefix).Let's try with another example:<jupyter_code>from math import *
from numpy import *
print(pi, log(32, 2))<jupyter_output><empty_output><jupyter_text>There is an error here... Let's see!These kinds of **"star imports"** `import *` can occasionally lead to weird, difficult-to-debug situations.The problem in this case is that the `math` and `numpy` modules both have functions called `log`, but they have different semantics. Because we import from `numpy` later, its `log` overwrites the `log` variable we imported from `math`.A good compromise is to import only the specific things we'll need from each module:<jupyter_code>from math import log, pi
from numpy import asarray<jupyter_output><empty_output><jupyter_text>SubmodulesWe've seen that modules contain variables which can refer to **functions** or **values**. Something to be aware of is that they can also have variables referring to *other modules*.<jupyter_code>import numpy
print("numpy.random is a", type(numpy.random))
print("it contains names such as...",
      dir(numpy.random)[-15:]
     )<jupyter_output>numpy.random is a <class 'module'>
it contains names such as... ['set_state', 'shuffle', 'standard_cauchy', 'standard_exponential', 'standard_gamma', 'standard_normal', 'standard_t', 'test', 'triangular', 'uniform', 'vonmises', 'wald', 'warnings', 'weibull', 'zipf']<jupyter_text>So if we import `numpy` as above, then calling a function in the `random` "submodule" will require ***two dots***.<jupyter_code># Roll 10 dice
rolls = numpy.random.randint(low=1, high=6, size=10)
rolls<jupyter_output><empty_output><jupyter_text>Oh the places you'll go, oh the objects you'll seeSo after 6 lessons, you're a pro with **ints**, **floats**, **bools**, **lists**, **strings**, and **dicts** (right?). Even if that were true, it doesn't end there. As you work with various libraries for specialized tasks, you'll find that they define their own types which you'll have to learn to work with. For example, if you work with the graphing library **`matplotlib`**, you'll be coming into contact with objects it defines which represent Subplots, Figures, TickMarks, and Annotations. **`pandas`** functions will give you DataFrames and Series. This section introduces to you a quick survival guide for working with strange types. Three tools for understanding strange objectsIn the cell above, we saw that calling a `numpy` function gave us an **`array`** (`roll` variable). We've never seen anything like this before (not in this course anyways). But don't panic: we have three familiar builtin functions to help us here.**1: `type()`** (what is this thing?)<jupyter_code>type(rolls)<jupyter_output><empty_output><jupyter_text>**2: `dir()`** (what can I do with it?)<jupyter_code>print(dir(rolls))<jupyter_output>['T', '__abs__', '__add__', '__and__', '__array__', '__array_finalize__', '__array_interface__', '__array_prepare__', '__array_priority__', '__array_struct__', '__array_ufunc__', '__array_wrap__', '__bool__', '__class__', '__complex__', '__contains__', '__copy__', '__deepcopy__', '__delattr__', '__delitem__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__', '__imatmul__', '__imod__', '__imul__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__ior__', '__ipow__', '__irshift__', '__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__', '__lshift__', '__lt__', '__matmul__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmatmul__', '__[...]<jupyter_text>What am I trying to do with this dice roll data? Maybe I want the average roll, in which case the **`mean`**Method looks promising...<jupyter_code>rolls.mean()<jupyter_output><empty_output><jupyter_text>Or maybe I just want to get back on familiar Python list, in which case I might want to check out **`tolist`**<jupyter_code>rolls.tolist()<jupyter_output><empty_output><jupyter_text>**3: `help()`** (tell me more)That **"ravel"** attribute sounds interesting. I'm a big classical music fan.<jupyter_code>help(rolls.ravel)<jupyter_output>Help on built-in function ravel:

ravel(...) method of numpy.ndarray instance
    a.ravel([order])
    
    Return a flattened array.
    
    Refer to `numpy.ravel` for full documentation.
    
    See Also
    --------
    numpy.ravel : equivalent function
    
    ndarray.flat : a flat iterator on the array.<jupyter_text>Okay, just tell me everything there is to know about **`numpy.ndarray`**<jupyter_code>help(rolls)<jupyter_output>Help on ndarray object:

class ndarray(builtins.object)
 |  ndarray(shape, dtype=float, buffer=None, offset=0,
 |          strides=None, order=None)
 |  
 |  An array object represents a multidimensional, homogeneous array
 |  of fixed-size items.  An associated data-type object describes the
 |  format of each element in the array (its byte-order, how many bytes it
 |  occupies in memory, whether it is an integer, a floating point number,
 |  or something else, etc.)
 |  
 |  Arrays should be constructed using `array`, `zeros` or `empty` (refer
 |  to the See Also section below).  The parameters given here refer to
 |  a low-level method (`ndarray(...)`) for instantiating an array.
 |  
 |  For more information, refer to the `numpy` module and examine the
 |  methods and attributes of an array.
 |  
 |  Parameters
 |  ----------
 |  (for the __new__ method; see Notes below)
 |  
 |  shape : tuple of ints
 |      Shape of created array.
 |  dtype : data-type, optional
 |      Any objec[...]
{'fraction_non_alphanumeric': 0.12999905455233052, 'fraction_numerical': 0.00538905171598752, 'mean_word_length': 4.590909090909091, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '289376', 'license': 'MIT', 'n_tokens_mistral': 3439, 'n_tokens_neox': 3298, 'n_words': 1235, 'path': 'python-for-data/Lec07 - Working with External Libraries.ipynb', 'repo_name': 'ngttnguyen/atom-assignments', 'size': '10577'}