<jupyter_start><jupyter_text>ModulesPython has a way to put definitions in a file so they can easily be reused. Such files are called a modules. You can define your own module (for instance see [here](https://docs.python.org/3/tutorial/modules.html)) how to do this but in this course we will only discuss how to useexisting modules as they come with python as the [python standard library](https://docs.python.org/3.9/tutorial/stdlib.html?highlight=library) or as third party libraries as they for instance are distributed through [anaconda](https://www.anaconda.com/) or your Linux distribution or you can install trough `pip` or from source. Definitions from a module can be imported into your jupyter notebook, your python script, and into other modules.The module is imported using the import statement. Here we import the mathematics library from the python standard library:<jupyter_code>import math<jupyter_output><empty_output><jupyter_text>You can find a list of the available functions, variables and classes using the `dir` method:<jupyter_code>dir(math)<jupyter_output><empty_output><jupyter_text>A particular function (or class or constant) can be called in the form `.`:<jupyter_code>math.exp(1)<jupyter_output><empty_output><jupyter_text>Documentation of a function (or class or constant) can be obtained using the `help` function (if the developer has written it):<jupyter_code>help(math.exp)<jupyter_output>Help on built-in function exp in module math:

exp(...)
    exp(x)
    
    Return e raised to the power of x.<jupyter_text>You can also import a specific function (or a set of functions) so you can directly use them without a prefix:<jupyter_code>from cmath import exp
exp(1j)<jupyter_output><empty_output><jupyter_text>In python terminology that means that  - in this case - the `exp` function  is imported into the main *name space*.This needs to be applied with care as existing functions (or class definition) with identical names are overwritten.For instance the `math` and the `cmath` module have a function `exp`. Importing both will create a problem in the main name space.  If you are conficdent in what you are doing you can import all functions and class definitions into the main name space:<jupyter_code>from cmath import *
cos(1.)<jupyter_output><empty_output><jupyter_text>Modules can contain submodules. The functions are then accessed `..`:<jupyter_code>import os
os.path.exists('FileHandling.ipynb')<jupyter_output><empty_output><jupyter_text>In these cases it can be useful to use an alias to make the code easier to read:<jupyter_code>import os.path as pth
pth.exists('FileHandling.ipynb')<jupyter_output><empty_output><jupyter_text>More on printingPython provides a powerful way of formatting output using formatted string.Basicly the ideas is that in a formatted string marked by a leading 'f' variablenames are replaced by the corresponding variable values. Here comes an example:<jupyter_code>x, y = 2.124, 3
f"the value of x is {x} and of y is {y}."<jupyter_output><empty_output><jupyter_text>python makes guesses on how to format the value of the variable but you can also be specific if values should be shown in a specific way. here we want to show `x` as a floating point numbers with a scientific number representation indicated by `e` and `y` to be shown as an integer indicated by `d`:<jupyter_code>f"x={x} x={x:10f} x={x:e} y={y:d}"<jupyter_output><empty_output><jupyter_text>More details on [Formatted string literals](https://docs.python.org/3.7/reference/lexical_analysis.htmlindex-24) Formatted strings are used to prettify output when printing:<jupyter_code>print(f"x={x:10f}")
print(f"y={y:10d}")<jupyter_output>x=  2.124000
y=         3<jupyter_text>An alternative way of formatting is the `format` method of a string. You can use the positional arguments:<jupyter_code>guest='John'
'Hi {0}, welcome to {1}!"'.format(guest, 'Brisbane')<jupyter_output><empty_output><jupyter_text>Or keyword arguments:<jupyter_code>'Hi {guest}, welcome to {place}!'.format(guest='Mike', place='Brisbane')<jupyter_output><empty_output><jupyter_text>and a combination of positional arguments and keyword arguments:<jupyter_code>'Hi {guest}, welcome to {1}! Enjoy your stay for {0} days.'.format(10, 'Brisbane',  guest="Bob")<jupyter_output><empty_output><jupyter_text>You can also introduce some formatting on how values are represented:<jupyter_code>'Hi {guest}, welcome to {0}! Enjoy your stay for {1:+10d} days.'.format('Brisbane', 10, guest="Bob")<jupyter_output><empty_output><jupyter_text>More details in particular for formating numbers are found [here](https://docs.python.org/3.9/library/string.html).  Writing and Reading filesTo open a file for reading or writing use the `open` function. `open()`returns a file object, and is most commonly used with two arguments: open(filename, mode).<jupyter_code>outfile=open("myRicker.csv", 'wt')<jupyter_output><empty_output><jupyter_text>It is commonly used with two arguments: `open(filename, mode)` where the `mode` takes the values:- `w` open for writing. An existing file with the same name will be erased.- `a` opens the file for appending; any data written to the file is automatically added to the end. - `r` opens the file for both reading only.By default text mode `t` is used that means, you read and write strings from and to the file, which are encoded in a specific encoding. `b` appended to the mode opens the file in binary mode: now the data is read and written in the form of bytes objects.  We want to write some code that writes the `Ricker` wavelet of a period of`length` and given frequency `f` to the files `myRicker.csv` in the comma-separated-value (CSV) format. The time is incremented by `dt`.<jupyter_code>length=0.128
f=25
dt=0.001
def ricker(t, f):
    """
    return the value of the Ricker wavelet at time t for peak frequency f
    """
    r = (1.0 - 2.0*(math.pi**2)*(f**2)*(t**2)) * math.exp(-(math.pi**2)*(f**2)*(t**2))
    return r
t=-length/2
n=0
while t < length/2:
    outfile.write("{0}, {1}\n".format(t, ricker(t, f)))
    t+=dt
    n+=1

print("{} records writen to {}.".format(n, outfile.name))<jupyter_output>128 records writen to myRicker.csv.<jupyter_text>You can download/open the file ['myRicker.csv'](myRicker.csv).** Notice ** There is an extra new line character `\n` at the of string in the `write` statement. This makes sure that separate rows can be identified in the file. Don't forget to close the file at the end:<jupyter_code>outfile.close()<jupyter_output><empty_output><jupyter_text>Now we want to read this back. First we need to open the file for reading:<jupyter_code>infile=open("myRicker.csv", 'r')<jupyter_output><empty_output><jupyter_text>We then can read the entire file as a string:<jupyter_code>content=infile.read()
content[0:100]<jupyter_output><empty_output><jupyter_text>In some cases it is more easier to read the file row-by-row. First we need to move back to the beginning of the file:<jupyter_code>infile.seek(0)<jupyter_output><empty_output><jupyter_text>Now we read the file line by line. Each line is split into the time and wavelet value which are collected as floats in two lists `times` and `ricker`:<jupyter_code>infile.seek(0)
line=infile.readline()
times=[]
ricker=[]
n=0
while len(line)>0:
    a, b=line.split(',')
    times.append(float(a))
    ricker.append(float(b))
    line=infile.readline()
    n+=1
print("{} records read from {}.".format(n, infile.name))<jupyter_output>128 records read from myRicker.csv.<jupyter_text>Notice that the end of file is reached when the read line is empty (len(line)=0). Then the loop is exited.<jupyter_code>time[:10]<jupyter_output><empty_output><jupyter_text>JSON FilesJSON files (JavaScript Object Notation) is an open-standard file format that uses human-readable text to transmit data objects consisting of dictionaries and lists. It is a very common data format, with a diverse range of applications in particular when exchanging data between web browsers and web services.A typical structure that is saved in JSON files are combinations of lists and dictionaries with string, integer and float entries. For instance<jupyter_code>course = [ { "name": "John", "age": 30, "id" : 232483948} ,
           { "name": "Tim", "age": 45, "id" : 3246284632} ]  
course<jupyter_output><empty_output><jupyter_text>The `json` module provides the necessary functionality to write `course` into file, here `course.json`:<jupyter_code>import json
json.dump(course, open("course.json", 'w'), indent=4)<jupyter_output><empty_output><jupyter_text>You can access the [course.json](course.json). Depending on your web browser the file is identified as JSON fileand presented accordingly. We can easily read the file back using the `load` method:<jupyter_code>newcourse=json.load(open("course.json", 'r'))<jupyter_output><empty_output><jupyter_text>This recovers the original list+dictionary structure:<jupyter_code>newcourse<jupyter_output><empty_output><jupyter_text>We can recover the names of the persons in the course:<jupyter_code>[ p['name'] for p in newcourse ]<jupyter_output><empty_output><jupyter_text>We can add new person to `newcourse`:<jupyter_code>newcourse.append({'age': 29, 'name': 'Jane', 'studentid': 2643746328})
newcourse<jupyter_output><empty_output><jupyter_text>VisualizationWe would like to plot the Ricker wavelet. The `matplotlib` library provides a convenient, flexible and powerful tool for visualization at least for 2D data sets. Here we can give only a very brief introduction with more functionality being presented as the course evolves. For a comprehensive documentation and list of examples we refer to the [matplotlib web page](https://matplotlib.org).Here we use the `matplotlib.pyplot` library which is a collection of command style functions but there is also a more general API which gives a reacher functionality:<jupyter_code>#%matplotlib notebook
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>It is very easy to plot data point we have read:<jupyter_code>plt.figure(figsize=(8,5))
plt.scatter(times, ricker)<jupyter_output><empty_output><jupyter_text>We can also plot this as a function rather than just data point:<jupyter_code>plt.figure(figsize=(8,5))
plt.plot(times, ricker)<jupyter_output><empty_output><jupyter_text>Let's use proper labeling of the horizontal axis:<jupyter_code>plt.xlabel('time [sec]')<jupyter_output><empty_output><jupyter_text>and for the vertical axis:<jupyter_code>plt.ylabel('aplitude')<jupyter_output><empty_output><jupyter_text>And maybe a title:<jupyter_code>plt.title('Ricker wavelet for frequency f = 25 hz')<jupyter_output><empty_output><jupyter_text>We can also change the line style, eg. red doted line:<jupyter_code>plt.figure(figsize=(8,5))
plt.plot(times, ricker, 'r:')
plt.xlabel('time [sec]')
plt.ylabel('aplitude')<jupyter_output><empty_output><jupyter_text>We can put different data sets or representations into the plot:<jupyter_code>plt.figure(figsize=(8,5))
plt.plot(times, ricker, 'r:', label="function")
plt.scatter(times, ricker, c='b', s=10, label="data")
plt.xlabel('time [sec]')
plt.ylabel('aplitude')
plt.legend()<jupyter_output><empty_output><jupyter_text>You can also add grid line to make the plot easier to read:<jupyter_code>plt.grid(True)<jupyter_output><empty_output><jupyter_text>Save the plot to a file:<jupyter_code>plt.savefig("ricker.png")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10481611208406305, 'fraction_numerical': 0.011908931698774081, 'mean_word_length': 6.260648442466624, 'pattern_counts': {'":': 4, '<': 154, '<?xml version=': 0, '>': 154, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '39', 'dir': 'jupyter-structured-clean-dedup', 'id': '9075', 'license': 'Apache-2.0', 'n_tokens_mistral': 3729, 'n_tokens_neox': 3588, 'n_words': 1262, 'path': 'A_Python/FilesAndVisualization.ipynb', 'repo_name': 'uqzzhao/Programming-Geophysics-in-Python', 'size': '11420'}