<jupyter_start><jupyter_text>Lesson 3: Function and IO operation>  catalogue>1. [Function](function) >2. [File operation](file)>3. [IO of csv, json, picture](csv) 1. Function     Function definition `def`>A function is a block of code which only runs when it is called. You can call this function by passing parameters into a function. Then the function can return data as a result.```pythondef function_name(parameters): define function    control flow use control flow    return  return to default or specified valuefunction_name("parameter1") call the function``` Function syntax rules>1. function announcement `def`>2. name: uniquely identify it, just behind the function announcement.>3. parameters: `required`, `*args`, `**kwargs`>4. A colon `:` to mark the end of function header.>5. content: Describe what the function does.>6. An optional return statement to return a value from the function. > Example 1: > absolute function<jupyter_code>def odds(number):
    return list(range(1,number,2))

def odds_2(number):
    if not isinstance(number,int):
        return "wrong type"
    else:
        return [i for i in range(1,number,2)]

def odds_3(number):
    try:
        return odds_2(number)
    except Exception as e:
        print(e)

def my_function(number):
    result = []
    i = 1
    while i<= number:
        result.append(i)
        i += 2
        
    return result


def abs_1(number):
    try:
        if number > 0:
            return number
        else:
            return -number
    except TypeError:
        print("please input a number")


abs_1("a")
abs(-2.3)
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

def my_abs(x):
    if x >= 0:
        return x
    else:
        return -x

#error handle
def my_abs2(x):
    try:
        my_abs(x)
    except TypeError:
        print("please input a number")

def my_abs3(x):
    if not isinstance(x, (int, float)):
        raise TypeError('bad operand type')
    if x >= 0:
        return x
    else:
        return -x

my_abs3("a")<jupyter_output><empty_output><jupyter_text>- Example 2: string split function>requirement：split any sentence to words.> input: sentence as a string> output: words as a list<jupyter_code>def string_split(string):
    for i in list(',.:;""!'):
        string = string.replace(i," ")
        
    splited_list = string.split(" ")
    
    return return_list

string_split("So great the day is.")

def split_sentence(string):
    replaced_string = string.replace(","," ")
    splited_list = replaced_string.split(" ")
    output = []
    for i in splited_list:
        if i !="":
            output += [i]
    return output

''' another method
    for i in split_list:
        try:
            splited_list.remove("")
        except ValueError:
            return splited_list
'''          
split_sentence("President <NAME> declared Tuesday that can shake China as he oversaw a huge military parade to celebrate 70 years of Communist Party rule.")<jupyter_output><empty_output><jupyter_text>>* Example 3>Find the solution of the two order equation>$ax^2+bx+c = 0$<jupyter_code>def quadratic(a, b, c):
    delta = (b**2-4*a*c)**0.5
    return (-b+delta)/(2*a),(-b-delta)/(2*a)

quadratic(1,1,2)<jupyter_output><empty_output><jupyter_text>>* Example 4>Return summation of a series of number using `*args` parameters `*args` will convert user's input as a list. We usually use loop to handle it.<jupyter_code>def sumit(*args):
    result = 0
    for i in args:
        result += i
    return result

sumit(1,2,3,4,5,3,4,5,6,7)
def summation(*args):
    result = 0
    for i in args:
        result += i
    return result



# def summation(*args):
#     result = 0
#     for arg in args:
#         try:
#             result += arg
#         except TypeError:
#             print("please input numbers")
#             return None
#     return result

summation(1,2,3,4,5)<jupyter_output><empty_output><jupyter_text>>* Example 5>Format print name and attribute using `*kwargs` parameters `*kwargs` will convert user's input as a dict.<jupyter_code>def format_print(**kwags):
    for k,v in kwags.items():
        print("The name is %s, the value is %s"%(k,v))
        
format_print(a=3,b=5,c=7)<jupyter_output>The name is a, the value is 3
The name is b, the value is 5
The name is c, the value is 7<jupyter_text>2. File operation We will learn how to get data, convert data and save data.> 1. Most of the file operation can be complete by `os` module.> We firt introduce some common method of `os` module.Function|Describe---|---os.getcwd()| Gets the string of the current working pathos.listdir('dirname') | List all file names and dictionary name of selected pathos.remove('filename') | delete file from current pathos.rename("oldname","newname") | rename fileos.path.split(path) | splite path and file nameos.path.join(path,name) | joint path and file name>2. `open`  function open the file and regard it as a `file` object       ```pythonfile_object = open(file_name [, access_mode])file_object.colse()file_object.write("data")file_object.readline()file.writelines(sequence)```   >* file_name contains path and file name   >* access_mode includemode|describe---|---`r`| read only(default mode)`w`| overwrite a file`a`| append data at end of file`b`| open files in binary mode`x`| new a file  The difference of those mode can be presented as below:![picture](https://www.runoob.com/wp-content/uploads/2013/11/2112205-861c05b2bdbc9c28.png)> **attributes of file object**>* file.closed>* file.mode>* file.name>* file.softspace<jupyter_code>file = open("new_file.txt","w+")
for i in range(1,101):
    file.write('%s\n'%i)
file.close()

import os
#current path
path = os.getcwd()
path

#list dictionary and name
os.listdir(path)

#new a file at current path
new_file = open("test.txt","w") 

#list dictionary and name
os.listdir(path)

#get absolute path
abs_path = os.path.join(path, "test.txt")
abs_path

#split path
path, file_name = os.path.split(abs_path)
path
file_name

#close file object
new_file.close()
new_file.closed

#delete test file
os.remove(file_name)

#list dictionary and name
os.listdir(path)<jupyter_output><empty_output><jupyter_text>IO of json, csv , picture 1. JSON (JavaScript Object Notation) > a lightweight data interchange format inspired by JavaScript object literal syntax.>* JSON is a syntax for storing and exchanging data.>* JSON is text, written with JavaScript object notation.```javascript{    "firstName": "Jane",    "lastName": "Doe",    "hobbies": ["running", "sky diving", "singing"],    "age": 35,    "children": [        {            "firstName": "Alice",            "age": 6        },        {            "firstName": "Bob",            "age": 8        }    ]}```>In simple terms, JSON is a dict, which has keys, each key corresponds to a value. The middle is separated by `:` , the outermost is surrounded by `{}`, and the different key-value pairs are separated by `,`. Example like this```python{'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}```If there is a case where a Key corresponds to multiple values, use [] to include all the corresponding values.```python{'key1': ['v11', 'v12', 'v13'], 'key2': 'v22'}```> **We convert json to string or dict for different purpose**function | describe---|---json.dumps("json")| convert json to stringjson.loads("dict") | convert string to string<jupyter_code>import json

data = [ { 'a' : 1, 'b' : 2, 'c' : 3, 'd' : 4, 'e' : 5 } ]
type(data)

js = json.dumps(data)
type(json)

convert_data = json.loads(js)
type(convert_data)

type(convert_data[0])

convert_data == data<jupyter_output><empty_output><jupyter_text>2. CSV (Comma Separated Values) >The CSV (Comma Separated Values) format is the most common import and export format for spreadsheets and databases,just like `xlsx` file, which is used for storing your data and make it easy to read and write. You don't need to pip3 install csv. Just `import csv` before using it.1. **Read csv**`csv.reader````pythonimport csvwith open('test.csv','r') as f:  open CSV    rows = csv.reader(f)  read CSV    for row in rows: loop every row        print(row)```**Note:*** `with open(...) as f` means you give f a definition, which stands for opening the file. In the example,`f` can be changed by any words you like. It just means you rename the step of the opening. It's equal to `f = open('chapter4-example-name_list.csv',mode='r')`.* `open()` means open the file. If there is no such file, it will create a new one. If there is a existing one, writer function will clear all the previous content and then write the new content.2. **write csv**`csv.writer()````pythonimport csvwith open('name.csv','w') as f:    mywriter=csv.writer(f)  writer is the function to write something    mywriter.writerow(['Chico','Male'])  you can just use writer.writerow()    mywriter.writerow(['Ri','Female']) write another row  ```**Note:*** `w` means write. By the way, if you want to read the file, you can input`r`,representing "read".* `csv.writer()` means to write something in the name.csv file.* `writerow()` means write one row and then another row. The input should be list type. `writerows()` means they will write row after row until loop all the elements from a list.* arguments in `writerow()` should be a list, because `csv` function treat a row as a list, therefore you should use`[]` to wrap up the argument.3. **using pandas**> In fact, I always use pandas to process csv data and excel data<jupyter_code>import pandas as pd
import numpy as np
import os

os.listdir(os.getcwd())
a = np.arange(16).reshape(4,-1)
b = pd.DataFrame(a)
b.to_csv("test.csv")
print(b)
os.listdir(os.getcwd())

with open("test.csv","r") as f:
    for line in f:
        print(line)<jupyter_output>0   1   2   3
0   0   1   2   3
1   4   5   6   7
2   8   9  10  11
3  12  13  14  15
,0,1,2,3

0,0,1,2,3

1,4,5,6,7

2,8,9,10,11

3,12,13,14,15<jupyter_text>3.  Picture>We treat picture as binary data, so byte string should be used in the process> `PIL` module is the standard python process library,   3. Jpeg (Joint Photographic Experts Group) > we treat jpeg as binary data. If you need to download picture from website, just write binary data into empty file.```pythonwhth open("test.jpg","wb") as f:    f.write(byte_data)```> `PIL` module is the standrad library for image process.```pythonfrom PIL import Imageim = Image.open('test.jpg')im.show()im.save('code.jpg', 'jpeg')```>The most important class in the Python Imaging Library is the `Image` class, defined in the module with the same name.function|describe---|---Image.open("path")|load an image from a fileim.show()| display the imageim.save("path")|save the image<jupyter_code>/bizhi/7900_98023_2.html
/bizhi/7899_98013_2.html
/bizhi/7898_97996_2.html
/bizhi/7897_97986_2.html
/bizhi/7896_97974_2.html
/bizhi/7895_97956_2.html
/bizhi/7894_97950_2.html
/bizhi/7893_97935_2.html
/bizhi/7892_97927_2.html
/bizhi/7891_97921_2.html
/bizhi/7890_97904_2.html
/bizhi/7889_97896_2.html
/bizhi/7888_97884_2.html
/bizhi/7887_97872_2.html
/bizhi/7886_97859_2.html
/bizhi/7885_97854_2.html
/bizhi/7884_97835_2.html
/bizhi/7883_97823_2.html
/bizhi/7882_97817_2.html
/bizhi/7881_97807_2.html
/bizhi/7880_97795_2.html
import requests as rq
from PIL import Image
from lxml import etree

out = rq.get("http://desk.zol.com.cn/pc/")
html = etree.HTML(out.text)
alist = html.xpath('//img/@src')



for j,i in enumerate(alist):
    jpg = rq.get(i)
    with open("%s.jpg"%j,"wb") as f:
        f.write(jpg.content)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12540025962786672, 'fraction_numerical': 0.03556901774123756, 'mean_word_length': 3.7930319369556202, 'pattern_counts': {'":': 10, '<': 37, '<?xml version=': 0, '>': 84, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '394719', 'license': 'MIT', 'n_tokens_mistral': 4048, 'n_tokens_neox': 3764, 'n_words': 1288, 'path': 'Lesson_3.ipynb', 'repo_name': 'silenceon/FML_lesson_1', 'size': '11559'}