<jupyter_start><jupyter_text>Files - data is usually stored in secondary storage medium such as hard drive, flash drive, cd-rw, etc. using named locations called files - files can be organized into folders - use open() built-in function to work with files<jupyter_code>help(open)<jupyter_output><empty_output><jupyter_text>write data to a file- open file with a name- write data- close file<jupyter_code># old school
fw = open('test.txt', 'a') # w is for write mode
fw.write('words\n=====\n')
fw.write('apple\nball\ncat\ndog\n')
fw.write('elephant\n')
fw.close()
# newer and better syntax
with open('words.txt', 'w') as fw:
    fw.write('apple\nball\ncat\ndog\n')
    fw.write('elephant\n')
    fw.write('zebra\n')
# file will be automatically closed when with block is finished executing<jupyter_output><empty_output><jupyter_text>read data from a file- open file with its name; can provide relative or absolute path- read in various ways; one line at a time, all lines, bytes, whole file, etc.- use data- close file<jupyter_code># read whole file as list of lines
fr = open('words.txt', 'r') # 'r' or read mode by default; file must exist
data = fr.readlines()
fr.close()
help(fr)
data
data.sort(reverse=True)
with open('sorted_words.txt', 'w') as newFile: 
    for word in data:
        newFile.write(word)<jupyter_output><empty_output><jupyter_text>fetch a page from the web<jupyter_code>import urllib.request
url = 'http://org.coloradomesa.edu/~rbasnet/teaching.html'
localfile = 'teaching.html'
urllib.request.urlretrieve(url, localfile)<jupyter_output><empty_output><jupyter_text>reading the whole file at once<jupyter_code># read /usr/share/dict/words file in linux
# windows path might be " c:/temp/words.txt" or c:\\temp\words.txt"
with open(localfile) as f:
    data = f.read()
words = data.split(' ')
print('There are {0} words in the file.'.format(len(words)))
print(words[:10])<jupyter_output>['<!DOCTYPE', 'html>\n<html', 'lang="en">\n<head>\n<meta', 'charset="utf-8"', '/>\n<title>Teaching', 'Interests', 'and', 'Current', 'Semester', 'Schedule</title>\n<link']<jupyter_text>parsing HTML using BeautifulSoup library- https://www.crummy.com/software/BeautifulSoup/bs4/doc/- Alternative is nltk (Natural Language Toolkit) library- http://www.nltk.org/<jupyter_code>from bs4 import BeautifulSoup
localfile = 'teaching.html'
with open(localfile) as f:
    soup = BeautifulSoup(f.read(), 'lxml')
text = soup.get_text()
print(text)
# break into lines and remove leading and trailing space on each line
lines = [line.strip() for line in text.splitlines()]
print(lines)
# create list of words by spliting multi-word elements
words = [word.strip() for line in lines for word in line.split()]
print(words)
print('There are {0} words in the file.'.format(len(words)))<jupyter_output>There are 185 words in the file.<jupyter_text>working with binary files<jupyter_code>with open('cmu-logo.png', 'rb') as rbf: #rb - read binary mode
    data = rbf.read() # read the whole binary file
    with open('cmu-logo-copy.png', 'wb') as wbf:
        wbf.write(data) # write the whole binary file
    
print(len(data))
# find the size of the data in bytes
import sys
print(sys.getsizeof(data))<jupyter_output>14088
{'fraction_non_alphanumeric': 0.1384471468662301, 'fraction_numerical': 0.004677268475210477, 'mean_word_length': 5.110476190476191, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 2, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '319075', 'license': 'MIT', 'n_tokens_mistral': 1115, 'n_tokens_neox': 1089, 'n_words': 329, 'path': 'Ch13-Files.ipynb', 'repo_name': 'rjhakes/rhakesAdvPy', 'size': '3207'}