<jupyter_start><jupyter_text>Chapter 9  1.File objects can be used to access not only normal disk files, but also any other type of “file” that uses that abstraction.The open() built-in function (see below) returns a file object that is then used for all succeeding operations on the file in question  2.As the key to opening file doors, the open() [and file()] built-in function provides a general interface to initiate the file input/output (I/O) process.* The open() BIF returns a file object on a successful opening of the file or else results in an error situation* file_object = open(file_name, access_mode='r', buffering=-1)* A 'U' mode also exists for universal NEWLINE support (see below).* Any file opened with mode 'r' or 'U' must exist.* Any file opened with 'w' will be truncated (clean the data) first if it exists, and then the file is (re)created.* All writes to files opened with 'a' will be from end-of-file, even if you seek elsewhere during access.* If the file does not exist, it will be created, making it the same as if you opened the file in 'w' mode.* If access_mode is not given, it defaults automatically to 'r.'* The other optional argument, buffering, is used to indicate the type of buffering that should be performed when accessing the file.     * A value of 0 means no buffering should occur,     * a value of 1 signals line buffering,     * and any value greater than 1 indicates buffered I/O with the given value as the buffer size.    * greater than 1 indicates buffered I/O with the given value as the buffer size.    * The lack of or a negative value indicates that the system default bufferingscheme should be used, which is line buffering for any teletype or tty-like device and normal buffering for everything else.  3. file() factory function * Both open() and file() do exactly the same thing and one can be used in place of the other. Anywhere you see references to open(), you can mentally substitute file() without any side effects whatsoever. * Generally, the accepted style is that you use open() for reading/writing files, while file() is best used when you want to show that you are dealing with file objects, i.e., if instance(f, file).  4.Universal NEWLINE Support (UNS) * os module can help you navigate files across different platforms, all of which terminate lines with different endings, i.e., \n, \r, or \r\n. * When you use the 'U' flag to open a file, all line separators (or terminators) will be returned by Python via any file input method, i.e., read*(), as a NEWLINE character ( \n ) regardless of what the line-endings are. * This feature will also support files that have multiple types of line-endings. A file.newlines attribute tracks the types of line separation characters “seen.” * file.newlines is a list that includes all the terminators it meet, otherwise it's None. Note that UNS only applies to reading text files. There is no equivalent handling of file output.  5. File methods come in four different categories: input, output, movement within a file, which we will call “intra-file motion,” and miscellaneous.  a. input * read(size_of_bytes=-1):  read bytes directly into a string, reading at most the number of bytes indicated. If no size is given (the default value is set to integer -1) or size is negative, the file will be read to the end. * readline(size_fo_bytes=-1): read a line and return the line ( including the line-terminator), size_to_read defaults to -1, mean- ing read until the line-ending characters (or EOF) are found. If present, it is possible that an incomplete line is returned if it exceeds size bytes. * readlines(sizhint):  it reads all (remaining) lines and returns them as a list of strings. sizhint, is a hint on the maximum size desired in bytes. If provided and greater than zero, approximately sizhint bytes in whole lines are read (perhaps slightly more to round up to the next buffer size) and returned as a list.  b. output * write(): It takes a string that can consist of one or more lines of text data or a block of bytes and writes the data to the file. * writelines(): operates on a list just like readlines(), but takes a list of strings and writes them out to a file. Line termination charac- ters are not inserted between each line, so if desired, they must be added to the end of each line before writelines() is called. * When reading lines in from a file using file input methods like read() or readlines(), Python does not remove the line termination characters. It is up to the programmer. * Similarly, output methods like write() or writelines() do not add line terminators for the programmer... you have to do it yourself before writing the data to the file.  C. intra-file motion * seek() method (analogous to the fseek() function in C) moves the file pointer to different positions within the file.  * The offset in bytes is given along with a relative offset location, whence.  * A value of 0, the default, indicates dis-tance from the beginning of a file (note that a position measured from the beginning of a file is also known as the absolute offset), * a value of 1 indicates movement from the current location in the file, and  * a value of 2 indicates that the offset is from the end of the file.  Use of the seek() method comes into play when opening a file for read and write access. * tell() is a complementary method to seek(); it tells you the current location of the file—in bytes from the beginning of the file.  5. file iteration:<jupyter_code>for eachLine in f: # f is an iterator, f.next()read in the next line
    pass
#Going through a file line by line is simple:<jupyter_output><empty_output><jupyter_text>In file iteration, file objects became their own iterators, meaning that users could now iterate through lines of a file using a for loop without having to call read*() methods. Alternatively, the iterator next method, file.next() could be called as well to read in the next line in the file. Like all other iterators, Python will raise StopIteration when no more lines are available.  6.close file * close() method completes access to a file by closing it. * The Python gar- bage collection routine will also close a file when the file object reference has decreased to zero. * The fileno() method passes back the file descriptor to the open file. * Rather than waiting for the (contents of the) output buffer to be written to disk, calling the flush() method will cause the contents of the internal buffer to be written (or flushed) to the file immediately. * isatty() is a Boolean built-in method that returns True if the file is a tty-like device and False otherwise.  * The truncate() method truncates the file to the size at the current file position or the given size in bytes.  7. os module* linesep: String used to separate lines in a file* sep: String used to separate file pathname components* pathsep: String used to delimit a set of file pathnames* curdir: String name for current working directory* pardir: String name for parent (of current working directory)  comma placed at the end of the print statement is to suppress the NEWLINE character that print normally adds at the end of output.  truncate() method, which takes one optional argument, size. If it is given, then the file will be truncated to, at most, size bytes. If you call truncate() without passing in a size, it will default to the current location in the file.<jupyter_code>import os
filename = input('Enter file name: ') 
fobj = open(filename, 'w')
while True:
    aLine = input("Enter a line ('.' to quit): ") 
    if aLine != ".":
        fobj.write('%s%s' % (aLine, os.linesep)) 
    else:
        break
fobj.close()<jupyter_output>Enter file name: test_input.txt
Enter a line ('.' to quit): This is a beautiful girl!
Enter a line ('.' to quit): agaghoahg asgha.
Enter a line ('.' to quit): .
{'fraction_non_alphanumeric': 0.06013021830716201, 'fraction_numerical': 0.002680965147453083, 'mean_word_length': 4.147174770039422, 'pattern_counts': {'":': 1, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '529196', 'license': 'Apache-2.0', 'n_tokens_mistral': 1960, 'n_tokens_neox': 1914, 'n_words': 1280, 'path': 'Chapter9.ipynb', 'repo_name': 'yangzhou95/notes', 'size': '7833'}