<jupyter_start><jupyter_text>gzip, zipfile, tarfile 模块：处理压缩文件<jupyter_code>import os, shutil, glob
import zlib, gzip, bz2, zipfile, tarfile<jupyter_output><empty_output><jupyter_text>gzip   zilb 模块 `zlib` 提供了对字符串进行压缩和解压缩的功能：<jupyter_code>orginal = "this is a test string"

compressed = zlib.compress(orginal)

print compressed
print zlib.decompress(compressed)<jupyter_output>x�+��,V�D�����⒢̼tS��
this is a test string<jupyter_text>同时提供了两种校验和的计算方法：<jupyter_code>print zlib.adler32(orginal) & 0xffffffff
print zlib.crc32(orginal) & 0xffffffff<jupyter_output>4236695221<jupyter_text>gzip 模块 `gzip` 模块可以产生 `.gz` 格式的文件，其压缩方式由 `zlib` 模块提供。我们可以通过 `gzip.open` 方法来读写 `.gz` 格式的文件：<jupyter_code>content = "Lots of content here"
with gzip.open('file.txt.gz', 'wb') as f:
    f.write(content)<jupyter_output><empty_output><jupyter_text>读：<jupyter_code>with gzip.open('file.txt.gz', 'rb') as f:
    file_content = f.read()

print file_content<jupyter_output>Lots of content here<jupyter_text>将压缩文件内容解压出来：<jupyter_code>with gzip.open('file.txt.gz', 'rb') as f_in, open('file.txt', 'wb') as f_out:
    shutil.copyfileobj(f_in, f_out)<jupyter_output><empty_output><jupyter_text>此时，目录下应有 `file.txt` 文件，内容为：<jupyter_code>with open("file.txt") as f:
    print f.read()
os.remove("file.txt.gz")<jupyter_output><empty_output><jupyter_text>bz2 模块 `bz2` 模块提供了另一种压缩文件的方法：<jupyter_code>orginal = "this is a test string"

compressed = bz2.compress(orginal)

print compressed
print bz2.decompress(compressed)<jupyter_output>BZh91AY&SY*�v	��@"� 10"zi��FLT`�軒)�P�˰
this is a test string<jupyter_text>zipfile 模块 产生一些 `file.txt` 的复制：<jupyter_code>for i in range(10):
    shutil.copy("file.txt", "file.txt." + str(i))<jupyter_output><empty_output><jupyter_text>将这些复制全部压缩到一个 `.zip` 文件中：<jupyter_code>f = zipfile.ZipFile('files.zip','w')

for name in glob.glob("*.txt.[0-9]"):
    f.write(name)
    os.remove(name)
    
f.close()<jupyter_output><empty_output><jupyter_text>解压这个 `.zip` 文件，用 `namelist` 方法查看压缩文件中的子文件名：<jupyter_code>f = zipfile.ZipFile('files.zip','r')
print f.namelist()<jupyter_output>['file.txt.9', 'file.txt.6', 'file.txt.2', 'file.txt.1', 'file.txt.5', 'file.txt.4', 'file.txt.3', 'file.txt.7', 'file.txt.8', 'file.txt.0']<jupyter_text>使用 `f.read(name)` 方法来读取 `name` 文件中的内容：<jupyter_code>for name in f.namelist():
    print name, "content:", f.read(name)

f.close()<jupyter_output>file.txt.9 content: Lots of content here
file.txt.6 content: Lots of content here
file.txt.2 content: Lots of content here
file.txt.1 content: Lots of content here
file.txt.5 content: Lots of content here
file.txt.4 content: Lots of content here
file.txt.3 content: Lots of content here
file.txt.7 content: Lots of content here
file.txt.8 content: Lots of content here
file.txt.0 content: Lots of content here<jupyter_text>可以用 `extract(name)` 或者 `extractall()` 解压单个或者全部文件。  tarfile 模块 支持 `.tar` 格式文件的读写：例如可以这样将 `file.txt` 写入：<jupyter_code>f = tarfile.open("file.txt.tar", "w")
f.add("file.txt")
f.close()<jupyter_output><empty_output><jupyter_text>清理生成的文件：<jupyter_code>os.remove("file.txt")
os.remove("file.txt.tar")
os.remove("files.zip")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16656091370558376, 'fraction_numerical': 0.015545685279187817, 'mean_word_length': 6.8825, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 51, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '479819', 'license': 'MIT', 'n_tokens_mistral': 1533, 'n_tokens_neox': 1512, 'n_words': 233, 'path': 'lijin-THU:notes-python/11-useful-tools/11.06-gzip,-zipfile,-tarfile.ipynb', 'repo_name': 'Maecenas/python-getting-started', 'size': '3152'}