<jupyter_start><jupyter_text>Looping Over Data Sets---- "How can I process many data sets with a single command?"objectives:- "Be able to read and write globbing expressions that match sets of files."- "Use glob to create lists of files."- "Write for loops to perform operations on files given their names in a list."keypoints:- "Use a `for` loop to process files given a list of their names."- "Use `glob.glob` to find sets of files whose names match a pattern."- "Use `glob` and `for` to process batches of files."  Use a `for` loop to process files given a list of their names.*   A filename is a character string.*   And lists can contain character strings.<jupyter_code>import pandas as pd
for filename in ['data/gapminder_gdp_africa.csv', 'data/gapminder_gdp_asia.csv']:
    data = pd.read_csv(filename, index_col='country')
    print(filename, data.min())<jupyter_output>data/gapminder_gdp_africa.csv gdpPercap_1952    298.846212
gdpPercap_1957    335.997115
gdpPercap_1962    355.203227
gdpPercap_1967    412.977514
gdpPercap_1972    464.099504
gdpPercap_1977    502.319733
gdpPercap_1982    462.211415
gdpPercap_1987    389.876185
gdpPercap_1992    410.896824
gdpPercap_1997    312.188423
gdpPercap_2002    241.165877
gdpPercap_2007    277.551859
dtype: float64
data/gapminder_gdp_asia.csv gdpPercap_1952    331.0
gdpPercap_1957    350.0
gdpPercap_1962    388.0
gdpPercap_1967    349.0
gdpPercap_1972    357.0
gdpPercap_1977    371.0
gdpPercap_1982    424.0
gdpPercap_1987    385.0
gdpPercap_1992    347.0
gdpPercap_1997    415.0
gdpPercap_2002    611.0
gdpPercap_2007    944.0
dtype: float64<jupyter_text>Use [`glob.glob`](https://docs.python.org/3/library/glob.htmlglob.glob) to find sets of files whose names match a pattern.*   In Unix, the term "globbing" means "matching a set of files with a pattern".*   The most common patterns are:    *   `*` meaning "match zero or more characters"    *   `?` meaning "match exactly one character"*   Python's standard library contains the [`glob`](https://docs.python.org/3/library/glob.html) module to provide pattern matching functionality*   The [`glob`](https://docs.python.org/3/library/glob.html) module contains a function also called `glob` to match file patterns*   E.g., `glob.glob('*.txt')` matches all files in the current directory     whose names end with `.txt`.*   Result is a (possibly empty) list of character strings.<jupyter_code>import glob
print('all csv files in data directory:', glob.glob('data/*.csv'))
print('all PDB files:', glob.glob('*.pdb'))<jupyter_output>all PDB files: []<jupyter_text>Use `glob` and `for` to process batches of files.*   Helps a lot if the files are named and stored systematically and consistently    so that simple patterns will find the right data.<jupyter_code>for filename in glob.glob('data/gapminder_*.csv'):
    data = pd.read_csv(filename)
    print(filename, data['gdpPercap_1952'].min())<jupyter_output>data\gapminder_all.csv 298.8462121
data\gapminder_gdp_africa.csv 298.8462121
data\gapminder_gdp_americas.csv 1397.7171369999999
data\gapminder_gdp_asia.csv 331.0
data\gapminder_gdp_europe.csv 973.5331947999999
data\gapminder_gdp_oceania.csv 10039.595640000001
{'fraction_non_alphanumeric': 0.09376966645689112, 'fraction_numerical': 0.10604153555695406, 'mean_word_length': 4.528695652173913, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '608082', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1329, 'n_tokens_neox': 1093, 'n_words': 326, 'path': '02_Day_2_Python_GIS/exercise/B7_Exercise_Loop_Datasets.ipynb', 'repo_name': 'TU-Delft-DCC/Intro-to-Python-forGIS', 'size': '3178'}