<jupyter_start><jupyter_text>Runner level<jupyter_code>import requests
import pandas as pd
from bs4 import BeautifulSoup
import re 
runner_link = 'https://www.parkrun.org.uk/results/athleteresultshistory/?athleteNumber=501501'
headers = {'User-Agent':'APIs-Google (+https://developers.google.com/webmasters/APIs-Google.html)'}
runner_soup = BeautifulSoup(requests.get(runner_link, 
                                         headers = headers).text, features="html.parser")
runner_title = re.compile('([a-zA-Z]* [a-zA-Z]*) \(([0-9]*) parkruns\)').search(runner_soup.find_all('h2')[0].text)
name = runner_title.group(1)
name
total_park_runs = runner_title.group(2)
total_park_runs<jupyter_output><empty_output><jupyter_text>Getting the main tables from `h1` tagWe can see that all the main tables on the runner summary page have the same structure. Contained within a `div` element is an `h1` tag and then a table below. We can define a function to pull this using the text within the `h1` tag.<jupyter_code>def table_from_h1(runner_soup, h1_tag):
    for i in runner_soup.find_all('h1'):
        if i.get_text() == h1_tag:
            table = i.find_parent('div').find('table')
    col_headers = [x.text for x in table.find('thead').find_all('th')]
    data = [[y.text for y in x.find_all('td')] for x in table.find_all('tr') if x.find('td')]
    return(pd.DataFrame(data, columns = col_headers))
table_from_h1(runner_soup, 'Most Recent Runs')
table_from_h1(runner_soup, 'Event Summaries')
table_from_h1(runner_soup, 'Volunteer Summary')<jupyter_output><empty_output><jupyter_text>Runner results by parkrun courseTo work on this we are going to need slightly more info from the tables, so probably cannot use our generic function.<jupyter_code>for i in runner_soup.find_all('h1'):
    if i.get_text() == 'Event Summaries':
        table = i.find_parent('div').find('table')
col_headers = [x.text for x in table.find('thead').find_all('th')]
data = [[y.text for y in x.find_all('td')] for x in table.find_all('tr') if x.find('td')]
history_links  = [[y.find('a').get('href') for y in x.find_all('td') if y.find('a') and 'athletehistory' in y.find('a').get('href')] for x in table.find_all('tr') if x.find('td')]
[x[0] for x in history_links if len(x)]
pd.concat([pd.DataFrame(data, columns = col_headers), pd.Series([x[0] for x in history_links if len(x)], name='History Link')], axis = 1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14859271523178808, 'fraction_numerical': 0.011175496688741722, 'mean_word_length': 5.027431421446384, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '102580', 'license': 'MIT', 'n_tokens_mistral': 867, 'n_tokens_neox': 846, 'n_words': 219, 'path': 'runner.ipynb', 'repo_name': 'willcanniford/park-run', 'size': '2416'}