<jupyter_start><jupyter_text>What datapoints are we going to collect about a country?- Country- Continent the country belongs to- Capital- Currency- Land Area- Population- Independence DayThese are primary things we are interested in.<jupyter_code>import requests
import pandas as pd
import numpy as np
from bs4 import BeautifulSoup as bs<jupyter_output><empty_output><jupyter_text>Let us first get the data of country and the capital<jupyter_code>url = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_and_their_capitals_in_native_languages'
dfs = pd.read_html(url, encoding='utf-8')<jupyter_output><empty_output><jupyter_text>dfs: 1-25 are useful<jupyter_code>dfs = dfs[1:26]
main_df = pd.concat(dfs, join='inner', ignore_index=True)
main_df<jupyter_output><empty_output><jupyter_text>Let us just focus on the first two columns<jupyter_code>data = main_df.iloc[:, :2]
data.head()<jupyter_output><empty_output><jupyter_text>Let us clean the column names now<jupyter_code>data.columns = ['country', 'capital']
data<jupyter_output><empty_output><jupyter_text>There are rows where we have [1] type things, we need to clean it up.<jupyter_code>import string

def clean_name(s: str) -> str:
    if not isinstance(s, str):
        return s
    t = ''
    for c in s:
        if (c not in string.ascii_letters) and (c not in ["'", '"', ',', ' ']):
            continue
        t += c
    return t 
clean_name('Afghanistan')
data['country'] = data['country'].apply(clean_name)
data['capital'] = data['capital'].apply(clean_name)<jupyter_output>/home/bruhh/.local/lib/python3.6/site-packages/ipykernel_launcher.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  """Entry point for launching an IPython kernel.
/home/bruhh/.local/lib/python3.6/site-packages/ipykernel_launcher.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
{'fraction_non_alphanumeric': 0.11235471373224279, 'fraction_numerical': 0.006887645286267757, 'mean_word_length': 5.367123287671233, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 25, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '164827', 'license': 'CC0-1.0', 'n_tokens_mistral': 814, 'n_tokens_neox': 791, 'n_words': 203, 'path': 'country_dataset.ipynb', 'repo_name': 'free-dataset/country', 'size': '2323'}