<jupyter_start><jupyter_text>Article Data EDA & Cleaning---The raw article dataset in 'articles.csv' has the following features and labels Labels- **bias** The bias score of the article- **quality** The reliability score of the article Features- **article_id** The article's unique ID number- **source_id** The unique ID number of the article's source- **url** - The article's URL- **score_count:** - ??? (I believe this may be the number of individual ratings of the article's bias and reliability)- **domain** - The domain of the article's source- **source** - The name of the article's source- **image_path** - The relative path of the source's logo for the website- **image_width** - The width of the source's logo image- **image_height** - The height of the source's logo image- **is_tv** - ???- **reach** - ???- **article_count** - The number of articles in the dataset from the article's source  Read in data<jupyter_code>import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
%matplotlib inline

articles = pd.read_csv('../data/interim/articles.csv')
articles.head()<jupyter_output><empty_output><jupyter_text>Drop unneeded features Currently, we know that we do not need the article_id, domain, image_path, image_width, and image_height features.<jupyter_code>to_drop = ['article_id', 'domain', 'image_path', 'image_width', 'image_height']
articles.drop(to_drop, axis=1, inplace=True)
articles.head()<jupyter_output><empty_output><jupyter_text>Since we're focused on the articles themselves, more than one feature describing their sources is redundant and including these features would increase training time as well as lead to overfitting. As a result, we will drop the source, is_tv, reach, and article_count features.<jupyter_code>to_drop = ['source_id', 'is_tv', 'reach', 'article_count']
articles.drop(to_drop, axis=1, inplace=True)
articles.head()
articles.to_csv('../data/interim/articles_sources_removed.csv', index=False)<jupyter_output><empty_output><jupyter_text>Explore Features<jupyter_code>articles.describe()
articles.groupby('source').mean()
fig, ax = plt.subplots()
sns.scatterplot(x='source', y='quality', hue='bias', data=articles.groupby('source').mean())
fig.set_size_inches(12, 12)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12538495380554335, 'fraction_numerical': 0.0026396832380114386, 'mean_word_length': 6.288461538461538, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '435987', 'license': 'MIT', 'n_tokens_mistral': 716, 'n_tokens_neox': 677, 'n_words': 232, 'path': 'notebooks/2.0-epm-initial-article-eda.ipynb', 'repo_name': 'vyathakavilocana/aiclubproject1newsfeed', 'size': '2273'}