<jupyter_start><jupyter_text>---layout: posttitle:  "AO3 Trivia: Popular Tags Part II"date:   2021-06-10categories: visualizationtags: Python Pandas Circlify Regex Matplotlib--- In this section, we use the python [Circlify](https://github.com/elmotec/circlify) library and matplotlib to visualize the popular tags on AO3. We'll be following the [Basic Circle Packing Chart](https://www.python-graph-gallery.com/circular-packing-1-level-hierarchy) example on Python Graph Gallery.* Table of Contents{:toc}  Loading File<jupyter_code># Load python libraries
import pandas as pd<jupyter_output><empty_output><jupyter_text>Here we use a csv file "fandom500.csv", it is the top 500 most popular fandoms based on cached_count extracted from the original file tags-20210226.csv. The previous post covers how to find popular tags.<jupyter_code># Load data
# Only load first 20 rows from the file
fandom = pd.read_csv("fandom500.csv", nrows=20)
# preview
fandom[:5]<jupyter_output><empty_output><jupyter_text>CirclifyWe use [Circlify](https://github.com/elmotec/circlify) to generate a basic circle packing chart representing the 20 most popular fandoms on AO3.<jupyter_code>%matplotlib inline

# Load matplotlib
import matplotlib.pyplot as plt

# Load the circlify library
import circlify
# Data is a list of positive values sorted from largest to smallest.
# Compute circle positions:
circles = circlify.circlify(
    fandom.cached_count.tolist(), 
    show_enclosure=False, 
    target_enclosure=circlify.Circle(x=0, y=0, r=1)
)
# Preview
circles[:10]
# Visualization with circlify.Bubbles()
circlify.bubbles(circles=circles)<jupyter_output><empty_output><jupyter_text>The default function circlify.Bubbles() generates a circle packing chart with the cached_count. It is difficult to add annotations and other customization on the chart. We'll follow the [Basic Circle Packing Chart](https://www.python-graph-gallery.com/circular-packing-1-level-hierarchy) example on Python Graph Gallery to plot the chart with matplotlib.  Matplotlib Circles<jupyter_code># Create just a figure and only one subplot
# Must use plt.subplots instead of subplot
# In Jupyter notebook, include all code in one cell

fig, ax = plt.subplots(figsize=(10,10))

# Find axes boundaries
# This is due to ax.add_patch() not updating boundaries
# Thus to avoid drawing circles outside of fig

lim = max([max(abs(circle.x)+circle.r, abs(circle.y)+circle.r) for circle in circles])
plt.xlim(-lim,lim)
plt.ylim(-lim,lim)

# List of labels to add on the circles
labels = fandom.name

# Remove axis
# Add title

plt.axis('off')
plt.title('AO3 Popular Fandoms')

# print circles

for circle, label in zip(circles, labels):
    x, y, r = circle
    ax.add_patch(plt.Circle((x,y), r, alpha=0.2,linewidth=2))
    plt.annotate(label, (x,y), va='center',ha='center')<jupyter_output>/usr/local/lib/python3.7/dist-packages/matplotlib/backends/backend_agg.py:238: RuntimeWarning: Glyph 20693 missing from current font.
  font.set_text(s, 0.0, flags=flags)
/usr/local/lib/python3.7/dist-packages/matplotlib/backends/backend_agg.py:238: RuntimeWarning: Glyph 12398 missing from current font.
  font.set_text(s, 0.0, flags=flags)
/usr/local/lib/python3.7/dist-packages/matplotlib/backends/backend_agg.py:238: RuntimeWarning: Glyph 12498 missing from current font.
  font.set_text(s, 0.0, flags=flags)
/usr/local/lib/python3.7/dist-packages/matplotlib/backends/backend_agg.py:238: RuntimeWarning: Glyph 12540 missing from current font.
  font.set_text(s, 0.0, flags=flags)
/usr/local/lib/python3.7/dist-packages/matplotlib/backends/backend_agg.py:238: RuntimeWarning: Glyph 12525 missing from current font.
  font.set_text(s, 0.0, flags=flags)
/usr/local/lib/python3.7/dist-packages/matplotlib/backends/backend_agg.py:238: RuntimeWarning: Glyph 12450 missing from current font.
  font.set_[...]<jupyter_text>We have successfully added the labels to the circles. However the text string is too long and not very visually pleasing. We're going to shorten the labels with regular expression.  Regular Expression Here's the plan. We want to get rid of some addtional information on the labels. We want to match string before the parentheses, or after the | sign, or before the hyphen. A great way to test out your regular expression is to use [regex101](https://regex101.com/). - Text string before the parentheses:We use [lookahead](https://www.regular-expressions.info/lookaround.html) assertion **(?=\\()** to matches a text string that is followed by a "(", without making the "(" part of the match. Note that "\\" is the escape character.- Text string before a hyphen:Same as above, we use lookahead **(?=\\-)** to match the string before the hyphen.- Text string after "|":We first flip the text string so that we can match the text before the first "|", then we flip the string back.<jupyter_code># Load library
import re
# Create a function to match string

def short(s):
    if "(" in s:
        return re.match(r".*(?=\()", s).group().strip()
    elif "-" in s:
        return re.match(r".*(?=\-)", s).group().strip()
    elif "|" in s:
        return re.match(r"^(.*?)(?=\|)", s[::-1]).group().strip()[::-1]
    else: 
        return s
# Original labels
labels[:5]
# Use .apply() to apply the function on the original labels
labels2 = labels.apply(lambda x: short(x))
# Short labels
labels2[:5]
# Repeat the above plotting process
# This time with short labels

# Create just a figure and only one subplot
# Must use plt.subplots instead of subplot
# In Jupyter notebook, include all code in one cell

fig, ax = plt.subplots(figsize=(10,10))

# Find axes boundaries
# This is due to ax.add_patch() not updating boundaries
# Thus avoid drawing circles outside of fig

lim = max([max(abs(circle.x)+circle.r, abs(circle.y)+circle.r) for circle in circles])
plt.xlim(-lim,lim)
plt.ylim(-lim,lim)

# List of labels to add on the circles
labels = fandom.name

# Remove axis
# Add title

plt.axis('off')
plt.title('AO3 Popular Fandoms')

# print circles

for circle, label in zip(circles, labels2):
    x, y, r = circle
    ax.add_patch(plt.Circle((x,y), r, alpha=0.2,linewidth=2))
    plt.annotate(label, (x,y), va='center',ha='center')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11584875301689461, 'fraction_numerical': 0.02268704746580853, 'mean_word_length': 4.77158774373259, 'pattern_counts': {'":': 1, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 6, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '225204', 'license': 'MIT', 'n_tokens_mistral': 2105, 'n_tokens_neox': 1984, 'n_words': 659, 'path': '06-10-trivia-popular-tags-II.ipynb', 'repo_name': 'amecreate/ao3-data-vis', 'size': '6215'}