<jupyter_start><jupyter_text><jupyter_code>#@title
#Imports & settings
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import plotly.express as px
import plotly.graph_objects as go
sns.set_theme(style="ticks", color_codes=True)
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Terra Validator RankingThis notebook explores the questions:  *Are the same validators that were in the top 10 (by rewards received) last month in the top 10 this month? Top 5? Break this down by change over the weeks in the past 2 months as well.* Top Validators for Last MonthIn the graph below examines the ranking of the Terra Validators over last month and this month (to July 11).  The rankings are based on the USD rewards collected via the 'withdraw_delegator_rewards' action from the Flipside Crypto terra.rewards table.  The graph is drawn with a log y-axis - this is to allow easier discrimination between the data groups near the top of the chart.We see in the graph that the top 10 in June are mostly the top 10 in July also, with a few positional changes.  A couple of noteable points are *hashed*, who dropped from 1st place to 27th, and *BTC.Secure* mooned from 100th to 3rd.  Of the top 5 validators in June, only 3 are in the top 5 in July.<jupyter_code>#@title
#Import the Flipside query via the API endpoint
df = pd.read_json('https://api.flipsidecrypto.com/api/v2/queries/15533707-11d8-4b0d-bc1d-d8cdeeebfb88/data/latest')
df.rename(columns = {'VALIDATOR_ADDRESS_LABEL':'Validator', 'MONTH':'Month', 'RANK' : 'Rank', 'USD_TOTAL':'USD Total'}, inplace = True)

#Plot the data using plotly.express
fig = px.line(df, x="Month", y="Rank", color='Validator',
              template="simple_white", width=1200, height=800, log_y = True,
              title= 'Validator Rank - June to July 2021')
fig.update_yaxes(autorange="reversed")
fig.update_xaxes(autorange="reversed")
fig.update_traces(mode="lines+markers")
fig.show()<jupyter_output><empty_output><jupyter_text>Validator Rank Weekly over the Last 2 MonthsHere we plot the same data over the last 2 months, but zooming in to weekly intervals instead.  We have used the same batch of validators as in the previous graph - either top 10 in June or July.These results were not as expected, making this chart very hard to interpret.  Take, for example, HuobiPool. It maintains a very low ranking, except on the 30th of May and the 27th of June when it rockets to the top of the charts.  A good portion of other validators follow a similar pattern.  Something is up with the reward collection timings.  We will drill further into the data to find out what is happening.<jupyter_code>#@title
#Import the Flipside query via the API endpoint
df = pd.read_json('https://api.flipsidecrypto.com/api/v2/queries/ca22046d-4c3d-4095-958d-2deb7b030620/data/latest',  convert_dates=['WEEK'])
df.rename(columns = {'VALIDATOR_ADDRESS_LABEL':'Validator', 'WEEK':'Week', 'RANK' : 'Rank', 'USD_TOTAL':'USD Total'}, inplace = True)
#df.sort('Week', 'Validator')
df.sort_values(by=['Week', 'Validator'], inplace=True)

#Plot the data using plotly.express
fig = px.line(df, x="Week", y="Rank", color='Validator',
              template="simple_white", width=1200, height=800, log_y = True,
              title= 'Validator Rank - Last 2 Months')
fig.update_yaxes(autorange="reversed")
fig.update_traces(mode="lines+markers")
fig.show()<jupyter_output><empty_output><jupyter_text>What's up with the reward collection?The graph below shows the validator reward collection, by day, from all validators.  Note the big spikes - 18th & 31st of May and the 30th of June.  It seems as though some big players are collecting infrequently - perhaps only once per month.<jupyter_code>#@title
#Import the Flipside query via the API endpoint
df = pd.read_json('https://api.flipsidecrypto.com/api/v2/queries/b31399e1-38dd-49e4-8ad2-240f1fc3653c/data/latest',  convert_dates=['DAY'])
df.rename(columns = {'DAY':'Day', 'USD_TOTAL':'USD Total'}, inplace = True)
df.sort_values(by=['Day'], inplace=True)

#Plot the data with plotly.express
fig = px.bar(df, x="Day", y="USD Total",
                  template="simple_white", width=1200, height=600,
                  title= 'Validator Reward Collection - Last 2 Months')
fig.show()<jupyter_output><empty_output><jupyter_text>Case Study - HuobiPoolLet's have a look at the rewards claimed by HuobiPool to test our hypothesis of infrequent collection.The graph below has the smoking gun - only two big collections over the past 2 months!  This changes our assumptions around the data for comparing the Validator Ranking.  Rather than look at rankings by week over the last 2 months, we should look at monthly data over a longer period.  We should also avoid part months in the dataset to allow fair comparisons between those who collection regularly and those who leave it until the end of the month.<jupyter_code>#@title
#Import the Flipside query via the API endpoint
df = pd.read_json('https://api.flipsidecrypto.com/api/v2/queries/d769cd50-d505-45c4-9308-979023bdf816/data/latest',  convert_dates=['DAY'])
df.rename(columns = {'DAY':'Day', 'USD_TOTAL':'USD Total'}, inplace = True)
df.sort_values(by=['Day'], inplace=True)

#Plot the data with plotly.express
fig = px.bar(df, x="Day", y="USD Total",
                  template="simple_white", width=1200, height=600,
                  title= 'HuobiPool Reward Collection - Last 2 Months')
fig.show()<jupyter_output><empty_output><jupyter_text>Validator Rank Monthly, Jan-Jun 2021Keeping the same set of validators identified in the first section (top 10 in either June or July), we will plot their relative rankings over the last 6 months.  This time we will use monthly buckets to avoid any bias from the reward collection behaviour of the validators.  The results are much more in line with expectations now.As seen from the graph below (again, note the log scale for ease of reading), the validator *hashed* has been either the 1st or 2nd highest returning validator for the last 4 months.  A couple of relatively new players, *HuobiPool* and *Orion.Money,* have rocketed to the top of the leaderboard in a short time.  Of the current leading validators, none of them were in the top 4 validators back in January.  It seems that there is plenty of competition in the validator space and we can expect to see more place rotation in future months.<jupyter_code>#@title
#Import the Flipside query via the API endpoint
df = pd.read_json('https://api.flipsidecrypto.com/api/v2/queries/8353870a-334c-433d-a3f4-c1c3321cef4e/data/latest',  convert_dates=['WEEK'])
df.rename(columns = {'VALIDATOR_ADDRESS_LABEL':'Validator', 'MONTH':'Month', 'RANK' : 'Rank', 'USD_TOTAL':'USD Total'}, inplace = True)
#df.sort('Week', 'Validator')
df.sort_values(by=['Month', 'Validator'], inplace=True)

#Plot the data using plotly.express
fig = px.line(df, x="Month", y="Rank", color='Validator',
              template="simple_white", width=1200, height=800, log_y = True,
              title= 'Validator Rank - Last 6 Months')
fig.update_yaxes(autorange="reversed")
fig.update_traces(mode="lines+markers")
fig.show()<jupyter_output><empty_output><jupyter_text>Top Validators For Last Month - RevisitedNow we know that taking a part month of data may lead to misleading rankings, let's revisit the original question looking at May & June 2021 - the last two *full* months.Of the top 10 validators in May, only 5 are still in the top 10 in June.  4 validators were bumped down into the top 20, but one, *Fair Square Lab*, fell all the way to position 69.  The top 5 was also shaken up - 2 new entrants forced their way into this esteemed group in June.  Of note is HuobiPool who went from position 81 to position 2 in just a month.  The validator *hashed* maintained top billing across both months.<jupyter_code>#@title
#Import the Flipside query via the API endpoint
df = pd.read_json('https://api.flipsidecrypto.com/api/v2/queries/0017bd64-66ea-4ddf-bc62-7cd848c36cf6/data/latest')
df.rename(columns = {'VALIDATOR_ADDRESS_LABEL':'Validator', 'MONTH':'Month', 'RANK' : 'Rank', 'USD_TOTAL':'USD Total'}, inplace = True)

#Plot the data using plotly.express
fig = px.line(df, x="Month", y="Rank", color='Validator',
              template="simple_white", width=1200, height=800, log_y = True,
              title= 'Validator Rank - May to June 2021')
fig.update_yaxes(autorange="reversed")
#fig.update_xaxes(autorange="reversed")
fig.update_traces(mode="lines+markers")
fig.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09939052977027661, 'fraction_numerical': 0.029418659165494607, 'mean_word_length': 4.880771881461062, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '465424', 'license': 'Apache-2.0', 'n_tokens_mistral': 2807, 'n_tokens_neox': 2563, 'n_words': 994, 'path': 'Terra_Validator_Ranking.ipynb', 'repo_name': 'aleanos/0x-launch-kit', 'size': '8532'}