<jupyter_start><jupyter_text>Address and PC ExperimentsIn all experiments, I ran the bzip2 program with 100,000,000 instructions<jupyter_code>from collections import Counter

import matplotlib.pyplot as plt
import seaborn as sns
ADDRESS_FILE_1 = "../../spec_stats/address_pc_experiment/address.out.1"
PC_FILE_1 = "../../spec_stats/address_pc_experiment/pc.out.1"
ADDRESS_FILE_2 = "../../spec_stats/address_pc_experiment/address.out.2"
PC_FILE_2 = "../../spec_stats/address_pc_experiment/pc.out.2"
addresses_1 = [int(val) for val in open(ADDRESS_FILE_1).read().splitlines()]
pcs_1 = [int(val) for val in open(PC_FILE_1).read().splitlines()]
addresses_2 = [int(val) for val in open(ADDRESS_FILE_2).read().splitlines()]
pcs_2 = [int(val) for val in open(PC_FILE_2).read().splitlines()]<jupyter_output><empty_output><jupyter_text>Q: How often does a program counter go back to the same address?A: quite frequently<jupyter_code>pc_counts = list(Counter(pcs_1).values())
sns.distplot(pc_counts, kde=False, rug=True)
plt.title("Number of times each PC was accessed")
plt.show()<jupyter_output><empty_output><jupyter_text>Q: Do PCs increase throughout program?A: **no** for the most part the PC is actually decreasing???<jupyter_code>increased = 0
for i in range(1, len(pcs_1)):
  if pcs_1[i] > pcs_1[i - 1]:
    increased += 1
print(increased / len(pcs_1))
total_pcs = len(set(pcs_1))
n_pcs_multiple = sum(1 for x in pc_counts if x > 1)
print("Percent of PCs with multiple calls:", 100 * n_pcs_multiple / total_pcs)<jupyter_output>Percent of PCs with multiple calls: 17.4565560821485<jupyter_text>Q: How long should our training period be? Can we figure out a cutoff where a large ratio of addresses have already been seen?A: Unfortunately with 100,000,000 instructions this looks largely linear.<jupyter_code>total_addresses = len(set(addresses_1))
seen_so_far = set()
seen_at_timestep = []
for address in addresses_1:
  seen_so_far.add(address)
  seen_at_timestep.append(len(seen_so_far))
ax = sns.scatterplot(list(range(len(addresses_1))), seen_at_timestep, linewidth=0)
ax.set(xlabel='Call to calculatePrefetch', ylabel='Number of Addresses')
plt.title("Number of Addresses Seen at Each Timestep")
plt.show()
diffs_so_far = set()
diffs_seen_at_timestep = [0]
for i in range(1, len(addresses_1)):
  diffs_so_far.add(addresses_1[i] - addresses_1[i - 1])
  diffs_seen_at_timestep.append(len(diffs_so_far))
ax = sns.scatterplot(list(range(len(addresses_1))), diffs_seen_at_timestep, linewidth=0)
ax.set(xlabel='Call to calculatePrefetch', ylabel='Number of Addresses')
plt.title("Number of Addresses Diffs Seen at Each Timestep")
plt.show()<jupyter_output><empty_output><jupyter_text>Q: If we run the program twice, is the program counter the same?A: Between two runs, we see > 50% overlap in PCs (overall)<jupyter_code>overlap = set(pcs_1).intersection(set(pcs_2))
total = set(pcs_1).union(set(pcs_2))
print("Amount of overlap in PCs:", len(overlap) / len(total))<jupyter_output>Amount of overlap in PCs: 0.6624671916010498<jupyter_text>Q: If we run the program twice, are the memory addresses the same?A: Between two runs, we see > 90% overlap in memory addresses (overall)\This suggests that there is a strong relationship and I think we can assume that there is overlap generally.<jupyter_code>overlap = set(addresses_1).intersection(set(addresses_2))
total = set(addresses_1).union(set(addresses_2))
print("Amount of overlap in addresses:", len(overlap) / len(total))<jupyter_output>Amount of overlap in addresses: 0.915083519810787
{'fraction_non_alphanumeric': 0.11926345609065156, 'fraction_numerical': 0.03456090651558073, 'mean_word_length': 6.743421052631579, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '79648', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1320, 'n_tokens_neox': 1252, 'n_words': 322, 'path': 'python/notebooks/Address_PC.ipynb', 'repo_name': 'lbiester/gem5', 'size': '3530'}