<jupyter_start><jupyter_text>Notes from testing:- `hidden_layer_dims = [300,100]` works well- learned `alpha` for perplexity of 100 is about 1.8- learned `alpha` for perplexity of 50 is > 2<jupyter_code>foo = ParametricTSNE(28*28, 2, 300, use_cuda=True, hidden_layer_dims=[300,100], alpha=1)
foo.fit(testdata[:20000], p_ij=p_precalc, batch_size=100, epochs=30, learning_rate=0.01, pretrain=True, verbose=True, loss_func='kl')
p_precalc = foo.p_ij
from matplotlib import pyplot as plt
import matplotlib.patches as mpatches
bar = foo(testdata[:20000].cuda()).cpu().detach().numpy()
fig = plt.figure(figsize=(20, 10))
ax = fig.add_subplot(111)
colors = [plt.cm.tab10.colors[i] for i in mnist.targets[:20000]]
ax.scatter(bar[:,0],bar[:,1],c=colors, s=2)
ax.set_aspect(1)
recs = []
for i in range(0,10):
    recs.append(mpatches.Rectangle((0,0),1,1,fc=plt.cm.tab10.colors[i]))
ax.legend(recs,list(range(10)),loc=2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1719409282700422, 'fraction_numerical': 0.08122362869198312, 'mean_word_length': 6.778688524590164, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '575465', 'license': 'MIT', 'n_tokens_mistral': 436, 'n_tokens_neox': 392, 'n_words': 73, 'path': 'ptsne-test.ipynb', 'repo_name': 'einbandi/ptsne-pytorch', 'size': '948'}