<jupyter_start><jupyter_text>HALHyperspace Analogue to Language (HAL) is an early vector space model (VSM) of semantics. VSMs did exist prior to HAL, but I gather that HAL was the first to construct the space automatically. Since Osgood et al. (1957), VSMs were built from human judgements on a predefined set of axes. Practically, this is a time-consuming process. It's also not a good model of the acquisition of semantics (although I doubt it's meant to be that).This notebook implements HAL based on Lund and Burgess (1996). The original paper was hard to follow. The comments in [this implementation](https://github.com/fozziethebeat/S-Space/blob/master/src/main/java/edu/ucla/sspace/hal/HyperspaceAnalogueToLanguage.java) provided the help I needed.The model is as follows. A word-word co-occurrence matrix $X$ is built. For $N$ words in the vocabulary, it will be an $N \times N$ matrix. For every word $w$ in the corpus, we look $m$ words to the left and $m$ to the right. Words appearing within this window (in either direction) are recorded with strength inversely proportional to the linear distance between the words. If word $j$ appears within $m$ words before $i$, $X_{ij}$ is increased. You can reduce the dimensionality of $X$ after training.<jupyter_code>from itertools import chain, combinations
import pandas as pd
import nltk
from nltk.util import ngrams
mycorpus = 'The horse raced past the barn fell .'.split()
mycorpus = [word.lower() for word in mycorpus]
def skipgrams(sequence, n, k):
    """Modified from NLTK to give distance."""
    
    SENTINEL = object()
    for ngram in ngrams(sequence, n + k, pad_right=True, right_pad_symbol=SENTINEL):
        head = ngram[:1]
        tail = ngram[1:]
        for i, skip_tail in enumerate(combinations(tail, n - 1)):
            if skip_tail[-1] is SENTINEL:
                continue
            yield (head + skip_tail, k-i)
def hal(corpus, window):
    skips = skipgrams(corpus, 2, window)
    X = pd.DataFrame(skips, columns=['skipgram', 'weight'])
    X[['word1', 'word2']] = X['skipgram'].apply(pd.Series)
    X.drop('skipgram', axis=1, inplace=True)
    X = X.groupby(['word1', 'word2']).sum().unstack().fillna(0).astype(int)
    X.columns = X.columns.levels[1].values
    X.index = X.index.values
    return X.T
hal(mycorpus, 5)<jupyter_output><empty_output><jupyter_text>Data<jupyter_code>brown = nltk.corpus.brown.words()
brown = list(brown)[:1000]
X = hal(brown, 5)
X<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10936238902340598, 'fraction_numerical': 0.010492332526230832, 'mean_word_length': 4.100823045267489, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '629014', 'license': 'MIT', 'n_tokens_mistral': 820, 'n_tokens_neox': 770, 'n_words': 291, 'path': 'semrep/models/hal/HAL.ipynb', 'repo_name': 'geoffbacon/semrep', 'size': '2478'}