<jupyter_start><jupyter_text>Formal Grammar Example: Barsalou Perceptual Symbol GrammarTo demonstrate the formal grammar and language framework I developed, I created a formal grammar that implements a weird version of Barsalou's Percetpual Symbol Systems. Each sentence is a "simulation", with each simulation having exactly one "simulator", which in turn could have many categories and instances that make it up. Not that that's exactly what he was saying, but it's a fun attempt to formalize his system from his 1999 BBS paper.<jupyter_code>from IPython.display import display, Latex

from grammar import Grammar, ProductionRule

# build a hierarchy of production rules
production_rules = [
    ProductionRule('simulation', ['simulator', 'category', 'instance'], n_outputs='all'),
        ProductionRule('simulator', ['driving', 'flying', 'seeing red', 'seeing blue', 'seeing green'], n_outputs='one'),
        ProductionRule('category', ['car', 'airplane', 'bird', 'grass', 'bed'], n_outputs='many'),
        ProductionRule('instance', ['2004 pontiac grand am', '2016 subaru crosstrack', '2010 chevy silverado', 
                                    'Boeing 747', 'queen bed with yellow sheets', 'cardinal', 'sky',
                                    'cheatgrass', 'kentucky bluegrass'], n_outputs='many')
]

# instantiate the grammar based on the rules
perceptual_symbol_grammar = Grammar(production_rules)
# use the print_latex() method of the Grammar class to print the formal grammar
display(Latex(perceptual_symbol_grammar.print_latex()))
# now let's create a language that will produce sentences based on the grammar
from grammar import Language

l = Language(perceptual_symbol_grammar)

prods = l.productions()
# You can run this as many times as you like and get a new production
for i in range(10):
    p = next(prods)
    print(next(prods) + '\n')<jupyter_output>seeing green bird car airplane grass 2010 chevy silverado cheatgrass sky Boeing 747

flying airplane 2004 pontiac grand am Boeing 747 2010 chevy silverado queen bed with yellow sheets kentucky bluegrass cheatgrass sky 2016 subaru crosstrack

seeing green grass airplane bird car bed sky cardinal 2016 subaru crosstrack kentucky bluegrass cheatgrass 2004 pontiac grand am Boeing 747 2010 chevy silverado queen bed with yellow sheets

driving car bird bed airplane grass 2010 chevy silverado Boeing 747 2016 subaru crosstrack

seeing red bird car kentucky bluegrass 2010 chevy silverado Boeing 747 cardinal cheatgrass sky 2016 subaru crosstrack

seeing blue airplane car bed bird queen bed with yellow sheets 2016 subaru crosstrack Boeing 747

seeing red bird airplane 2010 chevy silverado

flying car 2010 chevy silverado queen bed with yellow sheets kentucky bluegrass cheatgrass cardinal 2004 pontiac grand am sky 2016 subaru crosstrack Boeing 747

driving bird car queen bed with yellow sheets Boei[...]
{'fraction_non_alphanumeric': 0.06317250954529677, 'fraction_numerical': 0.03679278028462339, 'mean_word_length': 4.729622266401591, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '293036', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 896, 'n_tokens_neox': 726, 'n_words': 358, 'path': 'formal-grammar/PerceptualSymbolGrammar.ipynb', 'repo_name': 'mtpain/COGS-202-HW', 'size': '2881'}