<commit_before># Notes on the type system


- `:boolean`
  - `:code`
- `:booleans`
  - `:code`
  - `:collection`
  - `:vector`
- `:char`
  - `:code`
- `:code`
  - `:code`
- `:codeblock`
  - `:code`
  - `:collection`
- `:float`
  - `:code`
  - `:number`
  - `:complex`
  - `:ratio`
- `:floats`
  - `:code`
  - `:collection`
  - `:vector`
- `:generator`
  - `:code`
  - `:collection`
- `:integer`
  - `:code`
  - `:number`
  - `:float`
  - `:complex`
  - `:ratio`
- `:integers`
  - `:code`
  - `:collection`
  - `:vector`
- `:ref`
  - `:code`
- `:set`
  - `:code`
  - `:collection`
- `:string`
  - `:code`
  - `:collection`
  - `:vector`
- `:strings`
  - `:code`
  - `:collection`
  - `:vector`
- `:tagspace`
  - `:code`
  - `:collection`
  - `:structured`
- `:vector`
  - `:code`
  - `:collection`

- `:matrix`
  - `:code`
  - `:collection`
  - `:vector`
  - `:structured`
- `:complex`
  - `:code`
  - `:number`
  - `:collection`
  - `:vector`
  - `:structured`
- `:rational`
  - `:code`
  - `:number`
  - `:collection`
  - `:vector`
  - `:structured`


- `:structured`
  
  a `hashmap` with specified keys; anything complex enough to be represented as a Clojure `record`
- `:collection`
  
  any list, collection, map, tagspace or equivalent <commit_msg>notes on type system renovation<commit_after># Notes on the type system


- `:boolean`
  - `:code`
- `:booleans`
  - `:code`
  - `:collection`
  - `:vector`
- `:char`
  - `:code`
- `:code`
  - `:code`
- `:codeblock`
  - `:code`
  - `:collection`
- `:float`
  - `:code`
  - `:number`
  - `:complex`
  - `:ratio`
- `:floats`
  - `:code`
  - `:collection`
  - `:vector`
- `:generator`
  - `:code`
  - `:collection`
- `:integer`
  - `:code`
  - `:number`
  - `:float`
  - `:complex`
  - `:ratio`
- `:integers`
  - `:code`
  - `:collection`
  - `:vector`
- `:ref`
  - `:code`
- `:set`
  - `:code`
  - `:collection`
- `:string`
  - `:code`
  - `:collection`
  - `:vector`
- `:strings`
  - `:code`
  - `:collection`
  - `:vector`
- `:tagspace`
  - `:code`
  - `:collection`
  - `:structured`
- `:vector`
  - `:code`
  - `:collection`

- `:matrix`
  - `:code`
  - `:collection`
  - `:vector`
  - `:structured`
- `:complex`
  - `:code`
  - `:number`
  - `:collection`
  - `:vector`
  - `:structured`
- `:rational`
  - `:code`
  - `:number`
  - `:collection`
  - `:vector`
  - `:structured`

### Some definitions for abstract types

- `:structured`
  
  a `hashmap` with specified keys; anything complex enough to be represented as a Clojure `record`
- `:collection`
  
  any `list`, `codeblock` `map`, `tagspace` or equivalent 
- `:number`
  
  You can probably figure this out.
{'fraction_non_alphanumeric': 0.24056423942051086, 'fraction_numerical': 0.0, 'mean_word_length': 2.4166666666666665, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1180348', 'n_tokens_mistral': 1181, 'n_tokens_neox': 1160, 'n_words': 220}