<commit_before>ideas for v2
============
- move to a more natural format as shown in newformat.yml
- any node's content can be a hash or list
- any node can have additional information as children. this should be globbed and available to the user as "text not understood by sid, but important to user".
  - such text should be displayed in the html output
- the presence of DONE in the text of any component will mark that component as architecturally complete. this doesnt apply for existing tools/products being used. it primarily applies to define tasks and building statements
- if no outstanding TBDs are present, sid should display that the architecture is done.
- expand the syntax to support enhancements to existing systems as well. Something along the lines of:
    "to enhance appx ...rest of spec here" instead of "to build appx" should be sufficient
- add a simpler dsl for the architecture diagram than straight dot syntax. Something along the lines of:
    word1 actionwordorphrase word2 will convert it into the equivalent dot syntax: word1 -> word2 [label=actionwordorphrase]<commit_msg>updated ideas to include refactor tasks<commit_after>ideas for v2
============
- move to a more natural format as shown in newformat.yml
- any node's content can be a hash or list
- any node can have additional information as children. this should be globbed and available to the user as "text not understood by sid, but important to user".
  - such text should be displayed in the html output
- the presence of DONE in the text of any component will mark that component as architecturally complete. this doesnt apply for existing tools/products being used. it primarily applies to define tasks and building statements
- if no outstanding TBDs are present, sid should display that the architecture is done.
- expand the syntax to support enhancements to existing systems as well. Something along the lines of:
    "to enhance appx ...rest of spec here" instead of "to build appx" should be sufficient
- add a simpler dsl for the architecture diagram than straight dot syntax. Something along the lines of:
    word1 actionwordorphrase word2 will convert it into the equivalent dot syntax: word1 -> word2 [label=actionwordorphrase]
    
- add support to represent refactoring. something along the lines of:
"to refactor appx change componentx to <<do/not do y>>" and this would pick up the arch diagram, and show the deltas somehow

{'fraction_non_alphanumeric': 0.04722792607802875, 'fraction_numerical': 0.004106776180698152, 'mean_word_length': 4.25, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7270813', 'n_tokens_mistral': 566, 'n_tokens_neox': 555, 'n_words': 372}