<commit_before>* ECS Debt
** Add support for a step to process the csv files
** Also list the defined classes that are not referenced by other pieces
** Also uniqueness of defined functions would be nice.
** Support adding arguments to the components in the ecs files
   - ie; healthComponent(health=1)
   - refactor the goal entity through that.
   - This way we can have multiple of a component in an entity
** Check to ensure that a value belongs to a certain class
   - eg; check to make sure that sourceKey is actually a source
** Similarly to extending lists, support decorating the function of the parent class.
** Auto-generate csv columns from an ecs file
   - support a dynamic keyword so that you can mark fields in the ecs file that should not show up in the csv
   - Also try the other way around - generate the ecs file from the csv by figuring out columns and put all new columns into a new component
   - Both of these are meant to be used once to create the file. The file should be hand-edited from that point on.
** Define function signatures in base classes and ensure that derived classes use the same signature
** Inline the args so that you don't have to deep clone them with json.
   - On the other hand, this does let me change run time behaviour easily by updating the source args
   - I really don't like that though
** Support a shared reference so that all towers of a certain key can point to a single component
** Look into making conditions more consistent.
** Define necessary pieces for a component and return undefined if they are not present
** Generate statistics
   - The number of individual pieces
   - Number of complex pieces
** Show abstract base classes with only one child
** Shortcut for text substitution as naming conventions are typically strict
   - Alternately support for static checking naming conventions
** Shortcut for referencing allArgs
** Check to see if some definitions are very similar to each other
** Refactor out the line check
** Refactor the commands
** Support being able to define the functions directly.
** Make a tool that shows you the arguments that an ecs object can take.
** DONE Support components that only are made if a certain param exists.
** DONE Global parameter files. 1/1
** DONE Support extending the arrays of parent classes in an ecs file
** DONE Refactor the parseToken
** DONE Write a check to ensure uniqueness of keys
** DONE Check to make sure that all of the identifiers are used
* Rejected
** Make the definitions in a file that are not the first be inaccessible outside that file
** Make namespaces for the functions to reduce chance of overlap<commit_msg>Updated project file.<commit_after>* ECS Debt
** Add support for a step to process the csv files
** Also list the defined classes that are not referenced by other pieces
** Also uniqueness of defined functions would be nice.
** Support adding arguments to the components in the ecs files
   - ie; healthComponent(health=1)
   - refactor the goal entity through that.
   - This way we can have multiple of a component in an entity
** Check to ensure that a value belongs to a certain class
   - eg; check to make sure that sourceKey is actually a source
** Similarly to extending lists, support decorating the function of the parent class.
** Auto-generate csv columns from an ecs file
   - support a dynamic keyword so that you can mark fields in the ecs file that should not show up in the csv
   - Also try the other way around - generate the ecs file from the csv by figuring out columns and put all new columns into a new component
   - Both of these are meant to be used once to create the file. The file should be hand-edited from that point on.
** Define function signatures in base classes and ensure that derived classes use the same signature
** Inline the args so that you don't have to deep clone them with json.
   - On the other hand, this does let me change run time behaviour easily by updating the source args
   - I really don't like that though
** Support a shared reference so that all towers of a certain key can point to a single component
** Look into making conditions more consistent.
** Define necessary pieces for a component and return undefined if they are not present
** Generate statistics
   - The number of individual pieces
   - Number of complex pieces
** Show abstract base classes with only one child
** Shortcut for text substitution as naming conventions are typically strict
   - Alternately support for static checking naming conventions
** Shortcut for referencing allArgs
** Check to see if some definitions are very similar to each other
** Refactor out the line check
** Refactor the commands
** Support being able to define the functions directly.
** Make a tool that shows you the arguments that an ecs object can take.
** Be able to auto-generate gen_csv rows by having something of a particular key format be referred to by something else.
** DONE Support components that only are made if a certain param exists.
** DONE Global parameter files. 1/1
** DONE Support extending the arrays of parent classes in an ecs file
** DONE Refactor the parseToken
** DONE Write a check to ensure uniqueness of keys
** DONE Check to make sure that all of the identifiers are used
* Rejected
** Make the definitions in a file that are not the first be inaccessible outside that file
** Make namespaces for the functions to reduce chance of overlap
{'fraction_non_alphanumeric': 0.037371134020618556, 'fraction_numerical': 0.0011045655375552283, 'mean_word_length': 3.9707227813357733, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1911996', 'n_tokens_mistral': 1258, 'n_tokens_neox': 1207, 'n_words': 877}