<commit_before># NovaParse
An EV Nova data file and plug-in parser built for NovaJS, this project aims to parse entire nova data / plug-in files and does little post-processing of the retrieved data. It also tries to resolve the notorious id conflict issues Nova had by putting each plug-in in its own id space, letting it change Nova files but not other plug-ins. 

## Prerequisites
git, npm

## Installing
For deployment, [get it from npm](https://www.npmjs.com/package/novaparse) with
```
npm install novaparse
```

For development, download the repository and run 
```
npm install
```

## Usage

#### Reading Nova data and Nova plug-ins

```
var novaParse = require("novaParse");

// should contain "Plug-ins" and "Nova Files" subdirectories
var nova = new novaParse("/nova/source/folder");

nova.read();
```
This loads all nova files contained in ```/nova/source/folder/Nova\ Files/``` and then loads all nova plugins in
```/nova/source/folder/Plug-ins/```. 

NovaParse keeps plugins separated so that they can't overwrite each others' ids. If you have a plugin pack or want certain plugins to share an id space, put them in a subdirectory within plug-ins. For example,
if you want plugins from ARPIA to share the same id space ```ARPIA```, put them all in ```/nova/source/folder/Plug-ins/ARPIA/```.

#### Getting stuff out
Continued from above, ```nova.ids.resources``` contains all parsed nova resources organized by type (like oütf, spïn, etc). These types have ids for each resource paresd of the type. For example, Nova's light blaster weapon can be seen at ```nova.ids.resources.wëap['nova:128']```. Note that it is located at ```'nova:128'``` rather than just ```128```. This is due to the sectioning off of ids mentioned before.

Suppose a plug-in called ```foo``` added a weapon at id 300 and modified the weapon at id 129. Since nova data contains an entry for 129, this plug-in would overwrite ```nova.ids.resources.wëap['nova:129']``` and the id would remain ```nova:129``` in ```nova.ids.resources.wëap```. Since nova data does not have an entry for 300, it would add a new weapon at ```nova.ids.resources.wëap['foo:300']```.

#### ID Spaces
ID spaces provide an easy way for plug-ins to reference their ids. ID spaces are found at 
```
nova.ids.spaces
```

In the above example with the plug-in named ```foo```, the ID space ```fooSpace = nova.ids.spaces['foo']``` can be used to access the resources in ```foo``` as if ```foo``` were the only plug-in installed. 
* ```fooSpace.wëap[128]``` would be equivalant to ```nova.ids.resources.wëap['nova:128']```
* ```fooSpace.wëap[300]``` would be equivalant to ```nova.ids.resources.wëap['foo:300']```

This means ```foo``` always gets its own ids even if there is another plug-in that adds a weapon of id 300.

Plug-ins contained in the same subdirectory stored in ```Plug-ins``` share the same ID space, so they can reference each others' ids.


## Running Tests
In the root directory of the repository, run ```mocha```. There should be plenty of failed tests as plenty of the parsing is not yet written.

## Contributing
Feel free to look at the parsers in ```parsers/``` and write your own or add to the existing ones. If you run out of tests, you can write your own, but don't be suprised if I change how data is stored after accepting your pull request.<commit_msg>Rewrite README to reflect typescript version<commit_after># NovaParse
An EV Nova data file and plug-in parser built for NovaJS, this project parses a directory containing `Nova Data` and `Plug-ins` subdirectories according to the [NovaDataInterface](https://github.com/mattsoulanille/NovaDataInterface). 

In contrast to how EV Nova parsed files, NovaParse separates Plug-ins into their own namespaces to prevent ID conflicts. Each plug-in placed directly in the `Plug-ins` directory has access to its own IDs and to the global `Nova Files` IDs, but does not have access to IDs defined in other plug-ins. Creating a subdirectory inside the `Plug-ins` directory creates a shared namespace, and any plug-ins placed into that directory will have access to each other's IDs (in addition to the `Nova Files` ids), allowing plug-in packs like Extra Outfits and ARPIA2 to work correctly.


## Prerequisites
git, npm

## Installing
For deployment, [get it from npm](https://www.npmjs.com/package/novaparse) with
```
npm install --save novaparse
```

For development, download the repository and run 
```
npm install
```
to install dependencies,
```
npm run test
```
to run the tests, and
```
npm run build
```
to compile typescript into javascript for deployment (`npm run build` must be run before publishing to npm).


## Usage

#### Reading Nova data and Nova plug-ins

```
import { NovaParse } from "novaparse";

// Determines whether NovaParse should throw errors when a parse fails
// or should provide a default value for each failed resource parse.
var strict = false;

// should contain "Plug-ins" and "Nova Files" subdirectories
var parser = new NovaParse("/nova/source/folder", strict);

```
This loads all Nova Files contained in ```/nova/source/folder/Nova\ Files/``` and then loads all Nova Plug-ins in
```/nova/source/folder/Plug-ins/```, possibly overwriting IDs in `Nova Files`. Plug-ins are loaded in reverse alphabetical order, so a plug-in of earlier alphabetical order may overwrite a resource in `Nova Files` that was previously overwritten by a plug-in of later alphabetical order.

#### Getting data out of the parser
NovaParse implements `GameDataInterface` of [NovaDataInterface](https://github.com/mattsoulanille/NovaDataInterface), which defines how data can be accessed from it. In short, 
* `(await NovaDataInterface.ids)["ResourceType"]` gets the list of available IDs for resources of "ResourceType"
* `await NovaDataInterface.data["ResourceType"].get("GlobalID")` gets the `"ResourceType"` resource of id `"GlobalID"`.

The global IDs of `Nova Files` all start with `nova:` and end with their corresponding ID. Global IDs of things in the `Plug-ins` directory start with the name of the Plug-in or directory and end with a colon and their ID.


## Running Tests
Tests can be run with
```npm run test```

To run an individual test, run ```npm run test-only ./test/path/to/test.ts```

## Contributing
Pull requests are welcome if you find bugs, but more type definitions need to be written before new data types can be implemented. The next large project is parsing the `snd ` resource, which contains all the Nova sounds.
{'fraction_non_alphanumeric': 0.10138888888888889, 'fraction_numerical': 0.0070987654320987656, 'mean_word_length': 4.5251491901108265, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3762292', 'n_tokens_mistral': 1919, 'n_tokens_neox': 1788, 'n_words': 902}