<commit_before># Introduction to the Repository

This document aims to provide a introduction to the repository and it's structure to new contributors and developers.

## Repository structure

- The main codebase for the DSS is in `/pkg` and `/cmds`, the core organization and structure of the DSS is in these directories.
- The `/monitoring` folder contains tools to validate DSS deployments and has different directories detailing the tests.

### Monitoring toolset

The `monitoring` directory contains a set of folders containing different test suites to test different capabilities of the DSS during development and production use. 

### Running tools locally

- When running tools in the monitoring toolset for local debugging outside of Docker, the [monitoring](https://github.com/interuss/dss/tree/master/monitoring) folder must be accessible from one of the entries in your `PYTHONPATH`.  To accomplish this, add the root repo folder to your `PYTHONPATH`.

- To run a DSS instance locally, see [this documentation](build/dev/stand_alone_instance.md).

#### Prober
<commit_msg>Updated Introduction to repository (#504)<commit_after># Introduction to the Repository

This document aims to provide a introduction to the repository and it's structure to new contributors and developers.

## Repository structure

- The main codebase for the DSS is in `/pkg` and `/cmds`, the core organization and structure of the DSS is in these directories.
- The `/monitoring` folder contains tools to validate DSS deployments and has different directories detailing the tests.

### Introduction to the Monitoring toolset

The `monitoring` directory contains a set of folders containing different test suites to test different capabilities of the DSS during development and production use.

### Running tools locally

- When running tools in the monitoring toolset for local debugging outside of Docker, the [monitoring](https://github.com/interuss/dss/tree/master/monitoring) folder must be accessible from one of the entries in your `PYTHONPATH`.  To accomplish this, add the root repo folder to your `PYTHONPATH`.

- To run a DSS instance locally, see [this documentation](build/dev/stand_alone_instance.md).

### Building new monitoring tools

When building new monitoring tools, we recommend using Docker containers as a way to package and deploy / test them. Using Docker containers ensures that consistent runtime environments are created regardless of the OS / Architecture.

- When referring to services hosted on the host machine (whether in another Docker container or not) from a service in a Docker container, refer to the host machine from the service in a Docker container using `http://host.docker.internal` ([Windows documentation](https://docs.docker.com/docker-for-windows/networking/#use-cases-and-workarounds), [Mac documentation](https://docs.docker.com/docker-for-mac/networking/#use-cases-and-workarounds)). This often includes demonstration scripts like `run_locally.sh`

#### Prober

{'fraction_non_alphanumeric': 0.06635545181727243, 'fraction_numerical': 0.0010003334444814939, 'mean_word_length': 4.9405940594059405, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5911520', 'n_tokens_mistral': 737, 'n_tokens_neox': 689, 'n_words': 375}