<commit_before># son-schema
The schema files for the various descriptors used by SONATA

## Development

To contribute to the development of the SONATA schema files, you may use the very same development workflow as for any other SONATA Github project. That is, you have to fork the repository and create pull requests. Moreover, all discussions regarding the SONATA schemas take place on Github, and NOT on the wiki.

#### Integration with Jenkins

The repository is integrated with Jenkins. Thus, every pull request is checked automatically. The Jenkins job validates all the JSON and YAML files in the repository to be a valid JSON/YAML format. Moreover, Jenkins executes all files named 'test_*.sh' that reside in */test/ directories automatically. Thus, it is quite easy to add new tests. Just write a corresponding bash script. Tests are considered successful when they return with exit code 0, and to be a failure otherwise.<commit_msg>Added some links to useful tools.<commit_after># son-schema
The schema files for the various descriptors used by SONATA

## Development

To contribute to the development of the SONATA schema files, you may use the very same development workflow as for any other SONATA Github project. That is, you have to fork the repository and create pull requests. Moreover, all discussions regarding the SONATA schemas take place on Github, and NOT on the wiki.

#### Integration with Jenkins

The repository is integrated with Jenkins. Thus, every pull request is checked automatically. The Jenkins job validates all the JSON and YAML files in the repository to be a valid JSON/YAML format. Moreover, Jenkins executes all files named 'test_*.sh' that reside in */test/ directories automatically. Thus, it is quite easy to add new tests. Just write a corresponding bash script. Tests are considered successful when they return with exit code 0, and to be a failure otherwise.

#### Useful Tools

There are some useful tools that support working the JSON and YAML files.

- http://jsonviewer.stack.hu/ A nice JSON editor
- http://yamltojson.com/ Convert YAML to JSON
- http://jsontoyaml.com/ Convert JSON to YAML
- http://jsonschemalint.com/draft4/#/ Write JSON-Schema documents and validate JSON
- http://jsonschema.net/#/ Create JSON-Schema from JSON documents

#### Useful Links

- http://json-schema.org/ The general JSON-Schema standard

{'fraction_non_alphanumeric': 0.05480607082630692, 'fraction_numerical': 0.0012647554806070826, 'mean_word_length': 4.368778280542986, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4834173', 'n_tokens_mistral': 591, 'n_tokens_neox': 558, 'n_words': 346}