<commit_before>Miscue-js
=================

An open source JSON validator, committed to providing the most complete
and accurate reporting of errors and warnings.

## How do I get started? ##

This is a JavaScript application that runs in your 
web browser -- try it out [here](http://mattfenwick.github.io/Miscue-js/)!

## How does it work? ##

There are two main phases:

 1. **parsing** -- recognizes the tree structure
    of the JSON text.  Since every parse error is fatal,
    parsing stops at the first error and gives an informative
    error report.

 2. **semantic checks** -- checks for duplicate keys,
    number overflow, and number underflow.  Reports every problem
    found in the JSON input.

## Why should I use this? ##

I designed and implemented this JSON validator because of two major
inadequacies in every other JSON validator I've used:  

 1. the error reports don't mention the actual problem
 2. not all problems are reported

These problems are addressed by:

 - a novel error reporting strategy that includes a trace of the 
   parsing progress which is dumped immediately upon detection of
   a syntax error
 - a second (semantic) pass, if parsing completes successfully, that
   checks for problems which are outside the scope of the JSON spec

## A short list of common JSON errors ##

Parse errors:

 - unclosed strings, arrays, and objects
 - leading `0`s in numbers
 - invalid character escapes such as `\q` in strings
 - control characters (0 - 31) in strings
 - missing/extra separators (`:` and `,`)

Semantic errors:

 - number overflow
 - number underflow 
 - duplicate keys in objects<commit_msg>clarify goal<commit_after>Miscue-js
=================

Miscue-js provides a free, easy-to-use JSON validation service.

It is committed to providing complete
and accurate reporting of errors and warnings.

This is an open source project under the MIT license.


## Getting started ##

Try it out [here](http://mattfenwick.github.io/Miscue-js/)! 
(requires JavaScript)


## How does it work? ##

There are two main phases:

 1. **parsing** -- recognizes the tree structure
    of the JSON text.  Since every parse error is fatal,
    parsing stops at the first error and gives an informative
    error report.

 2. **semantic checks** -- checks for duplicate keys,
    number overflow, and number underflow.  Reports every problem
    found in the JSON input.


## What's the point? ##

I designed and implemented this JSON validator because of two major
inadequacies in every other JSON validator I've used:  

 1. the error reports don't mention the actual problem
 2. not all problems are reported

These problems are addressed by:

 - a novel error reporting strategy that includes a trace of the 
   parsing progress which is dumped immediately upon detection of
   a syntax error
 - a second (semantic) pass, if parsing completes successfully, that
   checks for problems which are outside the scope of the JSON spec


## A short list of common JSON errors ##

Parse errors:

 - unclosed strings, arrays, and objects
 - leading `0`s in numbers
 - invalid character escapes such as `\q` in strings
 - control characters (0 - 31) in strings
 - missing/extra separators (`:` and `,`)

Semantic errors:

 - number overflow
 - number underflow 
 - duplicate keys in objects
{'fraction_non_alphanumeric': 0.07397342995169082, 'fraction_numerical': 0.004830917874396135, 'mean_word_length': 3.5950069348127602, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6214060', 'n_tokens_mistral': 889, 'n_tokens_neox': 832, 'n_words': 457}