<commit_before>---
layout:		readme
title:		"Properties README"
---

# General Information
This is an introduction for people new to JSON, if you are already familiar with JSON schemas already, move along now, nothing to see here.

### Headers
Please make sure all property markdown files start with the following header:

```markdown
--- 
layout:			property
title:			"name of the data item as it appears in the schemas (ie. lowercase)"
schemas:		[a list of schemas where this data_item is used]
---
```

# JSON
JSON is short for "Javascript Object Notation" and defines the format for an "object." An object in programming terms can be a variable, data structure or a function (basically anything). More commonly, in Object-Oriented programming an object typically refers to an Instance of a Class. That is to say, a specific description of something more general. Like a Goat being an instance of the Class "Animal." 

For our purposes, we can just consider JSON to be the data and a JSON-schema to just define the format for that data, ie. what it needs to *look like* so the computer can understand where things are when it receives it.

### Properties, names and values
A property in JSON notation is a name-value pair and list of properties is what makes up the data in our schemas. What you would normally call a "field" (like interest_rate) is the *name of the property* and what we might call an attribute is the *value of the property*.

**One property**
```javascript
{"name": value}
```

**List of properties**
```javascript
{"name1": "value1", "name3": "value3", "name2": "value2"}
```

JSON is just text, so the formatting is purely visual, even the order (see above) does not matter. As things get more complicated however, it is considered best practice to format your JSON to look more human-friendly:

```javascript
{
  "name1": "value1",
  "name3": "value3",
  "name2": "value2"
}
```

You can visit: [http://jsonprettyprint.com/](http://jsonprettyprint.com/) to help with this formatting.


# JSON-Schema
While any data sent/received in JSON format will always look like the above, we need a way to make sure it is what we are expecting. For this we define a JSON-Schema which basically describes what the JSON formatted data should look like.

### Description
What if we want to add a little more information to describe our property? This is where the "description" parameter comes in. We add some curly brackets where the "value" is supposed to be and add a description as so: 

```javascript
{"name": {"description": "A little blurb about this property"} }
```

*or*

```javascript
{
  "name": {
   "description": "A little blurb about this property"
  }
}
```

### Types
Now that we have described our property value, let's go a step further and narrow it down to a specific type. Is it a number, a word, a list? The "type" parameter allows us to specify exactly this. So now our schema would look something like this:

```javascript
{
  "name": {
   "description": "A little blurb about this property",
   "type": "number"
  }
}
```

JSON has 7 standard types that we can use:

A **null** value means the value is unknown. Note that this is different from an empty or zero field.

An **integer** is a number without a fraction or exponent part.

> ex 1. 4  
> ex 2. 26908289076124561671021

An **number** is a number with or without a fraction or exponent part.

> ex 1. 26908289076124561671021  
> ex 2. 269082.89076124561671021

An **string** is a list of characters (except " or \ ) inside "quotes". You can think of a string as a word. Note that the "word" can also contain numbers (like your national insurance number). But also note that numbers represented as strings need to be converted back to numbers if you want to add or multiply them.

> ex 1. "sheep"  
> ex 2. "AS546NB8"

A **boolean** is simple *true* or *false* flag. Note that the true/false flag is lowercase and not inside "quotes."<commit_msg>Included difference between null and undefined<commit_after>---
layout:		readme
title:		"README"
---

# General Information
This is an introduction for people new to JSON, if you are already familiar with JSON schemas already, move along now, nothing to see here.

### Headers
Please make sure all property markdown files start with the following header:

```markdown
--- 
layout:			property
title:			"name of the data item as it appears in the schemas (ie. lowercase)"
schemas:		[a list of schemas where this data_item is used]
---
```

# JSON
JSON is short for "Javascript Object Notation" and defines the format for an "object." An object in programming terms can be a variable, data structure or a function (basically anything). More commonly, in Object-Oriented programming an object typically refers to an Instance of a Class. That is to say, a specific description of something more general. Like a Goat being an instance of the Class "Animal." 

For our purposes, we can just consider JSON to be the data and a JSON-schema to just define the format for that data, ie. what it needs to *look like* so the computer can understand where things are when it receives it.

### Properties, names and values
A property in JSON notation is a name-value pair and list of properties is what makes up the data in our schemas. What you would normally call a "field" (like interest_rate) is the *name of the property* and what we might call an attribute is the *value of the property*.

**One property**
```javascript
{"name": value}
```

**List of properties**
```javascript
{"name1": "value1", "name3": "value3", "name2": "value2"}
```

JSON is just text, so the formatting is purely visual, even the order (see above) does not matter. As things get more complicated however, it is considered best practice to format your JSON to look more human-friendly:

```javascript
{
  "name1": "value1",
  "name3": "value3",
  "name2": "value2"
}
```

You can visit: [http://jsonprettyprint.com/](http://jsonprettyprint.com/) to help with this formatting.


# JSON-Schema
While any data sent/received in JSON format will always look like the above, we need a way to make sure it is what we are expecting. For this we define a JSON-Schema which basically describes what the JSON formatted data should look like.

### Description
What if we want to add a little more information to describe our property? This is where the "description" parameter comes in. We add some curly brackets where the "value" is supposed to be and add a description as so: 

```javascript
{"name": {"description": "A little blurb about this property"} }
```

*or*

```javascript
{
  "name": {
   "description": "A little blurb about this property"
  }
}
```

### Types
Now that we have described our property value, let's go a step further and narrow it down to a specific type. Is it a number, a word, a list? The "type" parameter allows us to specify exactly this. So now our schema would look something like this:

```javascript
{
  "name": {
   "description": "A little blurb about this property",
   "type": "number"
  }
}
```

JSON has 7 standard types that we can use:

A **null** value means the value is unknown. Note that this is different from an empty or zero field. If the field is empty, undefined or does not exist, then it can simply be omitted (unless it is indicated as being "required" in which case you should provide a suitable default value).

An **integer** is a number without a fraction or exponent part.

> ex 1. 4  
> ex 2. 26908289076124561671021

An **number** is a number with or without a fraction or exponent part.

> ex 1. 26908289076124561671021  
> ex 2. 269082.89076124561671021

An **string** is a list of characters (except " or \ ) inside "quotes". You can think of a string as a word. Note that the "word" can also contain numbers (like your national insurance number). But also note that numbers represented as strings need to be converted back to numbers if you want to add or multiply them.

> ex 1. "sheep"  
> ex 2. "AS546NB8"

A **boolean** is simple *true* or *false* flag. Note that the true/false flag is lowercase and not inside "quotes."
{'fraction_non_alphanumeric': 0.08345697329376854, 'fraction_numerical': 0.022997032640949554, 'mean_word_length': 4.065122103944897, 'pattern_counts': {'":': 28, '<': 3, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4297191', 'n_tokens_mistral': 2345, 'n_tokens_neox': 2162, 'n_words': 1260}