<commit_before># Circe

Next, we're going to want to parse our JSON files. We'll walk through parsing the first one, then, we'll let you do the second by yourself.

We're going to use [Circe](https://circe.github.io/circe/) to parse the JSON. Let's add a dependency into our dependencies file:

```scala
import sbt._

object Dependencies {
  lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.1" % Test
  lazy val scalaCheck = "org.scalacheck" %% "scalacheck" % "1.13.4" % Test

  val circeVersion = "0.8.0"
  lazy val circeCore = "io.circe" %% "circe-core" % circeVersion
  lazy val circeGeneric = "io.circe" %% "circe-generic" % circeVersion
  lazy val circeParser = "io.circe" %% "circe-parser" % circeVersion
}
```

And our build file:

```scala
libraryDependencies ++= Seq(
  circeCore,
  circeGeneric,
  circeParser,
  scalaTest,
  scalaCheck
)
```

OK, let's look at our cart first. The structure looks like:

```json
[
  {
    "product-type": "hoodie",
    "options": {
      "print-location": "front",
      "colour": "white",
      "size": "small"
    },
    "artist-markup": 20,
    "quantity": 0
  }
]
```

So a cart is composed of a list of items. Each item contains a product type, a map of options (key/value pairs), an artist markup and a quantity.

The first thing that we want to do is to model the structure. Then, we'll parse (decode) the JSON into the classes we've just built. This process of parsing JSON into domain models is called decoding. When we go the other way - from classes to JSON - is called encoding. The decoder/encoder pair is called often called a codec.

In general, we want our classes to model the structure of the JSON we're decoding. However, we may wish to clean up the structure in our decoders to something more sane, if we're dealing with JSON from data sources we have no control over.

So, our classes will look something like this.

```scala
final case class CartItem(
    productType: String, options: Map[String, String], artistMarkup: Int, quantity: Int)

final case class Cart(items: Seq[CartItem])
```

Next, we'll want to write a decoder for this cart JSON. Decoding JSON using Circe is stateful, you essentially have a cursor that points to a position in the JSON, and you advance the cursor down or up the JSON tree.

There are very syntacticly simple ways to parse JSON in Circe, however they aren't as flexible, nor are they as explicit. If you want to use them feel free, but we'll use a more explicit way of parsing.

A basic `Decoder` looks like this:

```scala
val cartDecoder = Decoder.instance[Cart] { c =>
  ...
}
```

We essentially pass a function taking a cursor (the current position of the parser), and then from this cursor we descend down the fields we want to get the values of.

Let's parse the product type field in a cart item:

```scala
val cartItemDecoder = Decoder.instance[CartItem] { c =>
  for {
    productType <- c.downField("product-type").as[String]
  } yield CartItem(productType, Map.empty, 0, 0)
}
```

TODO Explain this


We can test this using something like the following:

```scala
{

}
```

We can plug this into our class using something like the following:

```scala

```

Now that we're done with the cart, go ahead & write a decoder for the base products. Note that the JSON isn't quite clean, feel free to change the data such that the product options are always an array.<commit_msg>Decoder explanation<commit_after># Circe

Next, we're going to want to parse our JSON files. We'll walk through parsing the first one, then, we'll let you do the second by yourself.

We're going to use [Circe](https://circe.github.io/circe/) to parse the JSON. Let's add a dependency into our dependencies file:

```scala
import sbt._

object Dependencies {
  lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.1" % Test
  lazy val scalaCheck = "org.scalacheck" %% "scalacheck" % "1.13.4" % Test

  val circeVersion = "0.8.0"
  lazy val circeCore = "io.circe" %% "circe-core" % circeVersion
  lazy val circeGeneric = "io.circe" %% "circe-generic" % circeVersion
  lazy val circeParser = "io.circe" %% "circe-parser" % circeVersion
}
```

And our build file:

```scala
libraryDependencies ++= Seq(
  circeCore,
  circeGeneric,
  circeParser,
  scalaTest,
  scalaCheck
)
```

OK, let's look at our cart first. The structure looks like:

```json
[
  {
    "product-type": "hoodie",
    "options": {
      "print-location": "front",
      "colour": "white",
      "size": "small"
    },
    "artist-markup": 20,
    "quantity": 0
  }
]
```

So a cart is composed of a list of items. Each item contains a product type, a map of options (key/value pairs), an artist markup and a quantity.

The first thing that we want to do is to model the structure. Then, we'll parse (decode) the JSON into the classes we've just built. This process of parsing JSON into domain models is called decoding. When we go the other way - from classes to JSON - is called encoding. The decoder/encoder pair is called often called a codec.

In general, we want our classes to model the structure of the JSON we're decoding. However, we may wish to clean up the structure in our decoders to something more sane, if we're dealing with JSON from data sources we have no control over.

So, our classes will look something like this.

```scala
final case class CartItem(
    productType: String, options: Map[String, String], artistMarkup: Int, quantity: Int)

final case class Cart(items: Seq[CartItem])
```

Next, we'll want to write a decoder for this cart JSON. Decoding JSON using Circe is stateful, you essentially have a cursor that points to a position in the JSON, and you advance the cursor down or up the JSON tree.

There are very syntacticly simple ways to parse JSON in Circe, however they aren't as flexible, nor are they as explicit. If you want to use them feel free, but we'll use a more explicit way of parsing.

A basic `Decoder` looks like this:

```scala
val cartDecoder = Decoder.instance[Cart] { c =>
  ...
}
```

We essentially pass a function taking a cursor (the current position of the parser), and then from this cursor we descend down the fields we want to get the values of.

Let's parse the product type field in a cart item:

```scala
val cartItemDecoder = Decoder.instance[CartItem] { c =>
  for {
    productType <- c.downField("product-type").as[String]
  } yield CartItem(productType, Map.empty, 0, 0)
}
```

Let's break this down a little:

1. We use the `Decoder.instance` method to create a new `Decoder`. This isn't magic, under the hood it's creating an anonymous `Decoder` and calling the passed function in the `apply` method (again, `apply` is a common pattern in Scala):

    ```
    final def instance[A](f: HCursor => Result[A]): Decoder[A] = new Decoder[A] {
      final def apply(c: HCursor): Result[A] = f(c)
    }
    ```

1. The function that does the actual work starts with a cursor, this points to the current position of the parser. Obviously this means that the decoder is relative, you must invoke it at the right point in the parsing, pointing at the right piece. This can mean that sometimes your decoder does odd things, and it's important to have adequate tests.

1. Given a cursor position, the next thing it does is go down the field called `product-type`, and parse it's value as a `String`. This operation could fail for a couple of reasons. Firstly, the `product-type` field may not exist in the JSON, and secondly it's value may not be a string (or convertible to a string).

1. We wrap all this up `for` comprehension. You will likely build your decoders using `for` comprehensions, as this allows us to handle the potential decoding failures in a syntactically clean way (the failures we mentioned above). Essentially, the for comprehensions hides the error handling from us in a clean way.

So now we've got our decoder, let's build a test for it.

```scala
{

}
```

We can plug this into our class using something like the following:

```scala

```

Now that we're done with the cart, go ahead & write a decoder for the base products. Note that the JSON isn't quite clean, feel free to change the data such that the product options are always an array.


**Further reading:**

You've been introduced to a few new concepts, now's probably a good time to read up on them.

* `for` comprehensions.
{'fraction_non_alphanumeric': 0.08888089240734078, 'fraction_numerical': 0.004078205589540602, 'mean_word_length': 3.833623188405797, 'pattern_counts': {'":': 14, '<': 5, '<?xml version=': 0, '>': 8, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5484919', 'n_tokens_mistral': 2525, 'n_tokens_neox': 2371, 'n_words': 1225}