<commit_before># JSON serialization in Dart with Dogma 

Serialization and deserialization of objects to/from JSON Dart is a hot topic, since there's no native way of doing that easily.

When I first learned that it was hard to deal with JSON conversion, I kept thinking about this quote from the Dart landing page:

> [Dart is made] for building web, server, and mobile apps.

Why would it be hard to do such an easy job for a language supposed to be "web oriented"? The answer lies in `dart2js`, the Dart to Javascript compiler.

The way one can convert an object in JSON in Dart is with a `Map` that gets translated in JSON.

Let's see the idiomatic way of writing a class that'll be translated from/to JSON.

```dart
import 'dart:convert';

class Person {
  String name;
  num age;

  Person();

  Person.fromJson(Map json) {
    name = json['name'];
    age = json['age'];
  }

  Map toJson() => {
    'name': name,
    'age': age
  };
}

main() {
  var p = new Person()
    ..name = 'john'
    ..age = 20;

  String jsonEncoded = JSON.encode(p);
  print(jsonEncoded);

  var decoded = new Person.fromJson(JSON.decode(jsonEncoded));
}
```

The above code shows how Dart manages to serialize/deserialize an object into JSON. The 'JSON.encode' function calls the `toJson` method on an object by default.

Imagine if you had nested objects, you would have to manage the hierarchy of `Map`s on your own. And your certainly don't want to do this job yourself for big projects.

Writing a `toJson` method and a `fromJson` constructor for tens and hundreds of model classes is not what I call something productive.

The community has come with different solutions to get over this problem.

## Reflection
The first one would be to use Reflection. I'm not going to talk a lot about it since it's not really good. The concept is to use introspection of code at runtime to scan the code base (the fields and their types) to write these conversion methods at runtime.

The problem with this is that to produce a small and efficient generated code, `dart2js` optimizes, tree-shakes and minifies the code. When using reflection, `dart2js` cannot minify the names of the fields/functions used by reflection, and therefore, when you use reflection, the code size of your web application will most likely explode.

## Transformer
The second one can be good, but it's not perfect. Many libraries out there write code to parse annotations used on classes and fields, so that objects can then be serialized or deserialized. [redstone_mapper](https://pub.dartlang.org/packages/redstone_mapper) does that.

The problem with code generation during the transformers phase, is that the generated code can hardly be debugged. Indeed, the code that's changed or generated by a transformer is usually not available to put breakpoints and inspect.

## Code generation
The last solution is, to my eyes the best. Code generation is pretty much like transformers, except that the code that is generated is available when developing. So you can easily find out bugs in generated code, because you have access to the analyzer, but you can also put breakpoints when debugging at runtime.

This is what [dogma-convert](https://github.com/dogma-dart/dogma-convert) does. 

## Dogma
Dogma is a set of packages made by [Don Olmstead](https://github.com/donny-dont), which contains a few packages that can help you with all the JSON serialization problems. You can generate model classes from different types of schemas, but the part I want to show here will be the generation of encoding and decoding classes for JSON conversion.

First, if you want a class' fields to be mapped 1 to 1, as is, to JSON, you don't need to annotate any field. Otherwise, if you want for example, to use different names for the class in Dart and the object in JSON, you can annotate the field with `@Serialize.field('id')`.

Here's a sample class I have in one of my projects:

```dart
import 'package:dogma_convert/serialize.dart';

class Attachment {
  @Serialize.field('id')
  int id;
  @Serialize.field('from_url')
  String fromUrl;
  @Serialize.field('image_url')
  String imageUrl;
}
``` 

Once you have the following class, you have to export it in a `models.dart` library. `dogma-convert` provides an executable that you can use from a script to generate the converters. See [this](https://github.com/Pacane/SlackHistoryKeeper/blob/master/shared/build.dart) for a sample build script.

The generated encoding/decoding classes look like this:
```dart
library slack_history_keeper_shared.src.convert.attachment_convert;


//---------------------------------------------------------------------
// Standard libraries
//---------------------------------------------------------------------
import 'dart:convert';

//---------------------------------------------------------------------
// Imports
//---------------------------------------------------------------------
import 'package:dogma_convert/convert.dart';
import 'package:slack_history_keeper_shared/models.dart';

//---------------------------------------------------------------------
// Library contents
//---------------------------------------------------------------------

/// A [ModelDecoder] for [Attachment].
class AttachmentDecoder extends Converter<Map, Attachment>
    implements ModelDecoder<Attachment> {
  @override
  Attachment create() => new Attachment();
  @override
  Attachment convert(Map input, [Attachment model]) {
    model ??= create();
<commit_msg>title<commit_after>Serialization and deserialization of objects to/from JSON Dart is a hot topic, since there's no native way of doing that easily.

When I first learned that it was hard to deal with JSON conversion, I kept thinking about this quote from the Dart landing page:

> [Dart is made] for building web, server, and mobile apps.

Why would it be hard to do such an easy job for a language supposed to be "web oriented"? The answer lies in `dart2js`, the Dart to Javascript compiler.

The way one can convert an object in JSON in Dart is with a `Map` that gets translated in JSON.

Let's see the idiomatic way of writing a class that'll be translated from/to JSON.

```dart
import 'dart:convert';

class Person {
  String name;
  num age;

  Person();

  Person.fromJson(Map json) {
    name = json['name'];
    age = json['age'];
  }

  Map toJson() => {
    'name': name,
    'age': age
  };
}

main() {
  var p = new Person()
    ..name = 'john'
    ..age = 20;

  String jsonEncoded = JSON.encode(p);
  print(jsonEncoded);

  var decoded = new Person.fromJson(JSON.decode(jsonEncoded));
}
```

The above code shows how Dart manages to serialize/deserialize an object into JSON. The 'JSON.encode' function calls the `toJson` method on an object by default.

Imagine if you had nested objects, you would have to manage the hierarchy of `Map`s on your own. And your certainly don't want to do this job yourself for big projects.

Writing a `toJson` method and a `fromJson` constructor for tens and hundreds of model classes is not what I call something productive.

The community has come with different solutions to get over this problem.

## Reflection
The first one would be to use Reflection. I'm not going to talk a lot about it since it's not really good. The concept is to use introspection of code at runtime to scan the code base (the fields and their types) to write these conversion methods at runtime.

The problem with this is that to produce a small and efficient generated code, `dart2js` optimizes, tree-shakes and minifies the code. When using reflection, `dart2js` cannot minify the names of the fields/functions used by reflection, and therefore, when you use reflection, the code size of your web application will most likely explode.

## Transformer
The second one can be good, but it's not perfect. Many libraries out there write code to parse annotations used on classes and fields, so that objects can then be serialized or deserialized. [redstone_mapper](https://pub.dartlang.org/packages/redstone_mapper) does that.

The problem with code generation during the transformers phase, is that the generated code can hardly be debugged. Indeed, the code that's changed or generated by a transformer is usually not available to put breakpoints and inspect.

## Code generation
The last solution is, to my eyes the best. Code generation is pretty much like transformers, except that the code that is generated is available when developing. So you can easily find out bugs in generated code, because you have access to the analyzer, but you can also put breakpoints when debugging at runtime.

This is what [dogma-convert](https://github.com/dogma-dart/dogma-convert) does. 

## Dogma
Dogma is a set of packages made by [Don Olmstead](https://github.com/donny-dont), which contains a few packages that can help you with all the JSON serialization problems. You can generate model classes from different types of schemas, but the part I want to show here will be the generation of encoding and decoding classes for JSON conversion.

First, if you want a class' fields to be mapped 1 to 1, as is, to JSON, you don't need to annotate any field. Otherwise, if you want for example, to use different names for the class in Dart and the object in JSON, you can annotate the field with `@Serialize.field('id')`.

Here's a sample class I have in one of my projects:

```dart
import 'package:dogma_convert/serialize.dart';

class Attachment {
  @Serialize.field('id')
  int id;
  @Serialize.field('from_url')
  String fromUrl;
  @Serialize.field('image_url')
  String imageUrl;
}
``` 

Once you have the following class, you have to export it in a `models.dart` library. `dogma-convert` provides an executable that you can use from a script to generate the converters. See [this](https://github.com/Pacane/SlackHistoryKeeper/blob/master/shared/build.dart) for a sample build script.

The generated encoding/decoding classes look like this:
```dart
library slack_history_keeper_shared.src.convert.attachment_convert;

//---------------------------------------------------------------------
// Standard libraries
//---------------------------------------------------------------------
import 'dart:convert';

//---------------------------------------------------------------------
// Imports
//---------------------------------------------------------------------
import 'package:dogma_convert/convert.dart';
import 'package:slack_history_keeper_shared/models.dart';

//---------------------------------------------------------------------
// Library contents
//---------------------------------------------------------------------

/// A [ModelDecoder] for [Attachment].
class AttachmentDecoder extends Converter<Map, Attachment>
    implements ModelDecoder<Attachment> {
  @override
  Attachment create() => new Attachment();
  @override
  Attachment convert(Map input, [Attachment model]) {
    model ??= create();

{'fraction_non_alphanumeric': 0.14440069590696822, 'fraction_numerical': 0.0012819338888380185, 'mean_word_length': 2.9586806814063067, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 13, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2961628', 'n_tokens_mistral': 2793, 'n_tokens_neox': 2703, 'n_words': 1405}