<commit_before>{-# LANGUAGE CPP, DataKinds, DeriveGeneric, FlexibleContexts,
             FlexibleInstances, GADTs, OverloadedStrings, PolyKinds,
             ScopedTypeVariables, TypeApplications, TypeOperators,
             ViewPatterns #-}
{-# OPTIONS_GHC -fno-warn-orphans #-}
-- | Demonstrate encoding a 'Rec' to JSON. Three approaches are shown:
-- the first utilizes 'ToJSON' instances for the record's
-- interpretation type constructor applied to each of its fields. This
-- has the advantage of being concise by virtue of re-using a lot of
-- existing pieces. The downside to relying on existing 'ToJSON'
-- instances is that they encode self-contained JSON values, when what
-- we want to do is construct a single JSON object encompassing each
-- record field as a named field of that JSON object. We can do this
-- by inspecting the JSON serialization of each field, and extracting
-- it as a key-value pair if it was serialized as a JSON object with a
-- single named field. This works, but means that the types do not
-- guarantee correctness (i.e. if a record field is serialized as a
-- 'Number', we won't be able to include it in the serialization of
-- the 'Rec').
--
-- The second approach uses a bit of @aeson@ internals to instead
-- serialize each 'Rec' field as a key-value pair with no additional
-- decoration. This should be faster as well as more tightly typed
-- since we do not need to undo any 'Value' wrapping of the individual
-- record fields.
--
-- The third approach uses aeson's built-in functions for working with
-- 'Generic'. This requires some post-processing to address precisely
-- the above problem: the function based on 'Generic' ends up
-- producing a self-contained 'Value' for each field of the
-- record. Specifically, each field becomes an 'Array' that is either
-- empty or contains an 'Object' with a single field as well as
-- another, nested, 'Array' for the rest of the record. We include
-- here a function to flatten that recursive structure into the
-- 'Object' shape we want.
import Control.Lens (view, deep)
import Control.Monad.State.Strict
import qualified Data.HashMap.Strict as H
#if __GLASGOW_HASKELL__ < 804
import Data.Semigroup ((<>))<commit_msg>Add a `module` declaration to Aeson test<commit_after>{-# LANGUAGE CPP, DataKinds, DeriveGeneric, FlexibleContexts,
             FlexibleInstances, GADTs, OverloadedStrings, PolyKinds,
             ScopedTypeVariables, TypeApplications, TypeOperators,
             ViewPatterns #-}
{-# OPTIONS_GHC -fno-warn-orphans #-}
-- | Demonstrate encoding a 'Rec' to JSON. Three approaches are shown:
-- the first utilizes 'ToJSON' instances for the record's
-- interpretation type constructor applied to each of its fields. This
-- has the advantage of being concise by virtue of re-using a lot of
-- existing pieces. The downside to relying on existing 'ToJSON'
-- instances is that they encode self-contained JSON values, when what
-- we want to do is construct a single JSON object encompassing each
-- record field as a named field of that JSON object. We can do this
-- by inspecting the JSON serialization of each field, and extracting
-- it as a key-value pair if it was serialized as a JSON object with a
-- single named field. This works, but means that the types do not
-- guarantee correctness (i.e. if a record field is serialized as a
-- 'Number', we won't be able to include it in the serialization of
-- the 'Rec').
--
-- The second approach uses a bit of @aeson@ internals to instead
-- serialize each 'Rec' field as a key-value pair with no additional
-- decoration. This should be faster as well as more tightly typed
-- since we do not need to undo any 'Value' wrapping of the individual
-- record fields.
--
-- The third approach uses aeson's built-in functions for working with
-- 'Generic'. This requires some post-processing to address precisely
-- the above problem: the function based on 'Generic' ends up
-- producing a self-contained 'Value' for each field of the
-- record. Specifically, each field becomes an 'Array' that is either
-- empty or contains an 'Object' with a single field as well as
-- another, nested, 'Array' for the rest of the record. We include
-- here a function to flatten that recursive structure into the
-- 'Object' shape we want.
module Main where
import Control.Lens (view, deep)
import Control.Monad.State.Strict
import qualified Data.HashMap.Strict as H
#if __GLASGOW_HASKELL__ < 804
import Data.Semigroup ((<>))
{'fraction_non_alphanumeric': 0.07780013413816231, 'fraction_numerical': 0.0013413816230717639, 'mean_word_length': 3.602880658436214, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6150154', 'n_tokens_mistral': 1213, 'n_tokens_neox': 1155, 'n_words': 633}