<commit_before>
                    
   ROADMAP DIFFTEST  
                    



Catchphrase: "Snapshot-driven testing" and/or "Forget test assertions"
  - do not set it until promotion, to avoid idea stealing

Logo:
  - "difftest" and include green + and red - inside letters
  - nice sans-serif non-monospace font

test|snapshot("TITLE", [...ARR], FUNC(t, ...ARG) [, ...OPTS])->PROMISE
  - must be called synchronously during file load
  - named export `test|snapshot` (not default export)

test():
  - assertion testing (no snapshot)

snapshot():
  - snapshot testing
  - should return VAL to snapshot
  - if return PROMISE, await it
     - snapshot (including reporting) should include that it's async
  - exceptions or rejected PROMISEs:
     - snapshot the ERROR
     - snapshot (including reporting) should include that it's been thrown|rejected (errored BOOL)
     - serializer should always snapshot ERRORs with String() (i.e. no stack trace)

"TITLE":
   - not optional
   - must be unique within file

...ARR + FUNC: use test-each

FUNC: should be fired on next tick

t OBJ:
  - index, indexes: like test-each
  - name STR
  - title STR: name + stringified data used for reporting
  - snapshot BOOL: whether snapshot() or test() was called
  - log(VAL,...)

Returned PROMISE:
  - resolved value:
      - OBJ:
         - title STR, args: ARR_ARR, options OPTS
         - tests OBJ_ARR: success BOOL, type 'new|removed|same|different|inconstant', action 'next|update|remove|move|input', value VAL, snapshot STR, args ARR, async BOOL, errored BOOL
         - others???
      - meant for plugins.
  - always resolved, even on test failure

Re-using:
   - TITLE (like describe()): test = (title, ...args) => test(`TITLE > ${title}`, ...args)
   - ...ARR: test = (title, ...args) => test(title, ...ARR, ...args)
   - OPTS: test = (...args) => test(...args, OPTS)
       - which is why OPTS is variadic

Hooks not needed:
   - beforeAll():
      - just execute code in top-level scope
      - for async, top-level code can return PROMISE then each test can use await PROMISE
   - afterAll(): use await Promise.all(...PROMISE) using returned PROMISEs
   - before|afterEach(): just repeat code at beginning|end of FUNC

CLI runner:
  - called "dt" to make it fast to type

Files selection:
  - use CLI argument "GLOB"... (whitelisting)
     - can be "!GLOB" (blacklisting)
     - can target DIRs
  - def: test[s]|spec|__tests__/**/*.EXT, **/*.[sS]pec|test.EXT, **/[*-]test[-*].EXT, !helpers|fixtures, !_*
     - EXT is .js[x]|ts[x]
  - always exclude:
     - snapshot files
     - node_modules, bower_components, jspm_packages

Tests selection:
  - CONF.skip REGEXP_ARR (-s|--skip):
     - partial matching
     - on the CLI, is 'REGEXP' case-insensitive
     - matched against test titles
     - def: /[skip]/i
     - do not run
     - do not report, except in summary
     - do not make snapshots "removed"
     - also skip with CONF.remove|input true
     - --tap should use SKIP directive
  - CONF.only REGEXP_ARR (-o|--only):
     - like adding CONF.skip true to all other tests
     - def: /[only]/i
     - priority over CONF.skip
     - file-wise, i.e. user might need to use files selection too

Test declaration:
  - once all tests of a worker have finished, do not terminate it
     - let worker end on its own (i.e. no more macrotasks) or on global timeout
     - reason: allow afterAll() hooks
  - if a worker does not define any tests during the initial synchronous load, make whole tests run fail
     - reason: avoiding spawning lots of useless workers because of misconfiguration
  - if a test is declared twice by different workers, do not error, but only run test once

Use worker threads / workers for each test file

Any Node flags should be allowed as CLI flags:
  - it should be passed as `execArgv` to workers
  - should whitelist them, so that wrong CLI flags still trigger a validation error
     - do not include flags that do not make sense like --interactive
  - including --require
     - make sure it works with "ts-node/register"
     - document it
  - including --inspect[-brk]:
     - make sure it works
     - document it
  - --experimental-repl-await should always be passed

CONF.timeout NUM (in secs):
  - def: 10 minutes
  - global timeout, because per-test timeout does not work with parallel tests
  - when hit, report which tests are still running

User interrupt:
  - on SIGINT|SIGTERM|SIGQUIT|etc.
  - if no tests finished in last 5 secs
  - then report which tests are still running
  - goal: allow debugging hanging code

CONF.bail BOOL (-b|--bail)
  - stops on first failure
     - still run|report other data of same test
  - "stops":
     - terminate workers
     - do not run|report any additional tests

Assertion testing:
  - any assertion library works:
     - just need to throw error
     - core 'assert', power-assert, chai, should.js
  - does not store|use any snapshot
  - failure reporting:
     - ERROR.name + ERROR.message
     - stack trace (in order):
        - remove node_modules/** and Node internals
        - apply source maps
        - strip common root from path
        - need to work with different browsers
        - see if libraries exist that do this (probably stacktrace.js)
     - preview the source code context:
        - like ava does
        - after source maps applied
        - only of the deepest stack call
        - do not do it if lines are very long, which probably indicates a concatenated build

CONF.update BOOL (-u|--update):
  - if false (def):
     - do not add|remove|modify snapshots in snapshot files
     - fail tests if any snapshot new|removed|different
  - if true:
     - add|remove|modify snapshots in snapshot files
     - always succeed snapshot tests
     - exception: "inconstant" snapshots

CONF.remove BOOL (-r|--remove):
  - remove snapshots in snapshot files
  - do not run tests nor spawn workers
  - no reporting
  - only for the target snapshots|tests: not for skipped tests or not selected files
  - not compatible with CONF.update|input true

CONF.input BOOL (-i|--input):
  - goal: test-first development for snapshots
  - not compatible with CONF.update|remove true
  - only report existing snapshot tests, i.e. do not report:
      - assertion tests
      - removed tests
      - "process" errors
  - file|test selection applies as normal
  - result type always "input"
  - content is either:
      - if new test: empty
      - otherwise: current snapshot
         - reported the same way as "same" snapshots
         - we are reporting currently persisted snapshots only (as opposed to snapshot() return value), so it's clear that only inputs are updating them
  - tests are not run (not needed)
      - but files are loaded, so we know which tests are defined (including new ones)
  - each result should prompt:
      - if submit without entering anything:
         - noop
         - must distinguish from explicitely entering undefined
      - if submit something:
         - updates snapshot in snapshot file with it
         - do not erase the prompt, so user can see previous prompts
         - moves to next one
      - use Node REPL ("readline" or "repl" core module)
      - can multiple statements (use the last one)
      - can use history (arrow up) to re-use previous inputs
      - retry if invalid input

Result types:
   - "pass" assertion (light green)
   - "fail" assertion (light red)
   - "new" snapshots (dark green)
   - "removed" snapshots (dark red)
   - "same" snapshots (i.e. unchanged) (gray)
   - "different" snapshots (same snapshot key but different snapshot) (dark yellow)
   - "renamed" snapshots (only file path or test title changed) (dark cyan)
   - "inconstant" snapshots (current test code created different snapshots when run several times) (dark magenta)
   - "input" snapshots (CONF.input) (light cyan)
   - "process" errors (light purple)

"removed" snapshots:
  - only when file selection targets either:
     - the removed file (even though it does not exist anymore)
     - all files (files that exist and have snapshots)

"renamed" result type:
   - challenges (document those in code comments):
      - renaming files: tests might also have been renamed|added|removed
      - renaming tests: to identify, cannot reliably use:
         - FUNC.name: arrow function name is ""
         - FUNC.name|toString(): tests might be wrapped, i.e. identical wrapper will show for several tests
            - including FUNC.bind()
         - FUNC.data: it is common for the same set of data to be re-used across tests
         - index in the file: adding one test at beginning change whole order
      - streaming issues:
         - if comparing snapshots content: must wait for all tests of a given file to complete, before reporting the first one
         - if comparing files: must wait for all files to synchronously load, before reporting the first one
   - current solution only works if:
      - test was passing and still is
      - only the test title or the file was changed
      - for file changing, only works if whole file was removed
   - matching:
      - any "new" test
      - with any "removed" test that look the same
      - for all data in the test
   - removed tests:
      - are coming either from:
         - removed files: known before starting workers
         - removed test in current file: known because tests are defined synchronously, and tests start running on next tick
      - should not be reported until the end, in case they are actually "renamed"
   - comparing tests:
      - either same test title or file
      - same data + FUNC.name + FUNC.toString():
         - persisted in snapshot files as 8 hex chars hash
      - same snapshots
   - streaming:
      - when a snapshot looks like it should be marked as "renamed" but we still need to wait for snapshots
      - then pause streaming of that test until we know the answer
   - report them with:
      - "renamed" result type
      - followed by oldName -> newName
      - test will fail if CONF.update false
      - should try to keep snapshots layout (so there is no git diff)
   - rename the test title or the file path in the snapshot files

"process" errors:
  - use log-process-errors
     - does not make process exit
  - result type "process"
  - reported the same way as assertion errors except:
     - reported at end of test run
     - reported type is "uncaughtException|unhandledRejection|..."
     - filepath "", title "Process errors"
  - CONF.processErrors STR_ARR
     - which types creates failures (others are ignored)
     - def: ["uncaughtException", "unhandledRejection"]
        - default is conservative to avoid annoying users

Reporting:
  - should take as input UI information: snapshots, titles, filepaths, data, whether an error was thrown|rejected, available actions, etc.
  - should not need to handle most logic including:
     - color highlighting
     - truncating
  - counter (like test-openapi): for all reporters except tap
  - end summary: for all reporters

CONF.reporters STR_ARR (-R|--reporter):
  - types:
     - pretty (default)
     - tap
     - json
     - notify
  - only "notify" can be used with others

Reporting ordering:
  - the following applies to all reporters
  - run in parallel but buffer and report in order (like test-openapi)
     - should stream it
  - order should be (by priority):
     - other files > new files > removed files > process errors
     - lexicographical ordering of "shortest filepath" (see below)
     - other tests > new tests > removed tests
     - when test() was called
     - result type: "same" > "different" > "new" > "removed"
     - argument order in parameters

"shortest filepath"
  - what is reported as file path
  - is:
      - if unique: filename without .EXT
      - if differ only by .EXT: filename with .EXT
      - otherwise, prepend first parent dir (and so on) until different (and do not use .EXT)

CONF.level STR:
  - among 'silent', 'error', 'warn', 'info'
  - do not report 'pass|same' result types:
     - for 'error|warn' but not 'info'
  - do not report content (but still report header) of tests, except first failed one of each test + result type
     - for 'error' but not 'warn|info'
  - 'silent': do not report anything
  - for any reporters
  - default value:
     - pretty|tap|notify: 'error'
     - json: 'info'

CONF.color BOOL (--[no-]color):
  - def: chalk default
  - color highlighting in snapshots
  - use a syntax highlighting library
     - so that YAML/HTML/etc. snapshots are highlighted too
     - should work well with prettyFormat()
  - all snapshots types:
     - but for "different" snapshots: not on the added|removed lines. They should instead be backgroundGreen|Red and auto in foreground

CONF.compact BOOL (--[no-]compact)
  - def: true
  - goals:
     - make each test fit a single screen
     - avoiding slowing down machine when test content is big
  - truncate:
     - ERROR.message (in assertion testing)
     - snapshots:
        - OBJ max depth 2
        - ARR max length 4
           - truncate in middle
     - snapshots|ERROR.messages to max lines NUM:
        - truncate in middle
        - newline should be either CRLF or LF
        - NUM is terminal height
           - minus test headers height
           - if cannot guess, use a typical terminal height
        - each series of 80 chars without any newlines should be considered a line:
           - increments the number of "lines"
           - might be truncated in the middle of the line
     - should try not to truncate first:
        - if over max lines, then truncate it
        - if not over max lines, keep it not truncated

t.log(VAL,...)
  - workers should not redirect stdout|stderr to parent
  - if a test is passing and CONF.level 'error|warn' (but not 'silent'), still report it if it contains t.log()
  - --pretty reporting:
     - all lines prefixed with =>
     - in gray
     - multiline formatting, same as snapshot reporting
  - --tap reporting:
     - TAP comment
     - single line
  - if several VAL: reported on separate lines
  - report always after snapshot content

Default reporter:
  - newlines:
     - test iteration with no content
        - no newline before|after
        - reason: more compact and make assertion testing look similar to other test runners
        - e.g. for:
           - "pass|renamed" result types
           - "input" result type on new test
           - non-first errors with CONF.level "warn"
     - test iteration with content:
        - no newline before, but newline after
  - tests with no data iteration:
     - single line:
        - result type:
           - padded so they all have same length
        - "shortest filepath" + test title: no background color, normal foreground color
     - content
  - tests with data iteration:
     - header:
        - once per test
        - 3 lines: bottom-filled color, full-filled color, up-filled color (like test-openapi)
        - color is inverted auto
        - "shortest filepath" + test title
     - subheader:
        - once per test iteration
        - result type: same as above
        - data:
           - space-separated ARGs
           - ARG stringified with prettyFormat()
           - ellipsis in middle of each ARG if needed
              - each ARG gets same amount of max chars
           - for "process", this is an explanatory text (like in log-process-errors)
           - no background color, normal foreground color
     - content
  - any tests:
     - test title: ellipsis in middle if needed
     - content (if any) either:
        - assertion error
        - "process" error
        - snapshot:
           - first column:
              - "removed" snapshots: red minuses for all lines
              - "new" snapshots: green pluses for all lines
              - "different" snapshots: red minuses or green minuses for relevant lines
              - "inconstant" snapshots: like "different" if snapshot exists, "new" otherwise
              - "same|renamed|input" snapshots: nothing
           - for "inconstant": data header + content is reported twice, because two snapshots are shown

Use itself for unit testing, but its latest release (not direct require())

Normalization:
   - to normalize snapshots, user needs to modify the return value of tests before returning it
   - create a separate package with some common normalization methods:
      - separate some of those methods to own package if makes sense, including deep(), compose(), error(), duration()
      - deep(FUNC(VAL)->VAL)->FUNC(OBJ)->OBJ:
         - make a mapping FUNC recursive
         - avoid infinite recursions
         - only enumerated own properties
         - applied to all methods in the package
      - compose(FUNC,...)->FUNC: functional compose
      - trim(STR)->STR
      - error(STR)->STR: replace stack traces (lines starting with \s+at ...) by single "at STACK TRACE"
         - should work on ERROR since error() is deep and should be applied to error.stack
         - should work on CLI error output (i.e. there is no available ERROR instance)
         - should work for any browser
      - colors(STR)->STR: remove colors
      - duration(STR[, STR2])->STR: replace anything that looks like durations by STR2 (def: "100ms")
      - timestamp(STR[, STR2])->STR: replace anything that looks like timestamps by STR2 (with a default of epoch)
      - newline(STR)->STR: CRLF or LF
      - removePath(STR)->STR: remove anything that looks like a path (including Windows PATH)
      - normalizePath(STR)->STR: normalize Windows paths to Unix paths
      - replace(REGEXP, STR)

Fuzz testing:
  - use test-each: NUM, input functions
      - rely on t.indexes to know whether they were originally used
  - CONF.repeat NUM (--repeat):
      - def: 1
      - >= 0
      - append NUM to each set of data, i.e. repeat each test NUM times
  - NUM as data dimension:
      - NUM is absent from test title or snapshot key
      - assertion testing: only report once, either all "pass" or the first one that "fail"
      - snapshot testing: only a single snapshot is created for all iterations:
          - if all the same, use any
          - if any differs: result type "inconstant":
              - no prompt
              - test always fails
              - first two different snapshots are reported
                  - if any snapshot is same as current one, should use it as the first one
  - test title or snapshot key should be appended with ` | ${info.title}`
      - for snapshot testing (not assertion testing), input functions should:
         - use `${FUNC.name}()` (or "function()" if anonymous) in snapshot key instead of generated value
            - generated value should be used in reporting
         - although test takes randomness as input, the test output should be consistent across runs, otherwise each new run will fail

Fuzz data generation:
  - similar to faker but specific to fuzz testing
  - data meant to test different types (and potential issues) of inputs
     - e.g. for numbers: Infinity, -Infinity, NaN, +0, -0, EPSILON, MAX|MIN_SAFE_INTEGER, MAX|MIN_VALUE, 1/3, 1, 10
     - e.g. for strings: get some ideas from big-list-of-naughty-strings but ensure those are printable, and smaller set
  - export:
     - each set of data
     - methods that retrieve random value among each set of data
  - random function should allow a seed as argument
  - try to search for similar packages
  - separate package
  - link to it from documentation of both test-each and snapshot testing tool

Watch mode:
  - CONF.watch BOOL (-w|--watch)
      - def true??? except if all tests passed???
  - re-run files on file change
      - test file change -> only that test file
      - source file or test helper change??? Possible ideas:
         - check filesystem atime to see if file accessed during synchronous load
             - problem: filesystem is global, and worker threads use tine slicing
             - problem: atime does not always work on some OS, or with some filesystem types/options (noatime)
         - checking require.cache or using require hooks:
             - problem: does not work in browsers
             - problem: does not work with ESM
         - static analysis of ESM:
             - problem: does not work with CommonJS
             - problem: does not work with dynamic import()
             - problem: might be harder with transpiled languages like TypeScript (although that might be supported by parsers)
      - cancel current prompt
  - add r/restart action to re-run all tests
  - when run done, keep prompt but remove all actions except quit and restart
     - show end summary first
  - initial run???

Config:
  - test OPTS can also be set as global defaults on CONF

Executing commands:
  - should create a separate repo which is a thin wrapper around execa()
  - returning { code, stdout, stderr[, error] }
  - always use opts.reject false
  - with correct normalization
  - allow using current package CLI name (finds it in `package.json`)
      - see https://github.com/sindresorhus/execa/issues/281

Snapshots???
  - should ideally:
     - use a single file per snapshot
     - be a simple human-friendly flat text file
     - be resistant to linter/IDE/text editors modifications
     - be diff friendly
  - have both Markdown and binary files like ava???
     - use v8.serialize() on binary files???
     - use brotli on binary files???
     - maybe should be normal text file instead of Markdown, because users will look at it in IDE, not browser
  - use sourcemaps to figure out where to put snapshots
     - or only allow overidding snapshot dir instead???
  - should try to keep existing sorting, so that git diff do not show changes due to sorting
  - key should be combination of:
      - "shortest filepath"
      - test title
      - ARR of data serialized with same logic as snapshot

Try to separate snapshot testing into own project that can be used with other test runners???

Should run in browsers too
  - add multi platform capability???

Getting inspirations???
  - from current testing docs
  - should also finish testing docs
<commit_msg>no commit message<commit_after>
                    
   ROADMAP DIFFTEST  
                    



Catchphrase: "Snapshot-driven testing" and/or "Forget test assertions"
  - do not set it until promotion, to avoid idea stealing

Logo:
  - "difftest" and include green + and red - inside letters
  - nice sans-serif non-monospace font

test("TITLE", [...ARR], FUNC(t, ...ARG) [, ...OPTS])->PROMISE
  - must be called synchronously during file load
  - named export `test` (not default export)

snapshot():
  - snapshot testing
  - should return VAL to snapshot
  - if return PROMISE, await it
     - snapshot (including reporting) should include that it's async
  - exceptions or rejected PROMISEs:
     - snapshot the ERROR
     - snapshot (including reporting) should include that it's been thrown|rejected (errored BOOL)
     - serializer should always snapshot ERRORs with String() (i.e. no stack trace)

"TITLE":
   - not optional
   - must be unique within file

...ARR + FUNC: use test-each

FUNC: should be fired on next tick

t OBJ:
  - index, indexes: like test-each
  - name STR
  - title STR: name + stringified data used for reporting
  - snapshot BOOL: whether snapshot() or test() was called
  - log(VAL,...)

Returned PROMISE:
  - resolved value:
      - OBJ:
         - title STR, args: ARR_ARR, options OPTS
         - tests OBJ_ARR: success BOOL, type 'new|removed|same|different|inconstant', action 'next|update|remove|move|input', value VAL, snapshot STR, args ARR, async BOOL, errored BOOL
         - others???
      - meant for plugins.
  - always resolved, even on test failure

Re-using:
   - TITLE (like describe()): test = (title, ...args) => test(`TITLE > ${title}`, ...args)
   - ...ARR: test = (title, ...args) => test(title, ...ARR, ...args)
   - OPTS: test = (...args) => test(...args, OPTS)
       - which is why OPTS is variadic

Hooks not needed:
   - beforeAll():
      - just execute code in top-level scope
      - for async, top-level code can return PROMISE then each test can use await PROMISE
   - afterAll(): use await Promise.all(...PROMISE) using returned PROMISEs
   - before|afterEach(): just repeat code at beginning|end of FUNC

CLI runner:
  - called "dt" to make it fast to type

Files selection:
  - use CLI argument "GLOB"... (whitelisting)
     - can be "!GLOB" (blacklisting)
     - can target DIRs
  - def: test[s]|spec|__tests__/**/*.EXT, **/*.[sS]pec|test.EXT, **/[*-]test[-*].EXT, !helpers|fixtures, !_*
     - EXT is .js[x]|ts[x]
  - always exclude:
     - snapshot files
     - node_modules, bower_components, jspm_packages

Tests selection:
  - CONF.skip REGEXP_ARR (-s|--skip):
     - partial matching
     - on the CLI, is 'REGEXP' case-insensitive
     - matched against test titles
     - def: /[skip]/i
     - do not run
     - do not report, except in summary
     - do not make snapshots "removed"
     - also skip with CONF.remove|input true
     - --tap should use SKIP directive
  - CONF.only REGEXP_ARR (-o|--only):
     - like adding CONF.skip true to all other tests
     - def: /[only]/i
     - priority over CONF.skip
     - file-wise, i.e. user might need to use files selection too

Test declaration:
  - once all tests of a worker have finished, do not terminate it
     - let worker end on its own (i.e. no more macrotasks) or on global timeout
     - reason: allow afterAll() hooks
  - if a worker does not define any tests during the initial synchronous load, make whole tests run fail
     - reason: avoiding spawning lots of useless workers because of misconfiguration
  - if a test is declared twice by different workers, do not error, but only run test once

Use worker threads / workers for each test file

Any Node flags should be allowed as CLI flags:
  - it should be passed as `execArgv` to workers
  - should whitelist them, so that wrong CLI flags still trigger a validation error
     - do not include flags that do not make sense like --interactive
  - including --require
     - make sure it works with "ts-node/register"
     - document it
  - including --inspect[-brk]:
     - make sure it works
     - document it
  - --experimental-repl-await should always be passed

CONF.timeout NUM (in secs):
  - def: 10 minutes
  - global timeout, because per-test timeout does not work with parallel tests
  - when hit, report which tests are still running

User interrupt:
  - on SIGINT|SIGTERM|SIGQUIT|etc.
  - if no tests finished in last 5 secs
  - then report which tests are still running
  - goal: allow debugging hanging code

CONF.bail BOOL (-b|--bail)
  - stops on first failure
     - still run|report other data of same test
  - "stops":
     - terminate workers
     - do not run|report any additional tests

Assertion testing:
  - any assertion library works:
     - just need to throw error
     - core 'assert', power-assert, chai, should.js
  - does not store|use any snapshot
  - failure reporting:
     - ERROR.name + ERROR.message
     - stack trace (in order):
        - remove node_modules/** and Node internals
        - apply source maps
        - strip common root from path
        - need to work with different browsers
        - see if libraries exist that do this (probably stacktrace.js)
     - preview the source code context:
        - like ava does
        - after source maps applied
        - only of the deepest stack call
        - do not do it if lines are very long, which probably indicates a concatenated build

CONF.update BOOL (-u|--update):
  - if false (def):
     - do not add|remove|modify snapshots in snapshot files
     - fail tests if any snapshot new|removed|different
  - if true:
     - add|remove|modify snapshots in snapshot files
     - always succeed snapshot tests
     - exception: "inconstant" snapshots

CONF.remove BOOL (-r|--remove):
  - remove snapshots in snapshot files
  - do not run tests nor spawn workers
  - no reporting
  - only for the target snapshots|tests: not for skipped tests or not selected files
  - not compatible with CONF.update|input true

CONF.input BOOL (-i|--input):
  - goal: test-first development for snapshots
  - not compatible with CONF.update|remove true
  - only report existing snapshot tests, i.e. do not report:
      - assertion tests
      - removed tests
      - "process" errors
  - file|test selection applies as normal
  - result type always "input"
  - content is either:
      - if new test: empty
      - otherwise: current snapshot
         - reported the same way as "same" snapshots
         - we are reporting currently persisted snapshots only (as opposed to snapshot() return value), so it's clear that only inputs are updating them
  - tests are not run (not needed)
      - but files are loaded, so we know which tests are defined (including new ones)
  - each result should prompt:
      - if submit without entering anything:
         - noop
         - must distinguish from explicitely entering undefined
      - if submit something:
         - updates snapshot in snapshot file with it
         - do not erase the prompt, so user can see previous prompts
         - moves to next one
      - use Node REPL ("readline" or "repl" core module)
      - can multiple statements (use the last one)
      - can use history (arrow up) to re-use previous inputs
      - retry if invalid input

Result types:
   - "pass" assertion (light green)
   - "fail" assertion (light red)
   - "new" snapshots (dark green)
   - "removed" snapshots (dark red)
   - "same" snapshots (i.e. unchanged) (gray)
   - "different" snapshots (same snapshot key but different snapshot) (dark yellow)
   - "renamed" snapshots (only file path or test title changed) (dark cyan)
   - "inconstant" snapshots (current test code created different snapshots when run several times) (dark magenta)
   - "input" snapshots (CONF.input) (light cyan)
   - "process" errors (light purple)

"removed" snapshots:
  - only when file selection targets either:
     - the removed file (even though it does not exist anymore)
     - all files (files that exist and have snapshots)

"renamed" result type:
   - challenges (document those in code comments):
      - renaming files: tests might also have been renamed|added|removed
      - renaming tests: to identify, cannot reliably use:
         - FUNC.name: arrow function name is ""
         - FUNC.name|toString(): tests might be wrapped, i.e. identical wrapper will show for several tests
            - including FUNC.bind()
         - FUNC.data: it is common for the same set of data to be re-used across tests
         - index in the file: adding one test at beginning change whole order
      - streaming issues:
         - if comparing snapshots content: must wait for all tests of a given file to complete, before reporting the first one
         - if comparing files: must wait for all files to synchronously load, before reporting the first one
   - current solution only works if:
      - test was passing and still is
      - only the test title or the file was changed
      - for file changing, only works if whole file was removed
   - matching:
      - any "new" test
      - with any "removed" test that look the same
      - for all data in the test
   - removed tests:
      - are coming either from:
         - removed files: known before starting workers
         - removed test in current file: known because tests are defined synchronously, and tests start running on next tick
      - should not be reported until the end, in case they are actually "renamed"
   - comparing tests:
      - either same test title or file
      - same data + FUNC.name + FUNC.toString():
         - persisted in snapshot files as 8 hex chars hash
      - same snapshots
   - streaming:
      - when a snapshot looks like it should be marked as "renamed" but we still need to wait for snapshots
      - then pause streaming of that test until we know the answer
   - report them with:
      - "renamed" result type
      - followed by oldName -> newName
      - test will fail if CONF.update false
      - should try to keep snapshots layout (so there is no git diff)
   - rename the test title or the file path in the snapshot files

"process" errors:
  - use log-process-errors
     - does not make process exit
  - result type "process"
  - reported the same way as assertion errors except:
     - reported at end of test run
     - reported type is "uncaughtException|unhandledRejection|..."
     - filepath "", title "Process errors"
  - CONF.processErrors STR_ARR
     - which types creates failures (others are ignored)
     - def: ["uncaughtException", "unhandledRejection"]
        - default is conservative to avoid annoying users

Reporting:
  - should take as input UI information: snapshots, titles, filepaths, data, whether an error was thrown|rejected, available actions, etc.
  - should not need to handle most logic including:
     - color highlighting
     - truncating
  - counter (like test-openapi): for all reporters except tap
  - end summary: for all reporters

CONF.reporters STR_ARR (-R|--reporter):
  - types:
     - pretty (default)
     - tap
     - json
     - notify
  - only "notify" can be used with others

Reporting ordering:
  - the following applies to all reporters
  - run in parallel but buffer and report in order (like test-openapi)
     - should stream it
  - order should be (by priority):
     - other files > new files > removed files > process errors
     - lexicographical ordering of "shortest filepath" (see below)
     - other tests > new tests > removed tests
     - when test() was called
     - result type: "same" > "different" > "new" > "removed"
     - argument order in parameters

"shortest filepath"
  - what is reported as file path
  - is:
      - if unique: filename without .EXT
      - if differ only by .EXT: filename with .EXT
      - otherwise, prepend first parent dir (and so on) until different (and do not use .EXT)

CONF.level STR:
  - among 'silent', 'error', 'warn', 'info'
  - do not report 'pass|same' result types:
     - for 'error|warn' but not 'info'
  - do not report content (but still report header) of tests, except first failed one of each test + result type
     - for 'error' but not 'warn|info'
  - 'silent': do not report anything
  - for any reporters
  - default value:
     - pretty|tap|notify: 'error'
     - json: 'info'

CONF.color BOOL (--[no-]color):
  - def: chalk default
  - color highlighting in snapshots
  - use a syntax highlighting library
     - so that YAML/HTML/etc. snapshots are highlighted too
     - should work well with prettyFormat()
  - all snapshots types:
     - but for "different" snapshots: not on the added|removed lines. They should instead be backgroundGreen|Red and auto in foreground

CONF.compact BOOL (--[no-]compact)
  - def: true
  - goals:
     - make each test fit a single screen
     - avoiding slowing down machine when test content is big
  - truncate:
     - ERROR.message (in assertion testing)
     - snapshots:
        - OBJ max depth 2
        - ARR max length 4
           - truncate in middle
     - snapshots|ERROR.messages to max lines NUM:
        - truncate in middle
        - newline should be either CRLF or LF
        - NUM is terminal height
           - minus test headers height
           - if cannot guess, use a typical terminal height
        - each series of 80 chars without any newlines should be considered a line:
           - increments the number of "lines"
           - might be truncated in the middle of the line
     - should try not to truncate first:
        - if over max lines, then truncate it
        - if not over max lines, keep it not truncated

t.log(VAL,...)
  - workers should not redirect stdout|stderr to parent
  - if a test is passing and CONF.level 'error|warn' (but not 'silent'), still report it if it contains t.log()
  - --pretty reporting:
     - all lines prefixed with =>
     - in gray
     - multiline formatting, same as snapshot reporting
  - --tap reporting:
     - TAP comment
     - single line
  - if several VAL: reported on separate lines
  - report always after snapshot content

Default reporter:
  - newlines:
     - test iteration with no content
        - no newline before|after
        - reason: more compact and make assertion testing look similar to other test runners
        - e.g. for:
           - "pass|renamed" result types
           - "input" result type on new test
           - non-first errors with CONF.level "warn"
     - test iteration with content:
        - no newline before, but newline after
  - tests with no data iteration:
     - single line:
        - result type:
           - padded so they all have same length
        - "shortest filepath" + test title: no background color, normal foreground color
     - content
  - tests with data iteration:
     - header:
        - once per test
        - 3 lines: bottom-filled color, full-filled color, up-filled color (like test-openapi)
        - color is inverted auto
        - "shortest filepath" + test title
     - subheader:
        - once per test iteration
        - result type: same as above
        - data:
           - space-separated ARGs
           - ARG stringified with prettyFormat()
           - ellipsis in middle of each ARG if needed
              - each ARG gets same amount of max chars
           - for "process", this is an explanatory text (like in log-process-errors)
           - no background color, normal foreground color
     - content
  - any tests:
     - test title: ellipsis in middle if needed
     - content (if any) either:
        - assertion error
        - "process" error
        - snapshot:
           - first column:
              - "removed" snapshots: red minuses for all lines
              - "new" snapshots: green pluses for all lines
              - "different" snapshots: red minuses or green minuses for relevant lines
              - "inconstant" snapshots: like "different" if snapshot exists, "new" otherwise
              - "same|renamed|input" snapshots: nothing
           - for "inconstant": data header + content is reported twice, because two snapshots are shown

Use itself for unit testing, but its latest release (not direct require())

Normalization:
   - to normalize snapshots, user needs to modify the return value of tests before returning it
   - create a separate package with some common normalization methods:
      - separate some of those methods to own package if makes sense, including deep(), compose(), error(), duration()
      - deep(FUNC(VAL)->VAL)->FUNC(OBJ)->OBJ:
         - make a mapping FUNC recursive
         - avoid infinite recursions
         - only enumerated own properties
         - applied to all methods in the package
      - compose(FUNC,...)->FUNC: functional compose
      - trim(STR)->STR
      - error(STR)->STR: replace stack traces (lines starting with \s+at ...) by single "at STACK TRACE"
         - should work on ERROR since error() is deep and should be applied to error.stack
         - should work on CLI error output (i.e. there is no available ERROR instance)
         - should work for any browser
      - colors(STR)->STR: remove colors
      - duration(STR[, STR2])->STR: replace anything that looks like durations by STR2 (def: "100ms")
      - timestamp(STR[, STR2])->STR: replace anything that looks like timestamps by STR2 (with a default of epoch)
      - newline(STR)->STR: CRLF or LF
      - removePath(STR)->STR: remove anything that looks like a path (including Windows PATH)
      - normalizePath(STR)->STR: normalize Windows paths to Unix paths
      - replace(REGEXP, STR)

Fuzz testing:
  - use test-each: NUM, input functions
      - rely on t.indexes to know whether they were originally used
  - CONF.repeat NUM (--repeat):
      - def: 1
      - >= 0
      - append NUM to each set of data, i.e. repeat each test NUM times
  - NUM as data dimension:
      - NUM is absent from test title or snapshot key
      - assertion testing: only report once, either all "pass" or the first one that "fail"
      - snapshot testing: only a single snapshot is created for all iterations:
          - if all the same, use any
          - if any differs: result type "inconstant":
              - no prompt
              - test always fails
              - first two different snapshots are reported
                  - if any snapshot is same as current one, should use it as the first one
  - test title or snapshot key should be appended with ` | ${info.title}`
      - for snapshot testing (not assertion testing), input functions should:
         - use `${FUNC.name}()` (or "function()" if anonymous) in snapshot key instead of generated value
            - generated value should be used in reporting
         - although test takes randomness as input, the test output should be consistent across runs, otherwise each new run will fail

Fuzz data generation:
  - similar to faker but specific to fuzz testing
  - data meant to test different types (and potential issues) of inputs
     - e.g. for numbers: Infinity, -Infinity, NaN, +0, -0, EPSILON, MAX|MIN_SAFE_INTEGER, MAX|MIN_VALUE, 1/3, 1, 10
     - e.g. for strings: get some ideas from big-list-of-naughty-strings but ensure those are printable, and smaller set
  - export:
     - each set of data
     - methods that retrieve random value among each set of data
  - random function should allow a seed as argument
  - try to search for similar packages
  - separate package
  - link to it from documentation of both test-each and snapshot testing tool

Watch mode:
  - CONF.watch BOOL (-w|--watch)
      - def true??? except if all tests passed???
  - re-run files on file change
      - test file change -> only that test file
      - source file or test helper change??? Possible ideas:
         - check filesystem atime to see if file accessed during synchronous load
             - problem: filesystem is global, and worker threads use tine slicing
             - problem: atime does not always work on some OS, or with some filesystem types/options (noatime)
         - checking require.cache or using require hooks:
             - problem: does not work in browsers
             - problem: does not work with ESM
         - static analysis of ESM:
             - problem: does not work with CommonJS
             - problem: does not work with dynamic import()
             - problem: might be harder with transpiled languages like TypeScript (although that might be supported by parsers)
      - cancel current prompt
  - add r/restart action to re-run all tests
  - when run done, keep prompt but remove all actions except quit and restart
     - show end summary first
  - initial run???

Config:
  - test OPTS can also be set as global defaults on CONF

Executing commands:
  - should create a separate repo which is a thin wrapper around execa()
  - returning { code, stdout, stderr[, error] }
  - always use opts.reject false
  - with correct normalization
  - allow using current package CLI name (finds it in `package.json`)
      - see https://github.com/sindresorhus/execa/issues/281

Snapshots???
  - should ideally:
     - use a single file per snapshot
     - be a simple human-friendly flat text file
     - be resistant to linter/IDE/text editors modifications
     - be diff friendly
  - have both Markdown and binary files like ava???
     - use v8.serialize() on binary files???
     - use brotli on binary files???
     - maybe should be normal text file instead of Markdown, because users will look at it in IDE, not browser
  - use sourcemaps to figure out where to put snapshots
     - or only allow overidding snapshot dir instead???
  - should try to keep existing sorting, so that git diff do not show changes due to sorting
  - key should be combination of:
      - "shortest filepath"
      - test title
      - ARR of data serialized with same logic as snapshot

Try to separate snapshot testing into own project that can be used with other test runners???

Should run in browsers too
  - add multi platform capability???

Getting inspirations???
  - from current testing docs
  - should also finish testing docs

{'fraction_non_alphanumeric': 0.08887603195442711, 'fraction_numerical': 0.0012906384209706492, 'mean_word_length': 2.546121676004103, 'pattern_counts': {'":': 8, '<': 3, '<?xml version=': 0, '>': 61, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '334511', 'n_tokens_mistral': 13889, 'n_tokens_neox': 13485, 'n_words': 5611}