<commit_before># Principles Document

This library is an experiment and not intended for use. See the
[README](README.md).

## High Level

1. Defaults matter. The right, and safe, thing should always require less typing
   than the maybe-wrong thing.
1. Memory safety should be the default. You don't pay (in bugs and CVEs) for
   premature optimization you didn't need.
    * Leaving this path should require more typing, and a clear signal what is
      happening. We will use either a naming convention, or a marker argument
      type, to do so in place of [a language keyword](
      https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/unsafe).
1. KISS Principle
    * "most systems work best if they are kept simple rather than made
      complicated; therefore, simplicity should be a key goal in design, and
      unnecessary complexity should be avoided" - most systems work best if they
      are kept simple rather than made complicated; therefore, simplicity should
      be a key goal in design, and unnecessary complexity should be avoided
    * Consider inherent vs accidental complexity and avoid the latter.
1. Principle of Least Astonishment
    * A component of a system should behave in a way that most users will expect
      it to behave. The behavior should not astonish or surprise users.
1. Single-choice
    * "Whenever a software system must support a set of alternatives, one and
      only one module in the system should know their exhaustive list." -
      Bertrand Meyer: Object-Oriented Software Construction
1. Persistence-Closure
    * "Whenever a storage mechanism stores an object, it must store with it the
      dependents of that object. Whenever a retrieval mechanism retrieves a
      previously stored object, it must also retrieve any dependent of that
      object that has not yet been retrieved." - Bertrand Meyer: Object-Oriented
      Software Construction
1. Single-responsibility principle
    * "Every module, class or function in a computer program should have
      responsibility over a single part of that program's functionality, and it
      should encapsulate that part. All of that module, class or function's
      services should be narrowly aligned with that responsibility." -
      [Wikipedia](https://en.wikipedia.org/wiki/Single-responsibility_principle)
1. Use the latest C++ language features to provide the simplest APIs possible.
1. Accept incompatibility. No attempt is made for backward compatibility with C,
   earlier versions of C++, compatibility with stdlib, etc.
    * When compatibility or conversion is desirable, it will be done through
      explicit functions.
1. Surprising, and hidden, performance cliffs are bad; we will avoid them.
1. Provide monadic functions to chain together functionality, and types that
   play well with them.
    * Monadic functions allow for functional programming, which results in less
      complex code: fewer program states are possible when functions have no
      side effects.
    * Monadic functions can provide general building blocks that allow for the
      construction of a wide variety of software from a small set of tools.

## Nitty gritty.

1. No public constructors, except for the default constructor. All other
   construction is through (named) static methods.
    * Avoid a default constructor when other constructors are present. Instead,
      name it `with_default()`.
    * Non-default constructors are prefixed with `with_()` unless there's a
      clear and better domain-specific name.
    * Would like to extend this to Copy and Move constructors too, through Clone
      and Move methods. Implementation details required.
1. No use-after-move. All types abort if they are used after a move.
1. No implicit conversions, ever.
1. No function overloads. Const is the default.
    * When there are const and mutable versions of a method, use the `_mut`
      suffix on the mutable one to distinguish them. The const method gets the
      shorter name.
    * Don't use a `get_` prefix on getters. Do use a prefix like `is_` to
      differentiate from a verb, like `is_empty()`.
    * Use an appropriate prefix like `set_`, or a clear verb name, to
      distinguish setters.
    * No default parameters, as they are a form of overloading.
    * Constructors (public copy/move, or private constructors) and assignment
      operators are the exception for overloads.
1. Functions that return a reference are lvalue-reference-qualified (with `&` or
   `const&`). If they are `const&`-qualified, then `=delete` the `&&` override
   to avoid const rvalues from using the method.
1. Bounds are always checked unless you explicitly ask for them to not be.
1. Lifetimes are always checked unless you explicitly ask for them to not be.
1. Small headers. C++ compilation speed is proportional to the amount of input.
    * We will split types into separate headers as much as possible. tokens.
    * We may provide headers which group together smaller headers for simplicity
      in smaller projects.
    * We will move implementations out of headers, and out of templates, to
      reduce header sizes where possible and where it does not have strong
      performance implications.
1. We will use doxygen to document all types, functions, and public methods.
    * Documentation will include examples where use is non-trivial.
    * All functions that can abort() will document under what conditions they do
      so, with the exception of: use-after-move.
1. No null pointers. Smart pointers have no null state. Functions never return
    null pointers.
1. No raw pointers by default. Raw pointers throw away all lifetime information,
    can produce memory safety bugs. And raw pointers can be null, which is
    undesirable.
1. No expensive and implicit copies.
    * Copies should generally be explicit. If we opt into implicit copies, they
      should be cheap - and
      [trivial](https://en.cppreference.com/w/cpp/types/is_trivially_copyable).
1. No surprising heap allocations. Types should live on the stack whenever
    possible. They only use internal heap allocations when strictly required:
    for instance, because they have a dynamic size. Instead, the user can choose
    what lives on the heap through the use of heap-based smart pointers.
1. Deep constness. Const methods do not mutate state in visible ways.
    * No const methods return non-const pointers or references.
    * No const methods call non-const methods through pointers or references.
    * No const_cast usage.
    * State in members marked `mutable` does not escape the class from const
      methods.
    * No mutable pointers or references constructed from fields, or reachable
      from fields, in const methods.
    * Pointer and reference fields accessed through a const class pointer are
      only used as const pointers and references.
    * We will use tooling to enforce this for the library, and it will be
      made available for user code.
    * Prior art and motivations: https://isocpp.org/files/papers/N4388.html
    * TODO: Maybe there's a more concise way to say this.
1. No static mutable state.
    * Static mutable state introduces potential thread races, and the complexity
      of global knowledge. A `static` variable declaration must also be `const`,
      and if it's a pointer, then "deep constness" applies.
1. No exceptions. Return results instead.
    * Every function and method is `noexcept`.
    * No use of `throw`.
    * Calls to functions in another library that can throw exceptions are
      banned.
    * We will provide vocabulary types for returning a success-or-failure and a
      value-or-none.
1. Traits, SFINAE and type tags to define behaviour on data. No inheritance
   unless from an abstract interface. All other classes are marked `final`.
    * Always use `final` instead of `override`.
    * Inheritance is [no longer
      needed](https://en.cppreference.com/w/cpp/language/ebo) for object sizes.
    * We will provide a public set of traits with defined behaviour and tools or
      guidance to ensure consistent behaviour in implementations of traits.
1. Tools to use SFINAE for common patterns, such as traits, without having to
   understand all of the stdlib type_traits.
1. Use constexpr everywhere it is correct to.
    * Functions or method that can ever perform non-constexpr actions may not be
      marked constexpr, with the exception of: abort().
    * Any other function or method should be marked constexpr.
1. Common C++ language behaviours are part of the class' public API. We will
   document _and test_ for being trivially copyable, movable, destructible,
   among others.

## What's included

1. Safe and predictable integer math, with safe casting and simple names.
    * Starting point: https://github.com/google/integers
    * On float to int conversion: https://blog.m-ou.se/floats/
1. Smart pointers that naturally express ownership, sharing, and threading
   semantics.
    * The smart pointers are not "leaky", and are not (safely) backward
      compatible with raw pointer ownership.
1. Containers with lifetime safety at runtime.
1. Function pointers with additional state bound to them, for purposes of
   currying.
1. Vocabulary types for returning success-or-failure with different embedded
   types, or value-or-none.
1. Tagged unions.
1. We will provide a required set of clang warnings to get expected behaviour
   from the library when working with user-provided types.
    * We will consider different "levels" of grouped warnings if needed.
1. We will provide clang static analysis tools and/or compiler plugins as needed
   to ensure expected behaviour from the library when it acts on user-provided
   types.

### Less clear ideas<commit_msg>Add principles about tests and about method qualifiers<commit_after># Principles Document

This library is an experiment and not intended for use. See the
[README](README.md).

## High Level

1. Defaults matter. The right, and safe, thing should always require less typing
   than the maybe-wrong thing.
1. Memory safety should be the default. You don't pay (in bugs and CVEs) for
   premature optimization you didn't need.
    * Leaving this path should require more typing, and a clear signal what is
      happening. We will use either a naming convention, or a marker argument
      type, to do so in place of [a language keyword](
      https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/unsafe).
1. KISS Principle
    * "most systems work best if they are kept simple rather than made
      complicated; therefore, simplicity should be a key goal in design, and
      unnecessary complexity should be avoided" - most systems work best if they
      are kept simple rather than made complicated; therefore, simplicity should
      be a key goal in design, and unnecessary complexity should be avoided
    * Consider inherent vs accidental complexity and avoid the latter.
1. Principle of Least Astonishment
    * A component of a system should behave in a way that most users will expect
      it to behave. The behavior should not astonish or surprise users.
1. Single-choice
    * "Whenever a software system must support a set of alternatives, one and
      only one module in the system should know their exhaustive list." -
      <NAME>: Object-Oriented Software Construction
1. Persistence-Closure
    * "Whenever a storage mechanism stores an object, it must store with it the
      dependents of that object. Whenever a retrieval mechanism retrieves a
      previously stored object, it must also retrieve any dependent of that
      object that has not yet been retrieved." - <NAME>: Object-Oriented
      Software Construction
1. Single-responsibility principle
    * "Every module, class or function in a computer program should have
      responsibility over a single part of that program's functionality, and it
      should encapsulate that part. All of that module, class or function's
      services should be narrowly aligned with that responsibility." -
      [Wikipedia](https://en.wikipedia.org/wiki/Single-responsibility_principle)
1. Use the latest C++ language features to provide the simplest APIs possible.
1. Accept incompatibility. No attempt is made for backward compatibility with C,
   earlier versions of C++, compatibility with stdlib, etc.
    * When compatibility or conversion is desirable, it will be done through
      explicit functions.
1. Surprising, and hidden, performance cliffs are bad; we will avoid them.
1. Provide monadic functions to chain together functionality, and types that
   play well with them.
    * Monadic functions allow for functional programming, which results in less
      complex code: fewer program states are possible when functions have no
      side effects.
    * Monadic functions can provide general building blocks that allow for the
      construction of a wide variety of software from a small set of tools.

## Nitty gritty.

1. No public constructors, except for the default constructor. All other
   construction is through (named) static methods.
    * Avoid a default constructor when other constructors are present. Instead,
      name it `with_default()`.
    * Non-default constructors are prefixed with `with_()` unless there's a
      clear and better domain-specific name.
    * Would like to extend this to Copy and Move constructors too, through Clone
      and Move methods. Implementation details required.
1. No use-after-move. All types abort if they are used after a move.
1. No implicit conversions, ever.
1. No function overloads. Const is the default.
    * When there are const and mutable versions of a method, use the `_mut`
      suffix on the mutable one to distinguish them. The const method gets the
      shorter name.
    * Don't use a `get_` prefix on getters. Do use a prefix like `is_` to
      differentiate from a verb, like `is_empty()`.
    * Use an appropriate prefix like `set_`, or a clear verb name, to
      distinguish setters.
    * No default parameters, as they are a form of overloading.
    * Constructors (public copy/move, or private constructors) and assignment
      operators are the exception for overloads.
1. Functions that return a reference are lvalue-reference-qualified (with `&` or
   `const&`). If they are `const&`-qualified, then `=delete` the `&&` override
   to avoid const rvalues from using the method.
1. Bounds are always checked unless you explicitly ask for them to not be.
1. Lifetimes are always checked unless you explicitly ask for them to not be.
1. Small headers. C++ compilation speed is proportional to the amount of input.
    * We will split types into separate headers as much as possible. tokens.
    * We may provide headers which group together smaller headers for simplicity
      in smaller projects.
    * We will move implementations out of headers, and out of templates, to
      reduce header sizes where possible and where it does not have strong
      performance implications.
1. We will use doxygen to document all types, functions, and public methods.
    * Documentation will include examples where use is non-trivial.
    * All functions that can abort() will document under what conditions they do
      so, with the exception of: use-after-move.
1. No null pointers. Smart pointers have no null state. Functions never return
    null pointers.
1. No raw pointers by default. Raw pointers throw away all lifetime information,
    can produce memory safety bugs. And raw pointers can be null, which is
    undesirable.
1. No expensive and implicit copies.
    * Copies should generally be explicit. If we opt into implicit copies, they
      should be cheap - and
      [trivial](https://en.cppreference.com/w/cpp/types/is_trivially_copyable).
1. No surprising heap allocations. Types should live on the stack whenever
    possible. They only use internal heap allocations when strictly required:
    for instance, because they have a dynamic size. Instead, the user can choose
    what lives on the heap through the use of heap-based smart pointers.
1. Deep constness. Const methods do not mutate state in visible ways.
    * No const methods return non-const pointers or references.
    * No const methods call non-const methods through pointers or references.
    * No const_cast usage.
    * State in members marked `mutable` does not escape the class from const
      methods.
    * No mutable pointers or references constructed from fields, or reachable
      from fields, in const methods.
    * Pointer and reference fields accessed through a const class pointer are
      only used as const pointers and references.
    * We will use tooling to enforce this for the library, and it will be
      made available for user code.
    * Prior art and motivations: https://isocpp.org/files/papers/N4388.html
    * TODO: Maybe there's a more concise way to say this.
1. No static mutable state.
    * Static mutable state introduces potential thread races, and the complexity
      of global knowledge. A `static` variable declaration must also be `const`,
      and if it's a pointer, then "deep constness" applies.
1. No exceptions. Return results instead.
    * Every function and method is `noexcept`.
    * No use of `throw`.
    * Calls to functions in another library that can throw exceptions are
      banned.
    * We will provide vocabulary types for returning a success-or-failure and a
      value-or-none.
1. Traits, SFINAE and type tags to define behaviour on data. No inheritance
   unless from an abstract interface. All other classes are marked `final`.
    * Always use `final` instead of `override`.
    * Inheritance is [no longer
      needed](https://en.cppreference.com/w/cpp/language/ebo) for object sizes.
    * We will provide a public set of traits with defined behaviour and tools or
      guidance to ensure consistent behaviour in implementations of traits.
1. Tools to use SFINAE for common patterns, such as traits, without having to
   understand all of the stdlib type_traits.
1. Use constexpr everywhere it is correct to.
    * Functions or method that can ever perform non-constexpr actions may not be
      marked constexpr, with the exception of: abort().
    * Any other function or method should be marked constexpr.
1. Common C++ language behaviours are part of the class' public API. We will
   document _and test_ for being trivially copyable, movable, destructible,
   among others.
1. Everything comes with tests. Tests are not flaky.
1. Avoid compiler-specific things whenever possible. Some exceptions:
    * `[[clang:trivial_abi]]` is a perf benefit, once Clang has C++20 support.
    * Compiler builtins for implementing things, to avoid including large stdlib
      headers.
1. Use method qualifiers extensively for improved safety.
    * Methods that return references should be lvalue-qualified. If const-ref
      qualified, `delete` the rvalue-reference overload.
    * Only omit a qualifier when the method makes sense for both lvalue and
      rvalue uses.

## What's included

1. Safe and predictable integer math, with safe casting and simple names.
    * Starting point: https://github.com/google/integers
    * On float to int conversion: https://blog.m-ou.se/floats/
1. Smart pointers that naturally express ownership, sharing, and threading
   semantics.
    * The smart pointers are not "leaky", and are not (safely) backward
      compatible with raw pointer ownership.
1. Containers with lifetime safety at runtime.
1. Function pointers with additional state bound to them, for purposes of
   currying.
1. Vocabulary types for returning success-or-failure with different embedded
   types, or value-or-none.
1. Tagged unions.
1. We will provide a required set of clang warnings to get expected behaviour
   from the library when working with user-provided types.
    * We will consider different "levels" of grouped warnings if needed.
1. We will provide clang static analysis tools and/or compiler plugins as needed
   to ensure expected behaviour from the library when it acts on user-provided
   types.

### Less clear ideas
{'fraction_non_alphanumeric': 0.05176598801208699, 'fraction_numerical': 0.004507851587655422, 'mean_word_length': 3.422343921139102, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 14, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2701403', 'n_tokens_mistral': 4986, 'n_tokens_neox': 4812, 'n_words': 2793}