<commit_before>= Moss Design Overview

This document captures core design principles and concepts for the Moss build system.

Moss produces completed software artifacts from source code.
The core concepts in this process are:

1. Spore - description of how to produce a final build artifact
2. Code - content that must be compiled before being included in the artifact
3. File - content that is directly used while forming the artifact
3. Artifact - software build output with content from one or more spores

Tools are used to produce artifacts from code.
All tools with Moss to perform one of three actions:

1. Transcode: source code -> source code
2. Compile: source code -> object files
3. Form: files + artifacts -> artifact

Tools may be specialized according to:

- Architecture - how source is compiled for target machine and execution environment
- Variant - global variations on compiling

== Spores

A spore describes how Moss produces artifacts from source code and files in an architecture and toolchain independent fashion.
Spores do not possess knowledge of Moss build architecture.
Instead, they are specialized for different architectures through settings.

=== Definition

Spores are defined entirely through variable definition.
Creation and management of rules and dependencies is managed automatically by Moss.
Knowledge of toolchain and architecture specifics are decoupled from the description of the software itself.
Moss manages the creation and application of all makefile rules required for multiple languages, artifacts, architectures, toolchains, and variants.
This avoids many pitfalls of traditional make-based build systems.

=== Specialization

A given spore may produce artifacts for multiple architectures without requiring duplication of the spore definition.
The architecture-specific 'specializations' are created from the base spore 'template'.
Architecture-specific options and overrides can be specified through variables that 'overload' the base definition for the spore.

=== Configuration

This is a formalized way to specify and use conditional boolean switches and named string options to specialize the artifacts produced by a spore.
Configuration is the ideal way to provide the setting that are used during specialization to fine tune the artifacts that are formed.
Think menuconfig or kconfig.

=== Isolation

Each spore is produced in isolation.
This guarantees that object files and artifacts will never be clobbered, even if multiple spores use the source code in different ways.
Spore artifacts are produced in a way that encourages creation of small libraries that are combined to produce complex artifacts.

== Code

Code is compiled before it is included in a final artifact.
It may also be transcoded before this compilation step.
Translation is converts code from one language from another language.
Compilation is converts source code into object code.
Moss manages the chaining of translation and compilation to ultimately produce and object file from source file as required.

== Files

Files are formed into artifacts without being passed through a compiler.
For example, a tarball is simply a collection of files packaged into a single artifact.
Files may also be additional content required for production of executable outputs, such as tool configurations, etc.
Object files are implicitly included in the list of files for an artifact base on the source code.

== Artifacts

Three types of artifacts are envisioned initially.

- Executable: a fully-linked binary executable
- Static Library: a library for static linking into one or more executables
- Shared Library: a dynamic library for use with one or more executable artifacts

Most artifacts will require creation of object files using a compiler.
Some artifacts may be produced directly from source files.

== Tools

Tool are used to form all build artifacts.

A tool does not possess any knowledge of Moss build architecture or variant.
Tools are specialized through architecture and variant by setting appropriate tool options to create appropriate artifacts.

=== Transcode

The output is generated source code, not object code.

=== Compile

The output is object code.

=== Form

The output is a build artifact that may contain object code or files.

== Architectures

An architecture definition specifies a toolchain. For example armv5-gcc would specify a gcc cross compiler. And armv5-clang would specify a clang cross compile. In order to build the same spore for the same processor with two different tools, two different architectures must be created.

Architecture is the mechanism for customized compilation of spores for a particular target. Different spores compiled for the same architecture may have different toolchain settings. The architecture is the means of selection. Spores compiled for the same architecture may be linked together. A mixing of architectures is not supported.

A build architecture is defined by a specific CPU type and associated instruction set.

Examples: host, amd64, cortex-m4, cortex-r7, avr, mipsel, etc.

Good discussion on architecture naming here: http://clang.llvm.org/docs/CrossCompilation.html

== Variants

A build variant is a selection that applies across all architectures and toolchains (in most cases). An example would be debug vs production.

Variants provide the mechanism for compiling multiple spores for the same architecture with different collections of settings that apply to all spores. Example use cases that apply here are debug, production, or special test build variants.<commit_msg>Update design as per latest thinking<commit_after>= Moss Design Overview

Moss is a library for GNU make that streamlines the creation of rules and dependencies for software build artifacts. The core concepts within this library are:

1. Seeds - modular units of source code that are formed into an artifact
2. Artifacts - outputs of seeds which may be created for multiple architectures and variants
3. Tools - used to compile source code and form artifacts
4. Architectures - used to specialize seeds and tools for artifact creation
5. Variants - global switches for tools

== Seeds

A seed describes how Moss produces artifacts from source code and files in an architecture and toolchain independent fashion.
Seeds do not possess knowledge of Moss build architecture.
Instead, they are specialized for different architectures through settings.

=== Definition

Seeds are defined entirely through variable definition.
Creation and management of rules and dependencies is managed automatically by Moss.
Knowledge of toolchain and architecture specifics are decoupled from the description of the software itself.
Moss manages the creation and application of all makefile rules required for multiple languages, artifacts, architectures, toolchains, and variants.
This avoids many pitfalls of traditional make-based build systems.

=== Specialization

A given seed may produce artifacts for multiple architectures without requiring duplication of the seed definition.
The architecture-specific 'specializations' are created from the base seed 'template'.
Architecture-specific options and overrides can be specified through variables that 'overload' the base definition for the seed.

=== Configuration

This is a formalized way to specify and use conditional boolean switches and named string options to specialize the artifacts produced by a seed.
Configuration is the ideal way to provide the setting that are used during specialization to fine tune the artifacts that are formed.
Think menuconfig or kconfig.

=== Isolation

Each seed is produced in isolation.
This guarantees that object files and artifacts will never be clobbered, even if multiple seeds use the source code in different ways.
Seed artifacts are produced in a way that encourages creation of small libraries that are combined to produce complex artifacts.

== Code

Code is compiled before it is included in a final artifact.
It may also be transcoded before this compilation step.
Translation is converts code from one language from another language.
Compilation is converts source code into object code.
Moss manages the chaining of translation and compilation to ultimately produce and object file from source file as required.

== Files

Files are formed into artifacts without being passed through a compiler.
For example, a tarball is simply a collection of files packaged into a single artifact.
Files may also be additional content required for production of executable outputs, such as tool configurations, etc.
Object files are implicitly included in the list of files for an artifact base on the source code.

== Artifacts

Three types of artifacts are envisioned initially.

- Executable: a fully-linked binary executable
- Static Library: a library for static linking into one or more executables
- Shared Library: a dynamic library for use with one or more executable artifacts

Most artifacts will require creation of object files using a compiler.
Some artifacts may be produced directly from source files.

== Tools

Tool are used to form all build artifacts.

A tool does not possess any knowledge of Moss build architecture or variant.
Tools are specialized through architecture and variant by setting appropriate tool options to create appropriate artifacts.

=== Transcode

The output is generated source code, not object code.

=== Compile

The output is object code.

=== Form

The output is a build artifact that may contain object code or files.

== Architectures

An architecture definition specifies a toolchain. For example armv5-gcc would specify a gcc cross compiler. And armv5-clang would specify a clang cross compile. In order to build the same seed for the same processor with two different tools, two different architectures must be created.

Architecture is the mechanism for customized compilation of seeds for a particular target. Different seeds compiled for the same architecture may have different toolchain settings. The architecture is the means of selection. Seeds compiled for the same architecture may be linked together. A mixing of architectures is not supported.

A build architecture is defined by a specific CPU type and associated instruction set.

Examples: host, amd64, cortex-m4, cortex-r7, avr, mipsel, etc.

Good discussion on architecture naming here: http://clang.llvm.org/docs/CrossCompilation.html

== Variants

A build variant is a selection that applies across all architectures and toolchains (in most cases). An example would be debug vs production.

Variants provide the mechanism for compiling multiple seeds for the same architecture with different collections of settings that apply to all seeds. Example use cases that apply here are debug, production, or special test build variants.
{'fraction_non_alphanumeric': 0.02989356779268857, 'fraction_numerical': 0.002221193891716798, 'mean_word_length': 4.772435897435898, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2516482', 'n_tokens_mistral': 2410, 'n_tokens_neox': 2233, 'n_words': 1572}