<commit_before>Moss Design Overview
====================

This document captures core design principles and concepts for the Moss build system.

Overview
--------

Moss produces completed software artifacts from source code.
The core concepts required by this process are:

1. Code - the source code everything is formed from
2. Spore - related code that will be formed together into a build artifact
3. Artifact - software build product with content from one or more spores

Tools are required to produce artifacts from code.
All tools with Moss are used to perform one of three actions:

1. Transcode: source code -> source code (M.transcode?)
2. Compile: source code -> object code (M.compile?)
3. Form: source or object code + artifacts -> artifact (M.form?)

Tools are specialized according to:

- Architecture - how source is compiled for target machine and environment
- Variant - global variations on compiling

Spores
------

The spore is the fundamental unit of modular software artifact production in Moss.
A spore describes how Moss produces artifacts from code in an architecture and toolchain independent fashion.

Spores are defined entirely through variable definition.
Creation and management of rules and dependencies is managed automatically by Moss.
Knowledge of toolchain and architecture specifics are decoupled from the description of the software itself.
Moss manages the creation and application of all makefile rules required for multiple languages, artifacts, architectures, toolchains, and variants.
This avoids many pitfalls of traditional make-based build systems.

Each spore is produced in isolation.
This guarantees that object and artifact files will never be clobbered, even if multiple spores use the same source code in different ways.
Spore artifacts are produced in a way that encourages creation of small libraries that are combined to produce complex artifacts.

Several design concepts are being considered to manage the multitude of spore varieties and artifacts that are required for non-trivial applications:

1. Varieties of spores through inheritance of common variables. A spore could be created that inherits from another spore but allows overriding specific definitions.

2. Configuration of spores through setting of variables (think menuconfig or kconfig). This would allow for specific conditional switches within the spore to turn options on or off.

3. Selection and application of architecture and tool optoins for building spores. A given spore can be built for multiple architectures while keeping architecture-specific options out of the general spore definition.

These concepts are each different and should be kept as orthogonal as possible. It is highly likely that option 3 is the only one needed initially.

Languages
---------

A language is named after its source form (for example, C or Rust).
The Moss language description defines the outputs that will be created (for example, object files or other source language files).
Translation is performed into a language from another source.
Compilation is performed from a language into an object file.
Moss manages the chaining of translation and compilation to ultimately produce and object file from source file as required.

Each toolchain supports at least one language that is used by source files.
Many toolchains support multiple languages (e.g. GCC and LLVM).

Artifacts
---------

Three types of artifacts are envisioned as default.

- Executable: a fully-linked binary executable
- Static Library: a library for static linking into one or more executables
- Shared Library: a dynamic library for use with one or more executable artifacts

Most artifacts will require creation of obejct files using a compiler.
Some artifacts may be produced directly from source files.

Architectures
-------------

An architecture definition specifies a toolchain. For example armv5-gcc would specify a gcc cross compiler. And armv5-clang would specify a clang cross compile. In order to build the same spore for the same processor with two different tools, two different architectures must be created.

Architecture is the mechanism for customized compilation of spores for a particular target. Different spores compiled for the same architecture may have different toolchain settings. The architecture is the means of selection. Spores compiled for the same architecture may be linked together. A mixing of architectures is not supported.

A build architecture is defined by a specific CPU type and associated instruction set.

Examples: host, amd64, cortex-m4, cortex-r7, avr, mipsel, etc.

Code generated for different architectures does not interoperate and cannot be linked together.

- `M.ARCHS`: All available architectures for current toolchain
- `M.ARCH`: Current architecture for current toolchain

Good discussion on architecture naming here: http://clang.llvm.org/docs/CrossCompilation.html

Variants
--------

A build variant is a selection that applies across all architectures and toolchains (in most cases). An example would be debug vs production.

Variants provide the mechanism for compiling multiple spores for the same architecture with different collections of settings that apply to all spores. Example use cases that apply here are debug, production, or special test build variants.

Toolchains
----------

A toolchain is a set of tools that are used to translate source code into binary form for execution on a specific architecture.

Toolchain settings are chosen based upon architecture and variant. Additional customization may be overlaid for each spore based upon architecture.

Code generated by one toolchain cannot be linked or combined with code generated for another toolchain unless it is explicitly forced in spore dependencies.

Examples: gcc, clang, gcc-debug, clang-release, clang-debug.

While a toolchain must include all tools required to produce an executable binary or library output, the name of the toolchain is generally based upon the compiler.

Generally, debug versus release build variants are handled at the toolchain level due to the significant impact that may be present on generated code.

- `M.TOOLS`: All available toolchains
- `M.TOOL`: Current toolchain

A toolchain uses both architecture and variant to customize settings to the specific code generation tools.
<commit_msg>Review & edit design notes<commit_after>Moss Design Overview
====================

This document captures core design principles and concepts for the Moss build system.

Moss produces completed software artifacts from source code.
The core concepts required by this process are:

1. Code - the source code everything is formed from
2. Spore - related code that will be formed together into a build artifact
3. Artifact - software build product with content from one or more spores

Tools are required to produce artifacts from code.
All tools with Moss are used to perform one of three actions:

1. Transcode: source code -> source code (M.transcode?)
2. Compile: source code -> object code (M.compile?)
3. Form: source or object code + artifacts -> artifact (M.form?)

Tools are specialized according to:

- Architecture - how source is compiled for target machine and execution environment
- Variant - global variations on compiling

Spores
------

The spore is the fundamental unit of modular software artifact production in Moss.
A spore describes how Moss produces artifacts from code in an architecture and toolchain independent fashion.

Spores are defined entirely through variable definition.
Creation and management of rules and dependencies is managed automatically by Moss.
Knowledge of toolchain and architecture specifics are decoupled from the description of the software itself.
Moss manages the creation and application of all makefile rules required for multiple languages, artifacts, architectures, toolchains, and variants.
This avoids many pitfalls of traditional make-based build systems.

Each spore is produced in isolation.
This guarantees that object and artifact files will never be clobbered, even if multiple spores use the same source code in different ways.
Spore artifacts are produced in a way that encourages creation of small libraries that are combined to produce complex artifacts.

Several design concepts are being considered to manage the multitude of spore varieties and artifacts that are required for non-trivial applications:

1. Varieties of spores through inheritance of common variables. A spore could be created that inherits from another spore but allows overriding specific definitions.

2. Configuration of spores through setting of variables (think menuconfig or kconfig). This would allow for specific conditional switches within the spore to turn options on or off.

3. Selection and application of architecture and tool optoins for building spores. A given spore can be built for multiple architectures while keeping architecture-specific options out of the general spore definition.

These concepts are each different and should be kept as orthogonal as possible. It is highly likely that option 3 is the only one needed initially.
Option 2 will become important for more complex spores or spores that have to be configured differently across architectures.
Option 1 may not be required at all if 2 and 3 are flexible enough.

Code and Languages
------------------

A language is named after its source form (for example, C or Rust).
The Moss language description defines the outputs that will be created (for example, object files or other source language files).
Translation is performed into a language from another source.
Compilation is performed from a language into an object file.
Moss manages the chaining of translation and compilation to ultimately produce and object file from source file as required.

Each toolchain supports at least one language that is used by source files.
Many toolchains support multiple languages (e.g. GCC and LLVM).

Artifacts
---------

Three types of artifacts are envisioned as default.

- Executable: a fully-linked binary executable
- Static Library: a library for static linking into one or more executables
- Shared Library: a dynamic library for use with one or more executable artifacts

Most artifacts will require creation of obejct files using a compiler.
Some artifacts may be produced directly from source files.

Tools
-----

A tool is used to form artifacts from code.

A toolchain is a set of tools that are used for a specific architecture.
A tool uses both architecture and variant to customize settings to the specific code generation tools.
Additional customization may be overlaid for each spore based upon architecture.

Code generated by one toolchain cannot be linked or combined with code generated for another toolchain unless it is explicitly forced through spore dependencies.

Generally, debug versus release build variants are handled at the tool level due to the significant impact that may be present on generated code.

Transcode
~~~~~~~~~

The output is generated source code, not object code.

Compile
~~~~~~~

The output is object code.

Form
~~~~

The output is a build artifact that may contain object code or files.


Architectures
-------------

An architecture definition specifies a toolchain. For example armv5-gcc would specify a gcc cross compiler. And armv5-clang would specify a clang cross compile. In order to build the same spore for the same processor with two different tools, two different architectures must be created.

Architecture is the mechanism for customized compilation of spores for a particular target. Different spores compiled for the same architecture may have different toolchain settings. The architecture is the means of selection. Spores compiled for the same architecture may be linked together. A mixing of architectures is not supported.

A build architecture is defined by a specific CPU type and associated instruction set.

Examples: host, amd64, cortex-m4, cortex-r7, avr, mipsel, etc.

Code generated for different architectures does not interoperate and cannot be linked together.

- `M.ARCHS`: All available architectures for current toolchain
- `M.ARCH`: Current architecture for current toolchain

Good discussion on architecture naming here: http://clang.llvm.org/docs/CrossCompilation.html

Variants
--------

A build variant is a selection that applies across all architectures and toolchains (in most cases). An example would be debug vs production.

Variants provide the mechanism for compiling multiple spores for the same architecture with different collections of settings that apply to all spores. Example use cases that apply here are debug, production, or special test build variants.
{'fraction_non_alphanumeric': 0.04182389937106918, 'fraction_numerical': 0.002830188679245283, 'mean_word_length': 4.4855541181543765, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6361104', 'n_tokens_mistral': 2889, 'n_tokens_neox': 2669, 'n_words': 1846}