<commit_before>\chapter{Bootstrapping}

\section{General technique}

\sysname{} is bootstrapped from an existing \commonlisp{}
implementation that, in addition to the functionality required by the
standard, also contains the library \texttt{closer-mop}.  This
\commonlisp{} system is called the \emph{host}.  The result of the
bootstrapping process is an \emph{image} in the form of an executable
file containing a \sysname{} system.  This system is called the
\emph{target}.  The target image does not contain a complete
\commonlisp{} system.  It only contains enough functionality to load
the remaining system from compiled files.
\seesec{sec-bootstrapping-viable-image}.

In general, the target image can be thought of as containing
\emph{graph} of \commonlisp{} objects that have been placed in memory
according to the spaces managed by the memory manager.  To create this
graph, we first generate an \emph{isomorphic} graph of host objects in
the memory of an executing host system.  To generate the target image,
the isomorphic host graph is traversed, creating a target version of
each object in the host graph, and placing that object on an
appropriate address in the target image.

The isomorphic host graph contains objects that are \emph{analogous}
to their target counterparts as follows:

\begin{itemize}
\item A target \texttt{fixnum} is represented as a host integer.
  Whether the integer is a host fixnum or not depends on the fixnum
  range of the host.
\item A target \texttt{character} is represented as a host character.
\item A target \texttt{cons} cell is represented as a host
  \texttt{cons} cell.
\item A target heap-allocated object other than a \texttt{cons} cell
  is represented as a host \texttt{struct} for the \emph{header} and a
  host \texttt{simple-vector} for the \emph{rack}.
\item Target objects such as bignums or floats are not needed at this
  stage of bootstrapping, so they do not have any representation as
  host objects.
\end{itemize}

\section{Global environments for bootstrapping}

During different stages of bootstrapping, a particular \emph{name} (of
a function, class, etc) must be associated with different objects.  As
a trivial example, the function \texttt{allocate-object} in the host
system is used to allocate all standard objects.  But
\texttt{allocate-object} is also a target function for allocating
objects according to the way such objects are represented by the
target system.  These two functions must be available simultaneously.

Most systems solve this problem by using temporary names for target
packages during the bootstrapping process.  For example, even though
in the final target system, the name \texttt{allocate-object} must be
a symbol in the \texttt{common-lisp} package, during the bootstrapping
process, the name might be a symbol in a package with a different
name.

In \sysname{} we solve the problem by using multiple \emph{first-class
  global environments}.

\section{Viable image}
\label{sec-bootstrapping-viable-image}

An image I is said to be \emph{viable} if and only if it is possible
to obtain a complete \commonlisp{} system by starting with I and loading a
sequence of ordinary compiled files.

There are many ways of making a viable image.  We are interested in
making a \emph{small} viable image.  In particular, we want the
initial image \emph{not to contain the compiler}, because we want to
be able to load the compiler as a bunch of compiled files. 

By requiring the viable image not to contain the compiler, we place
some restrictions on the code in it.  

One such restriction is that creating the discriminating function of a
generic function is not allowed to invoke the compiler.  Since doing
so is typical in any CLOS implementation, we must find a different
way.

Another such restriction concerns the creation of classes.  Typically,
when a class is created, the reader and writer methods are created by
invoking the compiler.  Again, a different way must be found.

In order to create an instance of a class, the following functions
are called, so they must exist and be executable:
<commit_msg>Explain that EVAL = COMPILE + TIE.<commit_after>\chapter{Bootstrapping}

\section{General technique}

\sysname{} is bootstrapped from an existing \commonlisp{}
implementation that, in addition to the functionality required by the
standard, also contains the library \texttt{closer-mop}.  This
\commonlisp{} system is called the \emph{host}.  The result of the
bootstrapping process is an \emph{image} in the form of an executable
file containing a \sysname{} system.  This system is called the
\emph{target}.  The target image does not contain a complete
\commonlisp{} system.  It only contains enough functionality to load
the remaining system from compiled files.
\seesec{sec-bootstrapping-viable-image}.

In general, the target image can be thought of as containing
\emph{graph} of \commonlisp{} objects that have been placed in memory
according to the spaces managed by the memory manager.  To create this
graph, we first generate an \emph{isomorphic} graph of host objects in
the memory of an executing host system.  To generate the target image,
the isomorphic host graph is traversed, creating a target version of
each object in the host graph, and placing that object on an
appropriate address in the target image.

The isomorphic host graph contains objects that are \emph{analogous}
to their target counterparts as follows:

\begin{itemize}
\item A target \texttt{fixnum} is represented as a host integer.
  Whether the integer is a host fixnum or not depends on the fixnum
  range of the host.
\item A target \texttt{character} is represented as a host character.
\item A target \texttt{cons} cell is represented as a host
  \texttt{cons} cell.
\item A target heap-allocated object other than a \texttt{cons} cell
  is represented as a host \texttt{struct} for the \emph{header} and a
  host \texttt{simple-vector} for the \emph{rack}.
\item Target objects such as bignums or floats are not needed at this
  stage of bootstrapping, so they do not have any representation as
  host objects.
\end{itemize}

\section{Global environments for bootstrapping}

During different stages of bootstrapping, a particular \emph{name} (of
a function, class, etc) must be associated with different objects.  As
a trivial example, the function \texttt{allocate-object} in the host
system is used to allocate all standard objects.  But
\texttt{allocate-object} is also a target function for allocating
objects according to the way such objects are represented by the
target system.  These two functions must be available simultaneously.

Most systems solve this problem by using temporary names for target
packages during the bootstrapping process.  For example, even though
in the final target system, the name \texttt{allocate-object} must be
a symbol in the \texttt{common-lisp} package, during the bootstrapping
process, the name might be a symbol in a package with a different
name.

In \sysname{} we solve the problem by using multiple \emph{first-class
  global environments}.

For the purpose of bootstrapping, it is convenient to think of
\texttt{eval} as consisting of two distinct operations:

\begin{itemize}
\item Compile.  A \emph{compilation environment} is used to expand
  macros and for other compilation purposes.  The result of
  compilation is code that is \emph{untied} to any particular
  environment.
\item Tie.  The untied code produced by the first step is \emph{tied}
  to a particular run-time environment.  Tying is accomplished by
  calling the top-level function created by the compilation, passing
  it the result of evaluating \texttt{load-time-value} forms in a
  particular environment.
\end{itemize}

The reason we need to separate these two operations is that for
bootstrapping purposes, the two are going to use distinct global
environments.

\section{Viable image}
\label{sec-bootstrapping-viable-image}

An image I is said to be \emph{viable} if and only if it is possible
to obtain a complete \commonlisp{} system by starting with I and loading a
sequence of ordinary compiled files.

There are many ways of making a viable image.  We are interested in
making a \emph{small} viable image.  In particular, we want the
initial image \emph{not to contain the compiler}, because we want to
be able to load the compiler as a bunch of compiled files. 

By requiring the viable image not to contain the compiler, we place
some restrictions on the code in it.  

One such restriction is that creating the discriminating function of a
generic function is not allowed to invoke the compiler.  Since doing
so is typical in any CLOS implementation, we must find a different
way.

Another such restriction concerns the creation of classes.  Typically,
when a class is created, the reader and writer methods are created by
invoking the compiler.  Again, a different way must be found.

In order to create an instance of a class, the following functions
are called, so they must exist and be executable:

{'fraction_non_alphanumeric': 0.05038201749529399, 'fraction_numerical': 0.0, 'mean_word_length': 4.411623726782505, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5611488', 'n_tokens_mistral': 2300, 'n_tokens_neox': 2269, 'n_words': 1336}