<commit_before>\section{Restart and Failure Scenarios}
Comprehension greatly benefits from writing out explicitly what should happen after a failure\footnote{Something we learned the hard way}.

\subsection{General power failure, after which the master is dead meat}
\tablehead{
  \multicolumn{2}{|c|}{A}& 
  \multicolumn{2}{|c|}{B}&
  \multicolumn{2}{|c|}{C}
  \\\hline}
\begin{supertabular}{|cl|cl|cl|}
  000 & $^s$MasterSet('C',\ldots) & 000 & $^s$ MasterSet('C',\ldots) & 000 & MasterSet('C',\ldots) \\
  001 & $^t$Set('x','X')          & 001 & $^t$ Set('x','X')          & 001 & $^{s,t}$Set('x','X')   \\
\hline
\end{supertabular}
\paragraph{}
The table shows the situation when the power goes off. 
Node C became master and dictated 1 update (\emph{Set('x','X')}), received \emph{Accepted} messages from all slaves,
after which it pushed that update to its store. 
The question is, what happens when node C is lost and A and B are started again?

\paragraph{}
It's obvious that globally, there should be consensus on the \emph{Set} update, but since there was no follow up, 
both nodes don't realise this yet, and are trying to find out what should happen.
A sends out a \emph{Prepare(n,i=1)} to which B answers with a \emph{Promise(n,i=1,Set('x','X'))}.
All goes well, and A receives the promise, and thus decides that this is indeed the value for \emph{i=1}.
It pushes the value, to the store. 
Node A goes into a state identical to C, before it became defunct.
\paragraph{}
If node A gets this far, it's a de-facto leader, and can start acting like it. 
The next thing A does is broadcast \emph{Accept(n,2,MasterSet('A',\ldots)} and things will soon be normal again.
<commit_msg>no progress scenario in manifest<commit_after>\section{Restart and Failure Scenarios}
Comprehension greatly benefits from writing out explicitly what should happen after a failure\footnote{Something we learned the hard way}.

\subsection{General power failure, after which the master is dead meat}
\tablehead{
  \multicolumn{2}{|c|}{A}& 
  \multicolumn{2}{|c|}{B}&
  \multicolumn{2}{|c|}{C}
  \\\hline}
\begin{supertabular}{|cl|cl|cl|}
  000 & $^s$MasterSet('C',\ldots) & 000 & $^s$ MasterSet('C',\ldots) & 000 & MasterSet('C',\ldots) \\
  001 & $^t$Set('x','X')          & 001 & $^t$ Set('x','X')          & 001 & $^{s,t}$Set('x','X')   \\
\hline
\end{supertabular}
\paragraph{}
The table shows the situation when the power goes off. 
Node C became master and dictated 1 update (\emph{Set('x','X')}), received \emph{Accepted} messages from all slaves,
after which it pushed that update to its store. 
The question is, what happens when node C is lost and A and B are started again?

\paragraph{}
It's obvious that globally, there should be consensus on the \emph{Set} update, but since there was no follow up, 
both nodes don't realise this yet, and are trying to find out what should happen.
A sends out a \emph{Prepare(n,i=1)} to which B answers with a \emph{Promise(n,i=1,Set('x','X'))}.
All goes well, and A receives the promise, and thus decides that this is indeed the value for \emph{i=1}.
It pushes the value, to the store. 
Node A goes into a state identical to C, before it became defunct.
\paragraph{}
If node A gets this far, it's a de-facto leader, and can start acting like it. 
The next thing A does is broadcast \emph{Accept(n,2,MasterSet('A',\ldots)} and things will soon be normal again.

\subsection{Unfortunate sequence of events}
\tablehead{
  \multicolumn{2}{|c|}{N1}& 
  \multicolumn{2}{|c|}{N2}&
  \multicolumn{2}{|c|}{N3}
  \\\hline}
\begin{supertabular}{|l|l|l|l|l|l|}
  i-1 & $^s$  v0 & i-1 & $^t$ v0 & i-1 & $^t$ v0\\\hline
  i   & $^t$  v1 &   i &         &   i &        \\
\hline
\end{supertabular}
\paragraph{}
A the starting situation is reached by a cluster where N1, being master accepted \emph{v1} at \emph{i} after which aferything goes down.
After this, there are some problems with the master node (N1) and only N2 and N3 are restarted.
They come up, resume service, and accept and reach consensus on a new value \emph{v2}.
\\
\tablehead{
  \multicolumn{2}{|c|}{N1}& 
  \multicolumn{2}{|c|}{N2}&
  \multicolumn{2}{|c|}{N3}
  \\\hline}
\begin{supertabular}{|l|l|l|l|l|l|}
  i-1 & $^s$  v0 & i-1 & $^s$ v0 & i-1 & $^s$ v0\\\hline
  i   & $^t$  v1 &   i & $^t$ v2 &   i & $^t$ v2\\
\hline
\end{supertabular}
\paragraph{}
Then, disaster strikes again, and both N2 and N3 are stopped. 
There seems to be a problem with N3, but the sysads did a great job and restored N1.
Both N1 and N2 are started. 
Now N1 promises v1 to N2 and N2 promises v2 to N1. 
Neither can reach consensus and progress is blocked.



{'fraction_non_alphanumeric': 0.14149924389717, 'fraction_numerical': 0.02117087923957658, 'mean_word_length': 3.9150743099787686, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2595918', 'n_tokens_mistral': 1608, 'n_tokens_neox': 1531, 'n_words': 616}