<commit_before>\section{The AMIDST modelling framework}\label{section:AMIDSTmodelClass}

One of the main goals of the AMIDST project is the definition of a general modelling framework with the following characteristics: 

\begin{itemize}
\item \textbf{Feature 1:} it should be applicable to the three considered use cases, i.e., Daimler, Cajamar and Verdande;

\item \textbf{Feature 2:}  it should be general enough to be applicable to any future, potentially similar, use cases;

\item \textbf{Feature 3:} it should allow the integration of domain/expert knowledge about the structure of the process being modelled, and

\item \textbf{Feature 4:} it should be scalable, supporting both inference and learning from massive data streams.

\end{itemize}


%When designing this model class, we also tried to define a model family that eases as much as possible the integration of domain/expert knowledge, which is a common place in the different application scenarios discussed in Section \ref{Section:PreliminaryModels}.

The AMIDST modelling framework has been designed with the goal of meeting these four requirements. It is introduced in
the following two sections. In Section \ref{StaticFramework}, we firstly discuss the static modelling framework that
covers the static models presented in the related application scenarios. In Section \ref{GeneralModelClass}, we
introduce the general AMIDST modelling framework as a dynamic extension of the previous static modelling framework. As
proof of concept of the generality of this modelling framework, we show how the framework can be instantiated to solve each of the application scenarios of the different use cases. The graphical notation introduced in Section \ref{SubSection:GraphicalNotation} will be employed when describing these probabilistic graphical models. Finally, a summary with a higher level view of the AMIDST modelling framework is included in Section \ref{summaryAMIDSTModels}.

%Taking these different characteristics into account and using the graphical notation introduced in Section \ref{SubSection:GraphicalNotation}, the general AMIDST model class, as well as its specific instantiations to each use case, are introduced and discussed in Section \ref{GeneralModelClass}. A summary with a higher level view of the AMIDST model class is finally included in Section \ref{summaryAMIDSTModels}.

%-----------------------------------------------------------------------------------------------------------------------------------------
\subsection{The static modelling framework}\label{StaticFramework}
%-----------------------------------------------------------------------------------------------------------------------------------------

Figure \ref{Figure:StaticModellingFramework} visually shows the proposed static modelling framework. This general model
is the result of combining the high-level structure of the static models described for two of the application scenarios discussed in Section \ref{Section:PreliminaryModels}. This static modelling framework has been designed having in mind that it should be easily extendable to a dynamic context. As it can be seen, subnetworks have been used to group variables with similar properties, so that the commonalities between all models are taken into account. 

\begin{figure}[ht!]
\begin{center}
\includegraphics[scale=0.4]{./figures/StaticModellingFramework}
\caption{\label{Figure:StaticModellingFramework} The static modelling framework.}
\end{center}
\end{figure}

The first main characteristic of this static model is that all the modelling variables are structured in three different
layers, each one with distinct semantics. We believe that this layered division of the modelling variables can
help facilitate the integration of domain/expert knowledge. These three layers can be described as follows:


\begin{itemize}

\item \textbf{Control/class layer:}  This upper layer can be instantiated as a target variable for classification
  models. Alternatively, it can  act as a ``control'' layer, by containing specific variables in the systems that
  directly or causally influence the rest of the variables in the model. This ``control'' interpretation will gain significance in the dynamic context.

The layer is represented by a subnetwork including either continuous or discrete observed variables. The links from the
continuous observed variables to the state subnetworks in the \textit{Hidden layer} (i.e., the set of ``State 1'' and
``State 2'' variables) will possibly be handled with logit and probit functions.

\item \textbf{Hidden layer:}  This layer attempts to capture those parts of the system/process that can not be directly
  observed or measured, but which is believed (e.g.\ based on domain/expert knowledge) to be part of the system/process. Moreover, we can also use this layer as a way to capture complex conditional dependencies and distributions (e.g. mixture of Gaussians) that might be present between variables in the observable layer. 

This layer is structured as a set of interconnected discrete and continuous hidden subnetworks, for which only links from discrete to continuous subnetworks are allowed. We will use the term \textit{state} variables to refer to the set of discrete hidden variables in both ``State 1'' and ``State 2'' subnetworks, and the term \textit{latent}\footnote{The term \textit{latent} for variables is generally used in statistics to refer to hidden variables as opposed to observed ones. However, we will use it here to exclusively refer to continuous hidden variables.} variables to refer to the set of continuous hidden variables in the ``Latent'' subnetwork. The distinction between the subnetworks ``State 1'' and ``State 2'' will be better understood in the dynamic context. 

\item \textbf{Observable layer:} This layer accounts for those variables modelling the part of the system/process that can be observed/measured. These observations are conditioned on the variables in the above two layers. 

This layer is represented by a subnetwork including both discrete and continuous observed variables. These variables can in principle be interconnected but, in our different use cases, only links from discrete to continuous nodes are required. However, in general, we would not need to restrict the direction of the links between the variables in this layer.

\end{itemize} 


According to the discussions in Section \ref{Section:PreliminaryModels} on the distribution families governing the use
cases, we have designed a modelling framework that can be parametrized in alternative ways to improve the expressibility
and applicability to different problem domains. This parametrization is also influenced by the possible impact it can
have in the inference and learning algorithms. In general, this modelling framework falls inside the conditional linear
Gaussian framework, and so the conditional probability distributions are parameterized as detailed in Section
\ref{SubSection:HybridBNs}. On the other hand, the modelling framework might also contain instantiations which are not
covered by this general CLG framework. More precisely, when the top layer class is instantiated to  a continuous
subnetwork, then the conditional probabilities of ``State 1'' and ``State 2'' may be instantiated with logistic or
probit distributions, because we encounter continuous parents with discrete children. Additionally, we also envision the
possibility of introducing in the observable layer, the use of probability distributions belonging to the MOTBF family
\cite{Langseth12}, in order to extend the expressive power of the modelling framework (see Section \ref{SubSection:EmpiricalDistributions}).

Full details about how this framework can be instantiated to solve the different application scenarios will be given in the next section. But, as an illustrative example, we show in Figure \ref{Figure:StaticModellingInstantiations} how this static modelling framework can be instantiated to the static models presented for Daimler (see Section \ref{Section:Daimler:EarlyRecognition}) and Cajamar (see Section \ref{SubSection:Predicting}). In the case of Verdande, no static models has been considered. 


\begin{figure}[ht!]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.4]{./figures/DaimlerStaticModelling}
&
\includegraphics[scale=0.4]{./figures/CajaMarStaticModelling}
\\Daimler Static Model & CajaMar Static Model \\
\end{tabular}
\caption{\label{Figure:StaticModellingInstantiations} An illustrative example with instantiations of the static modelling framework for the Daimler and Cajamar static models, respectively.}
\end{center}
\end{figure}
%-----------------------------------------------------------------------------------------------------------------------------------------
\subsection{The general AMIDST modelling framework}\label{GeneralModelClass}<commit_msg>no message<commit_after>\section{The AMIDST modelling framework}\label{section:AMIDSTmodelClass}

One of the main goals of the AMIDST project is the definition of a general modelling framework with the following characteristics: 

\begin{itemize}
\item \textbf{Feature 1:} it should be applicable to the three considered use cases, i.e., Daimler, Cajamar and Verdande;

\item \textbf{Feature 2:}  it should be general enough to be applicable to any future, potentially similar, use cases;

\item \textbf{Feature 3:} it should allow the integration of domain/expert knowledge about the structure of the process being modelled, and

\item \textbf{Feature 4:} it should be scalable, supporting both inference and learning from massive data streams.

\end{itemize}


%When designing this model class, we also tried to define a model family that eases as much as possible the integration of domain/expert knowledge, which is a common place in the different application scenarios discussed in Section \ref{Section:PreliminaryModels}.

The AMIDST modelling framework has been designed with the goal of meeting these four requirements. It is introduced in
the following two sections. In Section \ref{StaticFramework}, we firstly discuss the static modelling framework that
covers the static models presented in the related application scenarios. In Section \ref{GeneralModelClass}, we
introduce the general AMIDST modelling framework as a dynamic extension of the previous static modelling framework. As
proof of concept of the generality of this modelling framework, we show how the framework can be instantiated to solve each of the application scenarios of the different use cases. The graphical notation introduced in Section \ref{SubSection:GraphicalNotation} will be employed when describing these probabilistic graphical models. Finally, a summary with a higher level view of the AMIDST modelling framework is included in Section \ref{summaryAMIDSTModels}.

%Taking these different characteristics into account and using the graphical notation introduced in Section \ref{SubSection:GraphicalNotation}, the general AMIDST model class, as well as its specific instantiations to each use case, are introduced and discussed in Section \ref{GeneralModelClass}. A summary with a higher level view of the AMIDST model class is finally included in Section \ref{summaryAMIDSTModels}.

%-----------------------------------------------------------------------------------------------------------------------------------------
\subsection{The static modelling framework}\label{StaticFramework}
%-----------------------------------------------------------------------------------------------------------------------------------------

Figure \ref{Figure:StaticModellingFramework} visually shows the proposed static modelling framework. This general model
is the result of combining the high-level structure of the static models described for two of the application scenarios discussed in Section \ref{Section:PreliminaryModels}. This static modelling framework has been designed having in mind that it should be easily extendable to a dynamic context. As it can be seen, subnetworks have been used to group variables with similar properties, so that the commonalities between all models are taken into account. 

\begin{figure}[ht!]
\begin{center}
\includegraphics[scale=0.4]{./figures/StaticModellingFramework}
\caption{\label{Figure:StaticModellingFramework} The static modelling framework.}
\end{center}
\end{figure}

The first main characteristic of this static model is that all the modelling variables are structured in three different
layers, each one with distinct semantics. We believe that this layered division of the modelling variables can
help facilitate the integration of domain/expert knowledge. These three layers can be described as follows:


\begin{itemize}

\item \textbf{Control/class layer:}  This upper layer can be instantiated as a target variable for classification
  models. Alternatively, it can  act as a ``control'' layer, by containing specific variables in the systems that
  directly or causally influence the rest of the variables in the model. This ``control'' interpretation will gain significance in the dynamic context.

The layer is represented by a subnetwork including either continuous or discrete observed variables. The links from the
continuous observed variables to the state subnetworks in the \textit{Hidden layer} (i.e., the set of ``State 1'' and
``State 2'' variables) will possibly be handled with logit and probit functions.

\item \textbf{Hidden layer:}  This layer attempts to capture those parts of the system/process that can not be directly
  observed or measured, but which is believed (e.g.\ based on domain/expert knowledge) to be part of the system/process. Moreover, we can also use this layer as a way to capture complex conditional dependencies and distributions (e.g. mixture of Gaussians) that might be present between variables in the observable layer. 

This layer is structured as a set of interconnected discrete and continuous hidden subnetworks, for which only links from discrete to continuous subnetworks are allowed. We will use the term \textit{state} variables to refer to the set of discrete hidden variables in both ``State 1'' and ``State 2'' subnetworks, and the term \textit{latent}\footnote{The term \textit{latent} for variables is generally used in statistics to refer to hidden variables as opposed to observed ones. However, we will use it here to exclusively refer to continuous hidden variables.} variables to refer to the set of continuous hidden variables in the ``Latent'' subnetwork. The distinction between the subnetworks ``State 1'' and ``State 2'' will be better understood in the dynamic context. 

\item \textbf{Observable layer:} This layer accounts for those variables modelling the part of the system/process that can be observed/measured. These observations are conditioned on the variables in the above two layers. 

This layer is represented by a subnetwork including both discrete and continuous observed variables. These variables can in principle be interconnected but, in our different use cases, only links from discrete to continuous nodes are required. However, in general, we would not need to restrict the direction of the links between the variables in this layer.

\end{itemize} 


According to the discussions in Section \ref{Section:PreliminaryModels} on the distribution families governing the use
cases, we have designed a modelling framework that can be parametrized in alternative ways to improve the expressibility
and applicability to different problem domains. This parametrization is also influenced by the possible impact it can
have in the inference and learning algorithms. In general, this modelling framework falls inside the conditional linear
Gaussian framework, and so the conditional probability distributions are parameterized as detailed in Section
\ref{SubSection:HybridBNs}. On the other hand, the modelling framework might also contain instantiations which are not
covered by this general CLG framework. More precisely, when the top layer class is instantiated to  a continuous
subnetwork, then the conditional probabilities of ``State 1'' and ``State 2'' may be instantiated with logistic or
probit distributions, because we encounter continuous parents with discrete children. Additionally, we also envision the
possibility of introducing in the observable layer, the use of probability distributions belonging to the MOTBF family
\cite{Langseth12}, in order to extend the expressive power of the modelling framework (see Section \ref{SubSection:EmpiricalDistributions}).

Full details about how this framework can be instantiated to solve the different application scenarios will be given in the next section. But, as an illustrative example, we show in Figure \ref{Figure:StaticModellingInstantiations} how this static modelling framework can be instantiated to the static models presented for Daimler (see Section \ref{Section:Daimler:EarlyRecognition}) and Cajamar (see Section \ref{SubSection:Predicting}). In the case of Verdande, no static models has been considered. 


\begin{figure}[ht!]
\begin{center}
\begin{tabular}{cc}
\includegraphics[scale=0.4]{./figures/DaimlerStaticModelling}
&
\includegraphics[scale=0.4]{./figures/CajaMarStaticModelling}
\\Daimler static model & Cajamar static model \\
\end{tabular}
\caption{\label{Figure:StaticModellingInstantiations} An illustrative example with instantiations of the static modelling framework for the Daimler and Cajamar static models, respectively.}
\end{center}
\end{figure}
%-----------------------------------------------------------------------------------------------------------------------------------------
\subsection{The general AMIDST modelling framework}\label{GeneralModelClass}
{'fraction_non_alphanumeric': 0.09228518372629566, 'fraction_numerical': 0.0022508581396657474, 'mean_word_length': 4.139386928860613, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '975130', 'n_tokens_mistral': 3902, 'n_tokens_neox': 3734, 'n_words': 2246}