Property-based Code Slicing for Efficient Verification of OSEK/VDX Operating Systems

Abstract: Testing is a de-facto verification technique in industry, but insufficient for identifying subtle issues due to its optimistic incompleteness. On the other hand, model checking is a powerful technique that supports comprehensiveness, and is thus suitable for the verification of safety-critical systems. However, it generally requires more knowledge and cost more than testing. This work attempts to take advantage of both techniques to achieve integrated and efficient verification of OSEK/VDX-based automotive operating systems. We propose property-based environment generation and model extraction techniques using static code analysis, which can be applied to both model checking and testing. The technique is automated and applied to an OSEK/VDX-based automotive operating system, Trampoline. Comparative experiments using random testing and model checking for the verification of assertions in the Trampoline kernel code show how our environment generation and abstraction approach can be utilized for efficient fault-detection.

\section{Introduction}
The operating system is the core part of automotive control software; any malfunction
  can cause critical errors in the automotive system, which in turn may result in loss of lives and assets.
Testing has been widely used as a systematic and cost-effective safety analysis/assurance method~\cite{Broy:2006,Moessinger:IEEEsoftware10}, but its
optimistic incompleteness often misses critical problems and cannot guarantee the ``absence of wrong behavior".
As an alternative and complimentary technique, model checking~\cite{ModelCheckingBook,SPINBOOK} has been drawing attention from both academia and industry.

Model checking is a comprehensive formal verification technique, suitable for functional safety analysis. It can effectively identify subtle issues, such as
process dead lock, illegal behavior, and starvation, but may require more resources and domain knowledge. In particular, the use of model checking faces the following
challenges:

\begin{enumerate}
\item The size of model/code to be verified needs to be minimized to avoid state-space explosion.
\item Modeling of the environment, such as user tasks and hardware environment, is necessary and critical for embedded software.
\end{enumerate}

Since an operating system is a reactive system responding to environmental stimuli, the correctness of
 its behavior needs to be analyzed with respect to the behavior of its environments. A non-deterministic environment is typically used to over-approximate
 actual behavior, but it is often too expensive in model checking.
 The difficulty and importance of defining
 a {\it good} environment model has been addressed in a number of previous works~\cite{Penix:FMSD05,TDP03,GP02,Yatake:SPIN10,IS02,HP93}.

We note that these two problems apply to both model checking and testing. Though the level of comprehensiveness differs, both techniques rely on automated search techniques that are initiated by environmental stimuli. This is called environment model in model checking and test scenario in testing.
This work anticipates that the efficiency of automated verification techniques depends on the modeling of the environment and proposes
an application of property-based code slicing~\cite{Weiser84} for automatically generating an environment model using the data/function dependency analyzed from the operating system kernels.
The goal is to construct a valid and comprehensive usage model of the operating system with minimal dependency on the kernel code.

Our approach extracts functions that have a direct dependency on a given property to be verified and generates non-deterministic function-call sequences by imposing (1) external constraints from the OSEK/VDX standard~\cite{OSEK/VDX:Portal} for automotive operating systems, and (2) internal constraints identified from the function call structure of the operating system kernel. The external constraints are manually identified from the specifications of the standard and are imposed on the initially random sequence of function calls. The internal constraints are imposed by identifying the top-level functions using backward slicing from a given property and by computing the cone-of-influence from each top-level function using forward slicing.
The Environment model is defined as an arbitrary sequence of calls of those extracted functions. The operating system kernel is also abstracted as a collection of
extracted functions and its relevant code required for pre-processing them. This procedure reduces the size of the verification target and minimizes the behavior of the environment model.
The extraction and model construction process is automated with the aid of the static analysis tool Understand~\cite{Understand}.

The approach and the tool are applied to the verification of safety properties of the Trampoline operating system~\cite{Trampoline:Portal}, which is an open source automotive operating system compliant with OSEK/VDX.
Environment models are generated using the assertions identified from the kernel code, and the kernel code itself is reduced by including only those extracted functions and their relevant code.
The environment model is used to model-check/test the abstract code using CBMC~\cite{CBMC:TACAS04} and random testing. We compare their fault-detection capability, their comprehensiveness in terms of code coverage, and their efficiency in terms of resource consumption.

The remainder of this paper is organized as follows. Section~\ref{sec:related}  briefly discusses related work and Section~\ref{sec:background} provides the motivation for our work. Section~\ref{sec:overview} provides an overview of our approach and
 Section~\ref{sec:env_gen}  presents the methods and the process for the automated environment generation technique. Section~\ref{sec:env_setting} explains the environment settings for the collaborative
  verification, followed by experimental results and the evaluation using Trampoline OS as a case example in Section~\ref{sec:experiment}.
We conclude in Section~\ref{sec:conclusion}.

\section{Related Work}
\label{sec:related}

Environment modeling for efficient model checking has been an active research issue~\cite{Penix:FMSD05,TDP03,GP02,Yatake:SPIN10,IS02,HP93}.
Reference~\cite{HP93} is one of the earliest works concerning environment assumptions in verification. It introduced the $observer$ concept to represent assumptions about the environment. The approaches for assumption generation were developed further in~\cite{GP02,TDP03,Gupta:FMSD08,Nam:FMSD08}. Reference~\cite{TDP03} automatically generates the environment of Java programs from the specifications written by a user. \cite{Nam:FMSD08,Gupta:FMSD08} are concerned about automatic partitioning, learning, or minimizing assumptions for compositional verification. None of them considers environment generation for both
 model checking and testing.

Several specification-based environment generation methods exist: ~\cite{Parizek:EUROMICRO07} uses ADL to define protocols of Java components and constructs an environment
for the ADL specification. \cite{Dwyer:FSE98} describes environmental assumptions in LTL and uses them to filter a universal environment, which is adopted in
our approach to constrain the non-deterministic initial task model.
Reference~\cite{Yatake:SPIN10} automatically generates scripts in {\sc Promela} from environment models for OSEK/VDX-based operating systems that are modeled in {\sc Uml} diagrams.
Their approach, however, models all basic objects in OSEK/VDX using UML class diagrams and state diagrams, from which all combinations of deterministic environments
are generated and verified individually. The models are then used to automatically generate exhaustive test cases for the conformance testing of OSEK/VDX-compliant operating systems~\cite{Chen:APSEC11}.
Their approach assures the exhaustiveness of test cases, but the scalability issue remains, as the number of test cases may increase exponentially.

Program slicing~\cite{Weiser84} has been a popular technique for reducing verification complexity for both model checking and testing.
References~\cite{Binkley99} and \cite{Gupta92anapproach} use slicing algorithms to explicitly detect $def-use$ associations that are affected by a program change for efficient
regression testing. Reference~\cite{Chebaro:SANTE} performs program slicing for C programs with respect to the alarms generated from value analysis.
\cite{He:ICCD07} integrates aggressive program slicing and
a proof-based abstraction-refinement strategy for wireless cognitive radio systems. It is a representative example of using program slicing and bounded model checking for embedded software, but the slicing is integrated into the model checking process, and is thus not suitable for application in testing.

\section{Background}
\label{sec:background}
\subsection{OSEK/VDX}
{\sc Osek/vdx}
is
a joint project of the automotive industry,  which aims at establishing an industry standard for an open-ended architecture for distributed control units in vehicles~\cite{OSEK/VDX:Portal}.
The aim of {\sc Osek/vdx} is to provide standard interfaces independent of application, hardware, and network, and ultimately, to save the development costs for non-application
related aspects of control software.
It is specialized for automotive control systems, removing all undesired complexities such as dynamic memory allocation, circular waiting for resources,
multi-threading, and so on. Since its target system is safety-critical, it strictly prohibits uncontrolled dynamic behavior of the system.

Conformation testing is a standard verification method for the certification of OSEK/VDX-based operating systems. However, conformation testing suites
are typically insufficient to identify safety problems. As OSEK/VDX explicitly specifies more than 26 basic APIs, thorough conformation testing would require at
least $26\times 2\times 3$ test cases even if we assume two arguments per API and only boundary values for the arguments are chosen. The possible number of
execution sequences for these $26\times 2\times 3$ test cases would rise to 156 factorials, a large number to be tested in practice.

\subsection{Trampoline}

Trampoline~\cite{Trampoline:Portal} is an open source, real-time operating system compliant with OSEK/VDX version 2.2.3. It is developed in ANSI C and can be
ported to various hardware platforms such as Arm, POSIX, PPC, AVR, HCS12, C166, etc. Since it also supports POSIX, it can be test-run
on a UNIX/Linux environment before being ported to an actual operational environment.  As its target platform varies, its platform-dependent part is clearly
structured in a separate module that combines with the kernel module at compile time. Access to the hardware-specific part is abstracted using
$extern$ variables and macros so that the main control logic does not need to be aware of the specific hardware feature. As illustrated in Figure~\ref{fig:trampoline},
development of an automotive software using Trampoline requires four components; (1) application source code, (2) kernel configuration generated from
configuration description written in OIL (OSEK Implementation Language) using the Goil compiler, (3) generic OS kernel code compliant with the OSEK/VDX standard, and (4) platform-dependent
kernel code. The generic OS kernel code implements services for task management, resource management, interrupt handling, and event/counter/alarm management,
providing corresponding APIs.

\begin{figure}
\centering
\includegraphics[width=108mm]{trampoline_fig.pdf}
\caption{Components of Trampoline\label{fig:trampoline}
}
\end{figure}

\subsection{Model checking using CBMC}

Formal verification methods based on model checking~\cite{ModelCheckingBook} are an effective technique for identifying subtle issues in
software safety which is particulary important for embedded systems. Current technological advances in model checking enable engineers to
directly apply the technique to program source code, removing the manual model construction process. CBMC~\cite{CBMC:TACAS04} is one of these
model checking tools, which is capable of verifying almost full ANSI C. It can be used to verify buffer overflows, pointer safety, exceptions
and user-specified assertions. Furthermore, it can check ANSI C and C++ for consistency with other languages, such as Verilog. The main
advantage is that it is completely automated and generates counterexample traces when a property in question is refuted.

As with any other model checking tool, CBMC also suffers from the problem of scalability. When applied to the Trampoline kernel as a whole with
an arbitrary sequence of API calls, for example, it ran
out of memory for checking one assertion on a PC with 3GB of memory.

\section{Overall Approach}
\label{sec:overview}

Comprehensive verification, required by functional safety analysis, is too costly to be applied in practice. Reducing the cost while maintaining
comprehensiveness is a challenging, but crucial task.
Our approach attempts to achieve this goal with the following three strategies:

\begin{enumerate}
\item
Property-based environment generation: An environment of the operating system kernel is automatically generated using static code analysis for a given
     safety property.
\item
Property-based abstraction: The operating system kernel is abstracted by extracting only the code relevant to a given property.
\item
Collaborative verification using model checking and testing: Both model checking and testing are used complementarily for the verification of
 the abstract kernel code under the generated environment model.
\end{enumerate}

Cost reduction is achieved through property-based environment generation and code abstraction. The efficiency of verification is increased
by taking advantage of both verification techniques. Figure~\ref{fig:approach} is an overview of the suggested collaborative verification
approach. Our approach uses both model checking and testing to complimentarily utilize their different capabilities when only limited resources are available.

\begin{figure}
\centering
\includegraphics[width=132mm]{approach_fig.pdf}
\caption{Collaborative verification approach\label{fig:approach}
}
\end{figure}

\section{Environment Generation}
\label{sec:env_gen}

A straightforward way to include all possible task interactions with the operating system is to model the task with strongly connected states, where each state represents an API call to the kernel and each transition between states is not guarded.
However, this includes too many spurious and/or impossible behaviors and
increases the cost for verification as well as counterexample analysis; if 26 APIs are provided by the operating system, the task model would have at least 26 strongly connected states.
Our approach tries to minimize unnecessary verification cost by using property-based extraction of dependent functions.

\subsection{Abstraction through static code analysis}
Given a property, we first extract the variables specified in the property, which is called {\it Verification Target Variables}, and identify
all the variables that are used to define the {\it Verification Target Variable}, called {\it Extended Verification Target Variable}.
Then, functions modifying those {\it Extended Verification Target Variables},
called $End\_Level\_Functions$, are extracted.
The prototypes of the $End\_Level\_Functions$ are used to construct an end-level environment model.
The corresponding end-level abstract kernel code consists of all the $End\_Level\_Functions$ and their dependent code.
The {\it Root\_Level\_Functions} are identified by performing backward reachability analysis from each {\it End\_Level\_Function}.
The prototypes of the $Root\_Level\_Functions$ are used to construct a root-level environment model. Its corresponding abstract kernel
code is identified by performing forward reachability analysis from each $Root\_Level\_Function$. The result of forward reachability analysis
is also used to identify constraints for the end-level environment model.

\begin{definedef}
Property-related variables:
\begin{enumerate}
\item
A Verification Target Variable is a variable that appears in the property specification.
\item
An Extended Verification Target Variable is a variable that a Verification Target Variable depends on.
\end{enumerate}
\end{definedef}

\begin{definedef}
Classification of functions:
\begin{enumerate}
\item An {\it End\_Level\_Function} is a function that directly modifies, sets, or uses an Extended Verification Target Variable.
\item A {\it Root\_Level\_Function} is an API that is a terminal node of the called-by graph of an {\it End\_Level\_Function}.
\end{enumerate}
\end{definedef}

From this process, we extract two types of functions for constructing different levels of  environment models:
(1) functions for root-level environments,  and (2) functions for end-level environments.
Figure~\ref{fig:env_gen_concept} shows the conceptual diagram for the whole process.

\begin{figure}
\centering
\includegraphics[width=156mm]{env_gen_fig.pdf}
\caption{Backward and forward reachability analysis for environment generation \label{fig:env_gen_concept}
}
\end{figure}

For a simple example, if a property in question is
\[ Property_1: \ assert(tpl\_fifo\_rw[tpl\_h\_prio].size > 0),\]
then we first identify {\it Extended Verification Target Variables} and {\it End\_Level\_Functions} for {\it tpl\_fifo\_rw} and {\it tpl\_h\_prio}.
The identified set of $End\_Level\_Functions$ for the variable $tpl\_h\_prio$ is
\{ $tpl\_get\_proc$, $tpl\_put\_preempted\_proc$,
$tpl\_put\_new\_proc$, $tpl\_schedule\_from\_running$ \} in the Trampoline kernel.
An end-level environment model is constructed as non-deterministic calls to those end-level functions and
its corresponding abstract kernel encompasses all the identified $End\_Level\_Functions$ and their dependent code.
We then identify {\it Root\_level\_Functions} for each of the {\it End\_Level\_Functions}; For examples, \{$ReleaseResource$, $Schedule$, $ActivateTask$, $SetEvent$, $TerminateTask$,
$ChainTask$, $WaitEvent$, $StartOS$\} are {\it Root\_Level\_Functions} for $tpl\_get\_proc$, which are identified from its called-by graph.
A root-level environment model consists of non-deterministic calls to those API functions and its corresponding
abstract kernel encompasses all the identified root level functions and their dependent code.

\subsection{Implementation}

\begin{figure}[h]
\centering
\includegraphics[width=156mm]{scenarioGeneration.pdf}
\caption{Environment Generation \label{fig:env_gen}
}
\end{figure}
The suggested approach was implemented and fully automated. Figure~\ref{fig:env_gen} shows the overall structure of the automation.
The source code of the Trampoline OS is analyzed by the static analysis tool Understand~\cite{Understand}, which creates a data repository of the analysis results from which information on variable/function dependencies can be extracted using a C plug-in.
The environment generator first extracts target variables from the properties and then extracts Extended Target Variables and End-Level Functions by analyzing dependency relations among variables and functions. Root-Level Functions are then extracted from the called-by graph for each End-Level Function. A Root-Level Function Scenario is generated as an arbitrary sequence of function calls of Root-Level Functions, which complies with the external constraints from the OSEK/VDX standard.  Finally, the environment model is generated as an arbitrary sequence of End-Level Functions that complies with both the external and the internal constraints. The internal constraints are partial-order relations among End-Level Functions, which are generated from each Root-Level Function.

The last step is the property-based abstraction of the original code. Since the environment generation step identifies all necessary End-Level Functions modifying the Verification Target Variables together with the ordering relation among them in call sequences, verification requires only the source code of those End-Level Functions plus codes for preprocessing them. Therefore, for each safety property, verification is performed using the environment model generated with the tool and the property-based abstract code.

\section{Setting up Environments for Collaborative Verification}
\label{sec:env_setting}

\begin{figure}[h]
\centering
\includegraphics[width=132mm]{envModel.pdf}
\caption{Scenario generation process for random testing\label{fig:envModel}
}
\end{figure}

CBMC and testing require different settings for their verification environments even with the same set of $End\_Level\_Functions$; CBMC requires only the algorithm of non-deterministic function calls.
Random testing (both root-level and end-level), however, requires explicit function call sequences generated from a given environment model.
Figure~\ref{fig:envModel} shows the process for generating such function call sequences, which can be applied to both End-Level and Root-Level environment models.
It repeats the selection and checking process by arbitrarily selecting a function and checking whether the selected function satisfied the constraints or not.
We have implemented an OSEK/VDX simulator to check the external constraints in the process of root-level sequence generation.  Internal constraints are identified by call graph analysis.
Details are described in the following two sub-sections.

We note that the checking the constraints is not necessary to ensure the correctness of the scenario generation, but it is essential for making the verification efficient, since otherwise the verification produces too many false errors.

\subsection{Root-Level Scenario Generation}

\begin{figure}[h]
\centering
\resizebox{\linewidth}{!}{
\includegraphics[width=80mm]{trampoline_fig.pdf}
\includegraphics[width=70mm]{RootLevelEnvironmentModelGeneration.pdf}
}
\caption{Root-Level scenario and the testing environment\label{fig:RootLevelScenarioGeneration}
}
\end{figure}

Root-Level scenario generation is based on the scenario generation process illustrated in Figure~\ref{fig:envModel}. A Root-Level Function Scenario is an arbitrary sequence of function calls of Root-Level Functions, and it complies with the external constraints from the OSEK/VDX standard. Figure~\ref{fig:RootLevelScenarioGeneration} shows the Root-Level test environment for the Trampoline OS. Since an OIL file is required for testing, an OIL file is specified as an input, which is compiled with the extracted kernel code and the random sequence of Root-Level Functions generated from our Root-Level scenario generator.

There are two things to be done before implementing a scenario generator. First, every constraint specified in the OSEK/VDX standard should be identified. Second, an OSEK/VDX Simulator - an abstract OSEK/VDX model - should be implemented in order to trace all the changes and fully observe constraints.

\subsubsection{Identification of external constraints}

The OSEK/VDX standard explicitly specifies constraints among the APIs. The $description$ column of Figure~\ref{fig:constraint} lists some of the constraints manually identified
from the standard. These constraints are represented as pre-conditions with respect to other APIs. For example, the API function $TerminateTask$ can be called only if the task has been activated either
by $ActivateTask$ or $ChainTask$. Therefore, we set \{$ActivateTask$, $ChainTask$\} as preconditions of $TerminateTask$. Figure~\ref{fig:constraint} shows a couple of preconditions of other API functions.

\begin{figure}[h]
\centering
\includegraphics[width=156mm]{constraint.pdf}
\caption{Constrant list extrected from OSEK/VDX spec\label{fig:constraint}
}
\end{figure}

Identified constraints are then imposed in the Root-Level scenario generation process illustrated in Figure~\ref{fig:envModel}.
Figure~\ref{fig:constraintCheck} is an example algorithm of the constraint checking.

\begin{figure}[h]
\centering
\includegraphics[width=137.5mm]{constraintCheck.pdf}
\caption{ReleaseResource constraint checker\label{fig:constraintCheck}
}
\end{figure}

\subsubsection{OSEK/VDX Simulator}
To fully consider all the identified constraints, it is necessary to trace changes that previous function calls have made. For example, if {\it ActivateTask(t1)} is chosen as the first Root-Level Function in a scenario, task {\it t1} should be marked as $READY$ task, for further scenario validation. This process is fully automated by implementing an OSEK/VDX simulator.

The OSEK/VDX simulator traces run-time information such as list of resources, list of events, list of task models, reference to running task, ready queue (priority queue), and waiting queue. It provides Root-Level Function calls just like OSEK/VDX APIs. When one of these procedures is called, it simulates the behavior of OSEK/VDX. The simulator includes task model, task scheduler, event management, and resource management. Figure~\ref{fig:simulator} shows the overall process of the OSEK/VDX simulator.

\begin{figure}[h]
\centering
\includegraphics[width=132mm]{simulator.pdf}
\caption{OSEK/VDX simulator\label{fig:simulator}
}
\end{figure}

A randomly chosen function is added to the body of the running task by a module called Scenario Generator. When there is a function in the body of the running task that has not been executed by the simulator, it executes the function. If scheduling is necessary, it calls the scheduler and executes all the functions that already exist in the body of the preempted task, when preemption occurs. This process is repeated until there is no function left to execute, and the Scenario Generator is requested to generate another function randomly.

Whether scenario generation has been completed or not can be determined by checking all tasks that are initially generated from the given OIL file. If every task has completed its execution with $TerminateTask$ or $ChainTask$, it can be determined that the scenario generation is completed.

\subsection{End-Level Environment Model}
The End-Level scenario generation process is also based on the scenario generation process illustrated in Figure~\ref{fig:envModel}.
It executes an arbitrary sequence of $End\_Level\_Functions$, which is chosen from among the list of $End\_Level\_Functions$ serving as an environment of CBMC model checking and End-Level random testing.

This process considers internal constraints among End-Level functions; since End-Level functions can only be called by API-Level functions, the pre-conditional constraints of Root-Level functions identified from the OSEK/VDX constraints must be implicitly obeyed by the End-Level functions. These implicit constraints can be identified by analyzing call-graphs of Root-Level functions and their pre-conditional
relations. We call such implicit constraints internal constraints.

\begin{figure}[h]
\centering
\includegraphics[width=100mm]{callgraph.pdf}
\caption{End-Level Functions called by each API-Level Function\label{fig:callgraph}
}
\end{figure}

For example, if we consider the Root-Level APIs and their corresponding End-Level functions, the external constraint {\it $WaitEvent$ can be called after SetEvent is called} can be re-interpreted as {\it $tpl\_get\_proc$ can be called after $tpl\_put\_new\_proc$ and $tpl\_schedule\_from\_running$ are called}, which can be written in a regular expression $(tpl\_put\_new\_proc\ tpl\_schedule\_from\_running\ (End\_Level\_Function\ -\ tpl\_get\_proc)^*\ tpl\_get\_proc)^*$.

Checking constraints of this kind cannot be done using the OSEK/VDX simulator because the End-Level functions include implementation-specific function names that cannot be modeled from the standard.
Instead, the internal constraints are simplified using the characteristic that {\it a function cannot be called more times than its preceding functions in the partial order relation}. The example internal constraint can be simplified as {\it The number of $tpl\_get\_proc$ calls cannot exceed the one of either $tpl\_put\_new\_proc$ or $tpl\_schedule\_from\_running$}. The constraint checker keeps track of the
number of each End-Level functions calls and checks $(\#\ tpl\_get\_proc < \# \ tpl\_put\_new\_proc)\ \&\&\ (\#\ tpl\_get\_proc < \#\ tpl\_schedule\_from\_running)$.

\section{Experiments}
\label{sec:experiment}

We have conducted a series of experiments to show the impact of our approach using CBMC model checking,  End-Level random testing, and Root-Level random testing.
The target verification properties are three functional safety properties from the Trampoline kernel:

\begin{eqnarray}
& &assert(tpl\_h\_prio \ne -1) \\
& &assert(tpl\_kern \ne NULL) \\
& &assert(tpl\_kern\rightarrow state == RUNNING)
\end{eqnarray}

$tpl\_h\_prio$ is the value of the highest-priority task in the ready queue in the Trampoline kernel.
$tpl\_h\_prio \ne -1$ is supposed to be true whenever rescheduling is necessary.
$tpl\_kern$ stores the key information of the currently running task.
$tpl\_kern \ne NULL$ and $tpl\_kern\rightarrow state == RUNNING$ checks if the state of the running task is $RUNNING$ when the scheduler is called.

We performed verification of these three assertions using the model checker CBMC, Root-Level random testing, and End-Level random testing.
The verification cost in terms of the number of verification conditions and the resource requirements was measured for CBMC verification.
Branch coverage was measured using the Squish Coco code coverage tool~\cite{SquishCoco} for random testing.
All experiments were performed  on Linux Fedora 16 OS, with Intel Xeon 3.4GHz e3-1270 processor and 32GB of 1333MHz DDR3 RAM.

\begin{figure}[h]
\centering
\includegraphics[width=154mm]{table_mc.pdf}
\caption{Time and memory space to verify with CBMC\label{fig:table_mc}
}
\end{figure}

Figure~\ref{fig:table_mc} shows the time and memory space it took to verify the Trampoline operating system with the End-Level Environment Model using the model checker CBMC. Time and memory space increase exponentially as
the length of the End-Level function calls (unwind value) increases. CBMC verifies the assertions by searching through every possible scenario with the length of the unwind value, making it a powerful method. With the given resources, CBMC reported no counter examples up to the unwind value 10, but was not able to finish its verification process for the unwind value 15 after 6 days.
We do not report the result of CBMC model checking using Root-Level environment models since it is too costly to perform even with the value of unwind option 10.

\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{table_apiLevel.pdf}
\caption{Coverage, time, and memory space to verify with Root-Level Random Testing\label{fig:table_apiLevel}
}
\end{figure}
\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{table_endLevel.pdf}
\caption{Coverage, time, and memory space to verify with End-Level Random Testing\label{fig:table_endLevel}
}
\end{figure}

There are a few dozens of extracted functions in random testing environments, including End-Level Functions, as illustrated in Figures~\ref{fig:table_apiLevel} and ~\ref{fig:table_endLevel}. Due to a lack of space, only the test results of End-Level Functions are described. Root-Level Random Testing (Figure~\ref{fig:table_apiLevel}) is much faster (less than 1/100 seconds), consumes little memory (up to 2.64MB of memory), and achieves a certain level of test coverage quickly, but the coverage does not improve after test sequences of length 34.  In End-Level Random Testing (Figure~\ref{fig:table_endLevel}), a test sequence of length 100 achieves a certain level of  coverage both for $tpl\_h\_prio$ and $tpl\_kern$. The coverage stays the same afterwards. End-Level Random Testing required around 1.06~1.07 MBytes of memory.

For many cases, the coverage did not increase even with lengthier test cases. There can be two reasons why some part of the code are unreachable. The first reason is exception handling; parts of the code for exception handling are never reached unless an exceptional situation occurs. The second reason is that some variables in conditional statements are not included in the Extended Verification Target Variable.
So the behavior of updating these variables might not be fully extracted, which can make some conditional statements fixed.
\begin{figure}[h]
\centering
\includegraphics[width=120mm]{code_uncovered.pdf}
\caption{Example of uncovered conditional statement\label{fig:code_uncovered}
}
\end{figure}
An example of this case is illustrated in Figure~\ref{fig:code_uncovered}. This conditional statement is only executed when {\it tpl\_schedule\_from\_dying} or {\it tpl\_activate\_task} is executed before this code. But these two functions are out of boundary in this model because the model is generated only with regard to the property {\it “tpl\_h\_prio != 0”}. Thus coverage cannot be increased, and testing terminates.

In terms of comprehensiveness, CBMC is the most powerful method, verifying every possible scenario within the same length, but it is limited by the length. As shown in Figure~\ref{fig:table_mc}, the verification cost increases exponentially as unwinding depth increases. Therefore, CBMC cannot detect potential faults that can be identified only in long task scenarios.
Unlike verification with CBMC, the length of the scenarios is not limited in random testing since the cost is much cheaper than CBMC, as shown in Figure~\ref{fig:table_endLevel}.
Though it cannot be comprehensive, it can be more effective in stress testing, since the length of the test sequences can be sufficiently long.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{overflow.pdf}
\caption{Error caused by overflow\label{fig:overflow}
}
\end{figure}

End-Level Random Testing did in fact, catch some overflow errors in the Trampoline kernel as illustrated in Figure~\ref{fig:overflow}.
These errors have occurred because the size of the variables saving the activation count is limited to 8 bits;
the second line of Figure~\ref{fig:overflow} shows that $Task2$ has the activation count 255, but adding another activation changed its value to 0.
So Trampoline has changed the value of $tpl\_h\_prio$ to -1, meaning that the process table has no activated task available.
The variable size is implementation-specific and is not constrained to 8 bits, neither in the OIL specification nor in the OSEK/VDX specification. This problem could be addressed by constraining the size to 8 bits in the OIL specification.

A Model checker could find this type of potential faults if we can set the value of the unwind option larger than 255, but our experiments could not identify it due to resource limitations.
 End-Level Random Testing is appropriate for finding this kind of errors because the cost does not increase much even with lengthy test scenarios.

We could not identify the same fault using Root-Level Random Testing, either.
The main reason is that Root-Level Random Testing is coupled with a pre-defined OIL configuration file. The OIL file specifies the typical system configuration, and thus, activating a task over 255 times
is not likely to happen unless we specifically aim at stress testing. End-Level Random Testing is more effective in stress testing, since it is not constrained by the system configuration and can test abnormal cases.

API-Level Random Testing, however, is beneficial in that it is not necessary to do additional API-Level analysis when testing identifies faults, which is necessary in model checking and End-Level Random Testing.

\section{Conclusion}
\label{sec:conclusion}

This paper presented methods and tools for environment generation and code abstraction to improve the efficiency of verification using model checking and testing.
The effect of using the suggested approach was demonstrated through a series of experiments using the Trampoline operating system as a case example.
The benefit of property-based environment generation is two-fold: (1) it reduces verification cost by reducing the target code and by limiting its environment to the task interaction
scenario relevant to the verification property, and (2) it simplifies the analysis process and localizes the verification activity by focusing on the points of interest.

The experiments revealed relative pros and cons of the three verification methods and identified potential safety faults, which suggests the following collaborative use of model checking and testing;
\begin{enumerate}
\item
Apply End-Level Random Testing first for stress testing.
\item
Apply Root-Level Random Testing to conform the errors identified through End-Level Random Testing.
\item
Apply model checking using CBMC last for comprehensive verification within a limited scope.
\end{enumerate}

Our tool still needs some improvements. First, conditional dependencies need to be considered so that test coverage can be improved.
Second, Root-Level scenario generation currently assumes a fixed OIL configuration. We would like to relax the condition so that an arbitrary OIL can be handled by the tool.

\bibliographystyle{eptcs}


Deep Pairwise Learning To Rank For Search Autocomplete

Abstract: Autocomplete (a.k.a "Query Auto-Completion", "AC") suggests full queries based on a prefix typed by customer. Autocomplete has been a core feature of commercial search engine. In this paper, we propose a novel context-aware neural network based pairwise ranker (DeepPLTR) to improve AC ranking, DeepPLTR leverages contextual and behavioral features to rank queries by minimizing a pairwise loss, based on a fully-connected neural network structure. Compared to LambdaMART ranker, DeepPLTR shows +3.90% MeanReciprocalRank (MRR) lift in offline evaluation, and yielded +0.06% (p < 0.1) Gross Merchandise Value (GMV) lift in an Amazon's online A/B experiment.

\section{Introduction}

Autocomplete is a common feature of most modern commercial search engines. It refers to the task of suggesting full queries after customer typed a prefix of a few characters \cite{cai2016survey}. Study shows that AC feature can significantly reduce the number of characters typed \cite{zhang2015adaqac}. An Autocomplete system generally consists of 2 steps: Matching and Ranking. Matching refers to generating query candidates given a typed prefix. Ranking refers to ranking the candidates. In this paper, we focus on the Ranking step and propose a NN-based online deep pairwise learning to rank (DeepPLTR) ranker. Our work makes the following  contributions:

\begin{itemize}
  \item Propose a novel NN-based ranker to rank queries by minimizing pairwise loss.
  \item Propose a novel method to incorporate the contextual signals into ranking.
\end{itemize}

The paper is organized as follows: Section 2 describes the related work in the literature; Section 3 proposes the DeepPLTR model; Section 4 presents the datasets, evaluation methods and experiments; Section 5 summarizes our work and discusses possible future research directions.

\section{Related Work}

A detailed survey of AC work could be found in \cite{cai2016survey}, which categories the Autocomplete into 2 types: heuristic approaches and learning-based approaches. Heuristic approaches use fixed algorithms to compute a ranking score for each query. Learning-based approaches treat the Autocomplete as a ranking problem, and rely on the research of learning-to-rank(LTR) field \cite{liu2009learning}, it requires a large number of training samples and features, and generally outperforms the heuristic approaches \cite{cai2016survey}. Ranking features can be categorized into 3 types: time-senstive features, personalized features and contextual features. Time-sensitive features are based on query popularity and could change over time. Personalized features, such as location, are typically limited and hard to access. Contextual features rely on the customer's previous activity and are easy to extract from the search logs. Incorporating contextual signal into model is challenging. Bar-Yossef and Kraus \cite{bar2011context} use user's previous queries in their context-aware AC. Shokouhi \cite{shokouhi2013learning} used a combination of context-based textual features and personalized features in their context-aware AC, and showed that the user’s long term search history and location are most effective signals. NN-based approach is also widely studied for AC ranking task. Session-based, personalized, and attention-based models were proposed in \cite{jiang2018neural} Fiorini and Lu \cite{fiorini2018personalized} extracted user history based features and time features as an input to an RNN model. Latest approach for AC ranking was proposed by Manojkumar et al. \cite{kannadasan2019personalized}, this approach was to train query and context embeddings on the reformulations behavior in search sessions, calculate contextual similarity features from the context and query embeddings, and then feed contextual features to a LambdaMART \cite{burges2010ranknet} ranker. This embedding approach is similar to the ``whole query embedding'' proposed in this paper when computing contextual similarity features.

\section{Deep Pairwise Learning To Rank Model}
In this section, we first describe a common labeling strategy for pairwise AC ranking task, then demonstrate the DeepPLTR architecture and ranking loss, then we propose a scalable approach to learn whole query embeddings and apply it to generate contextual features for ranking model.

\subsection{Training Sample Generation}

\begin{figure}
\centering
\includegraphics[width=\linewidth]{Training_Generation.png}
\caption{Illustration of one AC session: Customer typed keystrokes "h"=>"a"=>"n" and clicked ``hangers'', and then purchased \$100 products.}
\label{fig:Training Generation}
\end{figure}

AC session dataset is the main source to extract training samples. One AC session consists of aggregated impression logs that starts from the first keystroke in search box, all the way until customer submits the query. AC session has been attributed with the right metrics (GMV, sales, clicks etc) to each impression, as the metrics are only observed after customer submits the query, e.g. viewing or purchasing products. We propose a similar labeling strategy: we start by sampling a set of ac sessions. For each ac session, we assume that the query that was eventually submitted is the only right (or the most relevant) query, which should have been suggested right after the first keystroke and all the way until submission. With this assumption in mind, we obtained all the impressed queries from each suggestion; for each pair of <prefix, completion list> constructed this way, we assign positive label to the query submitted by user at the end (if it appears in the list) as {\em relevant keywords} and negative label to other impressed queries as {\em irrelevant keywords}. Then we grouped these positive and negative keywords from the same impression into pairs. Figure \ref{fig:Training Generation} provides an example: here we sampled one series of impression logs in which the user had submitted ``hangers'' as query. For each prefix, the only query that is assigned a positive label is specified with a checkmark (\cmark), and the query that is assigned a negative label is specified with a crossmark (\xmark). The attribution of the pairs is set to 100 as GMV attribution is \$100 for this AC session.

\begin{figure*}[b]
\centering
\includegraphics[scale=0.44]{Model_Architecture.png}
\caption{Architecture of DeepPLTR}
\label{fig:DeepPLTR Architecture}
\end{figure*}

\subsection{Model Architecture}
Given the extracted pairs, the DeepPLTR architecture is designed as in Figure \ref{fig:DeepPLTR Architecture}. It's a typical RankNet \cite{burges2010ranknet} structure based on fully-connected neural network \cite{sainath2015convolutional}. Different from RankNet \cite{burges2010ranknet}, it adds a context representation layer, which is scalable to host contextual signals. Most of the dense layers are weight-sharing as they represents the same concepts (e.g. query representation, query-context representation). The LSTM \cite{gers1999learning} layer captures the sequential signal of the time series related features, which has better performance than dense structure. The final output layer applies the sigmoid activation \cite{han1995influence} function and binary cross-entropy loss \cite{de2005tutorial} by subtracting the two ranking scores from previous layers. After fine-tuning the parameters through a hyper-parameter tuning process, query representation layer has 128 neurons, LSTM has 16 neurons, context representation layer has 128 neurons, and dropout layer ratio is 0.1. All dense layers apply the ReLU \cite{nair2010rectified} activation function. The output labels are set to 1, as the model is trained based on positive keywords ranking higher than negative keywords.

\subsection{Ranking Loss}
Ranking loss is composed of 3 parts: binary cross entropy loss, $|\Delta NDCG|$ and sample weights. Binary cross entropy loss represents the loss when reversing the ranking order of positive-negative pair. Sample weights represent the optimization target, for example, if the model optimizes GMV, we set GMV as the weight of each pair. $|\Delta NDCG|$ considers the change in NDCG when swapping the ranking positions of positive and negative keywords, so that a keyword pair more separated in the ranking list gets a larger loss. Then we define the loss as below:

$S$ denotes the full set of pairs from Section 3.1. $p$ denotes the positive keywords and $n$ denotes the negative keywords. $w_{p,n}$ denotes the sample weight of the pair. $f$ is the learned ranking function which minimizes the ranking loss. $f_p$ and $f_n$ denote the ranking score of the keywords $p$ and $n$. $|\Delta NDCG_{p,n}|$ denotes the change in NDCG when swapping the positions of keywords $p$ and $n$. $rank_p$ denotes the ranking position of the keywords $p$ ($rank_p \in [1, 2, 3, ...]$. $y$ denotes the output label. $P_{p,n}$ denotes the probability that the positive keyword $p$ is ranked above the negative keyword $n$, which serves the purpose of a contrastive loss that helps distinguish the positive keyword from the negative keyword: when the difference between $f_p$ and $f_n$ is larger, $P$ will be larger. Then we have:

\begin{equation*}
\resizebox{0.45\textwidth}{!}{$L = - \frac{1}{|S|} \sum_{(p,n) \in S} \Biggl[ (y \log P_{p,n} + (1 - y) \log (1-P_{p,n})) \cdot |\Delta NDCG_{p,n}| \cdot w_{p,n} \Biggl]$}
\label{eq:1}
\end{equation*}

where $P_{p,n}$ is defined as
\begin{equation}
    P_{p,n} = \frac{1}{1 + e^{-(f_p - f_n)}}
\end{equation}

and $|\Delta NDCG_{p,n}|$ is defined as:
\begin{equation}
    |\Delta NDCG_{p,n}| = |\frac{2^{rel_p} - 1}{\log (rank_p + 1)} - \frac{2^{rel_p} - 1}{\log (rank_n + 1)}|.
\end{equation}

Given $y$ is always 1 and $rel_p=1$, we simplify the loss from Eq. \eqref{eq:1} as:

\begin{equation}
\label{eq:2}
\resizebox{0.42\textwidth}{!}{$L = - \frac{1}{|S|} \sum_{(p,n) \in S} \Biggl[ (\log \frac{1}{1 + e^{f_n - f_p}}) \cdot |\frac{1}{\log (rank_p + 1)} - \frac{1}{\log (rank_n + 1)}| \cdot w_{p,n} \Biggl]$}
\end{equation}

\subsection{Query Representation And Contextual Features}

\begin{figure*}[b]
\centering
\includegraphics[width=\linewidth]{Query_Representation.png}
\caption{Query Embedding Generation Process}
\label{fig:Query Representation}
\end{figure*}

We construct contextual features by calculating the cosine distance between candidate queries and past search queries using pre-trained query embeddings. In this paper, we use the publicly available $fastText$ library \cite{bojanowski2017enriching} to learn the query embeddings. We use the skipGram model where the goal is to independently predict the presence or absence of the context words. The problem is framed as a binary classification task. For the word at position $t$, $fastText$ considers all context words as positive examples and sample random negatives words from dictionary as described in \cite{bojanowski2017enriching, mikolov2013distributed}. It uses the negative log likelihood, $l : x \to log( 1 + e^{-x})$ as the binary logistic loss. The objective function is defined as:

\begin{equation}
\sum_{t=1}^{T} \Biggl[ \sum_{c \in Context} l(s(w_t,w_c)) + \sum_{n \in N_{t,c}} l(-s(w_t,n)) \Biggr]
\end{equation}

where $w_t$ is the target word, $w_c$ is the context word, $C$ is context words, and $N_{t,c}$ is a set of negative examples sampled from the vocabulary. The scoring function $s(w,c)$ is the scalar product between word and context vectors: $s(w_t , w_c) = u_{w_t}^T v_{w_c} $, where vectors $u_{w_t}$ and $v_{w_c}$ corresponding to words $w_t$ and context words $w_c$ respectively.

Our goal is to learn scalable embeddings across different marketplaces and languages. For US, we collect the past 60 days' search sessions (a sequential search actions), and divide each session into multiple sequences of queries, such that no consecutive queries in a sequence are separated by > 10 minutes. We normalize queries by removing special characters and replace space character ‘\ ’ to ‘\_’. We treat each query as a “word”, and sequences of queries as a “document”, and feed them into $fastText$ \cite{bojanowski2017enriching} to train and generate embeddings with 50 dimensions. We call it the {\em whole query embedding} approach. Figure \ref{fig:Query Representation} illustrates the whole query embedding generation process. Compared to the token embedding approach, whole query embedding is a scalable approach as it generates query embeddings WW without language specific processing, e.g. tokenization for Chinese and Japanese. Then at online, we apply the pre-trained embeddings to calculate the cosine similarity between each candidate and past search query as contextual features. In particular, given recently searched query $Q$ and candidate $S$, query embedding $vec()$. $cosine(S, Q) = \frac{vec(S) \cdot vec(Q)}{|vec(S)| \cdot |vec(Q)|}$.

\section{Experiments}
\subsection{Dataset and Experiment Settings}
To conduct experiments, we follow the labeling strategy described in Section 3.1 to extract 77M training pairs, and 11M validation pairs from AC sessions which lead to purchase events (GMV $> 0$) during a week of US data in 2020. This gives us a good coverage of user context features to learn deep model. In order to benchmark on MRR and NDCG, we collect 150k impression logs with ground truth as the test datasets in Feb. 8, 2020. Different models are trained and evaluated with the same sampled data.

\subsection{Benchmark Rankers}
To evaluate DeepPLTR, we constructed 4 benchmark rankers:

\noindent \textbf{MPC Ranker:} the MostPopularCompletion ranker \cite{bar2011context}, which is a common benchmark for AC task. It ranks queries by the decayed search popularity of the query.

\noindent \textbf{MPGC Ranker:} the MostPopularGMVCompletion ranker, which ranks by the decayed GMV of query. It's designed to benchmark with GMV-optimized model.

\noindent \textbf{LambdaMART Ranker (context-aware):} train a LambdaMART ranker using XGBoost library \cite{chen2016xgboost} with the same features as DeepPLTR. From which, we will learn whether NN architecture could improve performance. We set pairwise loss as the optimization target, and fine-tuned number of trees as 150 and max depth as 6 to XGBoost to train the model. We treat this as a baseline ranker.

\noindent \textbf{DeepPLTR-NDCG Ranker (context-aware):} train a DeepPLTR excluding $|\Delta NDCG|$ loss in the Eq. \ref{eq:2} to learn the position impact.

\subsection{Evaluation Metrics}
\noindent \textbf{MeanReciprocalRank:} the average of the reciprocal rank of the target queries in the AC results weighed on the business metrics. Given a test dataset $S$, the MRR for ranker $A$ is computed as:
\begin{equation}
    MRR(A) = \frac{1}{|S|} \sum_{C_t,q_t \in S} \frac{w}{hitRank(A,C_t,q_t)},
\end{equation}

where $C_t$ represents the user context at time $t$, $q_t$ represents the target query, and $w$ represents the sample weight. The function $hitRank$ computes the rank of the relevant query based on the order created by algorithm $A$. The relevant query refers to the clicked query in AC.

\noindent \textbf{NormalizedDiscountedCumulativeGain:} Discounted Cumulative Gain ($DCG$) represents the usefulness or gain of the query based on its position in the ranked list. Given AC task having one relevant query, $IDCG$ will always be 1, so NDCG@p can be simplified as below:

\begin{equation}
    NDCG_p = \frac{\sum_{s \in S} \Biggl[ w_s \cdot (\sum_{i=1}^{p} {\frac{2^{rel_i} - 1}{\log (i + 1)})} \Biggl]}{\sum_{s \in S} w_s},
\end{equation}

where $i$ denotes the rank, $rel_i \in \{0,1\}$ is the relevance of query at rank $i$, and $w_s$ is the sample weight.

\subsection{Results}

To learn the impact of contextual signals on the model's performance, we construct three versions of test samples: All Samples (AS), Samples With Past Searches (SWPS) and Samples Without Past Searches (SWOPS). Evaluation metrics are set to MRR, NDCG@1, NDCG@3.

\begin{table}
  \caption{Evaluate on All Samples}
  \begin{tabular}{l | c | c | c }
     \toprule
     Model & MRR & NDCG@1 & NDCG@3 \\
     \midrule
     MPC & 0.469(-16.84\%) & 0.293(-22.49\%) & 0.464(-18.74\%) \\
     MPGC & 0.488(-13.48\%) & 0.308(-18.52\%) & 0.485(-15.06\%) \\
     LambdaMART & 0.564 & 0.378 & 0.571 \\
     DeepPLTR-NDCG & 0.577(+2.30\%) & 0.400(+5.82\%) & 0.584(+2.28\%) \\
     DeepPLTR & 0.586(\textbf{+3.90\%}) & 0.409(\textbf{+8.20\%}) & 0.594(\textbf{+4.03\%}) \\
     \bottomrule
  \end{tabular}
  \label{tab:Evaluate on All Samples}
\end{table}

\begin{table}
  \caption{Evaluate on Samples WithOut Past Searches}
  \begin{tabular}{l | c | c | c }
     \toprule
     Model & MRR & NDCG@1 & NDCG@3 \\
     \midrule
     MPC & 0.481(-16.20\%) & 0.297(-23.06\%) & 0.474(-18.56\%) \\
     MPGC & 0.505(-12.02\%) & 0.315(-18.39\%) & 0.502(-13.75\%) \\
     LambdaMART & \textbf{0.574} & 0.386 & \textbf{0.582} \\
     DeepPLTR-NDCG & 0.570(-0.70\%) & 0.385(-0.26\%) & 0.580(-0.34\%) \\
     DeepPLTR & 0.572(-0.35\%) & 0.388(\textbf{+0.52\%}) & 0.581(-0.17\%) \\
     \bottomrule
  \end{tabular}
  \label{tab:Evaluate on Samples With No Past Searches}
\end{table}

\begin{table}
  \caption{Evaluate on Samples With Past Searches}
  \begin{tabular}{l | c | c | c }
     \toprule
     Model & MRR & NDCG@1 & NDCG@3 \\
     \midrule
     MPC & 0.459(-17.45\%) & 0.289(-22.10\%) & 0.455(-19.04\%) \\
     MPGC & 0.474(-14.75\%) & 0.303(-18.33\%) & 0.470(-16.37\%) \\
     LambdaMART & 0.556 & 0.371 & 0.562 \\
     DeepPLTR-NDCG & 0.582(+4.68\%) & 0.413(+11.32\%) & 0.587(+4.45\%) \\
     DeepPLTR & 0.598(\textbf{+7.55\%}) & 0.427(\textbf{+15.09\%}) & 0.606(\textbf{+7.83\%}) \\
     \bottomrule
  \end{tabular}
  \label{tab:Evaluate on Samples With Past Searches}
\end{table}

Evaluation on AS are shown in Table \ref{tab:Evaluate on All Samples}. LambdaMART ranker outperforms the naive rankers (MPC, MPGC), which is expected with more advanced signals. DeepPLTR outperforms the LambdaMART by 3.90\% in MRR, by 8.20\% in NDCG@1, and by 5.99\% in NDCG@3, showing that NN-based architecture imrpoves the performance. DeepPLTR outperforms DeepPLTR-NDCG, shows that considering position factor in ranking boosts the ranking performance.

Evaluation on SWOPS are shown in Table \ref{tab:Evaluate on Samples With No Past Searches}. LambdaMART ranker slightly outperforms DeepPLTR on $NDCG@3$, which shows that DeepPLTR has similar performance to LambdaMART ranker without past search history. LambdaMART ranker slightly outperforms PLTR, showing that device type feature improves performance. Evaluation on SWPS are shown in Table \ref{tab:Evaluate on Samples With Past Searches}. DeepPLTR outperforms baseline by 7.55\% in MRR and 15.09\% in NDCG@1, showing that DeepPLTR can learn implicit ranking features based on past searches features. DeepPLTR boosts performance the most on SWPS samples.

We also conduct an online A/B experiment to compare the DeepPLTR and LambdaMART ranker, experiment shows DeepPLTR yields a +0.06\% (p < 0.1) GMV lift compared to LambdaMART ranker, which demonstrates the DeepPLTR outperforms LambdaMART.

\section{Conclusions and Future Work}
In this paper, we have proposed a neural-network based pairwise ranker and ranking loss, and described a scalable approach to learn query embeddings using search session data. The architecture of DeepPLTR is scalable for accommodating a variety of ranking signals and can be applied to other search tasks such as product search. We conduct offline and online experiments to verify that DeepPLTR outperforms LambdaMART ranker. In the future, we will improve AC in several directions: 1) Incooporate richer contextual information, e.g. past product clicks \& purchases; 2) Add exploration mechanism to explore unshown suggestions.

\begin{acks}
We would like to thank the team members in Amazon Search: Trishul Chilimbi, Adam Kiezun, Wenyang Liu, Hsiang-Fu Yu, Erte Pan, Yun Hyokun, Zia Hasan, Pratik Lahiri, RJ, Yukun Lin, Haining Yu, Ben Wing, Hye Jin Jang for their support.
\end{acks}

\bibliographystyle{ACM-Reference-Format}

Information Retrieval (cs.IR); Machine Learning (cs.LG)