Virtual Adversarial Humans finding Hazards in Robot Workplaces

Abstract: During the planning phase of industrial robot workplaces, hazard analyses are required so that potential hazards for human workers can be identified and appropriate safety measures can be implemented. Existing hazard analysis methods use human reasoning, checklists and/or abstract system models, which limit the level of detail. We propose a new approach that frames hazard analysis as a search problem in a dynamic simulation environment. Our goal is to identify workplace hazards by searching for simulation sequences that result in hazardous situations. We solve this search problem by placing virtual humans into workplace simulation models. These virtual humans act in an adversarial manner: They learn to provoke unsafe situations, and thereby uncover workplace hazards. Although this approach cannot replace a thorough hazard analysis, it can help uncover hazards that otherwise may have been overlooked, especially in early development stages. Thus, it helps to prevent costly re-designs at later development stages. For validation, we performed hazard analyses in six different example scenarios that reflect typical industrial robot workplaces.

\section{INTRODUCTION}
	\label{sec:Introduction}
	In recent years, there has been a trend in industrial automation towards human-robot collaboration \cite{Bauer2016}. Traditionally, safety issues with industrial robots were resolved by physically separating robot and human worker. In collaborative robotics this is often not possible. Instead, various combinations of different safety measures including detection sensors (e.g. laser scanners, light curtains), (partial) fencing, and software-based safety functions (e.g. velocity limitation, collision detection) are used, depending on the application and its required degree of collaboration. Flaws in the configuration of these safety measures can lead to hazards for human workers. To foresee potential hazards before commissioning, a hazard analysis is required \cite{ISO10218-2}. Currently, hazard analyses are based on experience, expert knowledge \cite{ISO14121}, and simple tools such as checklists (e.g. in \cite[Annex A]{ISO10218-2} and \cite[Annex B]{ISO12100}).\newline With the trend towards collaborative robotics, hazard analyses become more challenging due to greater system complexity. Thus, development of hazard analysis tools is an active field of research.
	\begin{figure}
		\includegraphics[width=1\columnwidth]{images/ExampleSituation_annotated_cropped.png}
		\caption{\small The virtual human autonomously explores the workplace to uncover hazards. By receiving rewards for high-risk behavior, it learns to actively provoke unsafe states. In this situation, for example, it reaches through a feed opening to provoke a collision.}
		\label{fig:ExampleSituation}
	\end{figure}
	Tools proposed in recent years typically use semi-formal or formal system models, e.g. control structure diagrams \cite{Leveson2016}, Product-Process-Resource models \cite{Awad2017}, UML models \cite{Guiochet2016}, or system descriptions in a formal language \cite{Askarpour2016,Vicentini2019}. These models have in common that they have a relatively \textit{high level of abstraction}, that is, system properties and dynamics are simplified to make complexity manageable. This has two major drawbacks:
	\begin{itemize}
		\item The level of detail is limited. Especially geometric aspects (e.g. cell layout, detection zones, safety fences) and dynamic aspects (e.g. collision forces, robot stopping time) require precise modeling, and thus, are difficult to consider in abstract system models.
		\item Common simulator models (CoppeliaSim, Process Simulate, etc.), which are often already available in the development process, are not used. Instead, new purpose-specific models are obtained which increases the cost and time needed for hazard analysis.
	\end{itemize}
	Considering these drawbacks, there is a need for a hazard analysis tool that operates directly in a dynamic simulation model (i.e. on a \textit{low abstraction level}) without the need to first derive another model that is abstracted or simplified.\newline
	A major challenge of such a simulation-based hazard analysis is to create conditions where hazards are uncovered: In complex simulation models the number of possible simulation sequences can be vast and an exhaustive search can be infeasible. If certain hazards only manifest themselves in very specific simulation sequences, these hazards can remain undiscovered. We address this problem by introducing a virtual human into the simulation that acts in a manner adversarial to the safety measures: It learns to deliberately provoke unsafe situations, which increases the chance of uncovering hazards. We initially proposed this idea in \cite{Huck2020}. In this paper, we further develop the idea and present a concept for a simulation-based hazard analysis tool. In particular, we extend our work from \cite{Huck2020} with a body-region-specific collision-force estimation to evaluate hazards with respect to the potential collision severity. Furthermore, we use an articulated arm model to determine reachability of hazardous areas. This enables the finding of hazards that result from reaching behind or around safety barriers (compare Fig. \ref{fig:ExampleSituation}). Our conceptual tool was evaluated by testing it on six different industrial robot workplaces with different safety configurations and hazards.
	\section{RELATED WORK AND STANDARDS}
	\label{sec:RelatedWork}
	\subsection{Tools and Methods for Hazard Analysis}
	\label{sec:MethodsAndTools}
	In industrial practice, hazard analysis is usually conducted according to the \mbox{ISO 12100} standard \cite{ISO12100}. However, the procedures recommended in \mbox{ISO 12100} are quite generic and do not address the specific challenges arising for robot systems. Thus, several novel tools and methods have been proposed specifically for the analysis of robot systems.\newline
	\textbf{Semi-formal:} Semi-formal analysis methods like “Hazard and Operability Analysis” (HAZOP) \cite{IEC61882} or “Systems-Theoretic Process Analysis” (STPA) \cite{Leveson2016} have been adapted to analyze robot systems. These methods are based on semi-formal system descriptions, e.g. flow diagrams (HAZOP) or control structure diagrams (STPA). Guiochet proposed a HAZOP-Extension that uses UML diagrams for the analysis of robot systems \cite{Guiochet2016}. Marvel et al. have proposed a method that is based on an ontology of basic human-robot collaborative tasks \cite{Marvel2014}
	\newline
	\textbf{Formal and Rule-based: } Awad et al. have developed an expert system for automated risk assessment of robot workplaces \cite{Awad2017} using a model of processes, products and resources (PPR-model). A rule base maps the PPR-model to hazards. This approach was further developed by Wigand et al \cite{Wigand2020}. The method "SAFER-HRC", developed by Askarpour et al. \cite{Askarpour2016,Askarpour2017}, and Vicentini et al. \cite{Vicentini2019}, uses formal verification for hazard analysis. Based on a system description in the formal language TRIO, a bounded satisfiability checker is used to verify safety properties of collaborative robot systems.\newlin
	\textbf{Simulation-based:} In this paper, we propose a simulation-based approach. The use of simulation for analysis of safety-critical systems is well-researched \cite{Corso2020}. However, in the domain of robot safety, simulation-based approaches are usually employed on a component- rather than a system-level, for example to verify control code or control strategies \cite{Araiza2015,Araiza2016,Uriagereka2019, Bobka2016}. Askarpour et al. recently presented a co-simulation approach which couples SAFER-HRC with the simulator MORSE \cite{Askarpour2020}. However, MORSE is used primarily for visualization, and the underlying hazard identification is still conducted with a formal model.
	\subsection{Safety Standards in Industrial Robotics}
	\label{sec:Standards}
	For a better understanding of the following contents, we will shortly introduce relevant safety standards besides the already mentioned \mbox{ISO 12100}. Robot safety is governed by the standard \mbox{ISO 10218} \cite{ISO10218-1,ISO10218-2} and the technical specification \mbox{ISO/TS 15066}\cite{ISOTS15066}. Here, especially \mbox{ISO/TS 15066} is important. It defines body-region specific force and pressure limits for human-robot collisions. Safety measures must either ensure that collisions are avoided, or that collision force and pressure remain below these limits. \mbox{ISO/TS 15066} Annex A provides a calculation model with body-region specific parameters to estimate collision forces. Other relevant standards are \mbox{ISO 13855}\cite{ISO13855} and \mbox{ISO 13857}\cite{ISO13857}. \mbox{ISO 13855} regulates the use of contactless safeguards (e.g. laser scanners, light curtains) which are used to detect human workers. \mbox{ISO 13857} regulates the placement and dimensions of safety fences and ensures that hazardous areas cannot be accessed by reaching over or around the fences.

	\section{{OUR APPROACH}}\label{sec:Approach}
	In contrast to the (semi-)formal and rule-based methods from Section \ref{sec:MethodsAndTools}, our goal is a purely simulation-based approach to hazard analysis. We focus on finding hazards that result from systematical faults in the design, configuration, or programming of the robot system. We do not consider hazards that result from random faults (e.g. component failures), because these types of hazards are typically addressed on a component- rather than a system level, and there are already established safety engineering methods to regard them (e.g. Failure Mode, Effects and Diagnostics Analysis, FMEDA \cite{Goble1999}, or Fault Tree Analysis, FTA \cite{IEC61025}).	Under these premises, we assume that for a given human action, the reaction of the robot system is deterministic. If the system has inherent hazards, then there will be sequences of human actions that can provoke hazardous situations. Thus, the problem of hazard analysis can be framed as a search problem, where the goal is to find sequences of human actions that result in a hazardous situation. Formally, this approach can be described as follows:
	\begin{equation}
	\langle S,U,A,\phi, s_0 \rangle
	\end{equation}
	where $S$ is a set of simulator states $s$ describing the configuration of human, robot, and workplace environment and $A$ is a set of actions which can be performed by the virtual human in simulation. $U$ is a subset of $S$ representing unsafe states (i.e. hazardous situations in simulation). We define an unsafe state as a state where there is contact between human and robot while the robot is moving with a velocity that could potentially lead to a collision force exceeding the \mbox{ISO/TS 15066} limits:
	\begin{equation}
	U=\left\{s\ |\ d_{HR} = 0,\ F_{\mathrm{coll}}>F_{\mathrm{max}}\right\}
	\end{equation}
	Here, $d_{HR}$ is the human-robot distance, $F_{\mathrm{coll}}$ the potential collision force estimated according to \mbox{ISO/TS 15066} equations (A.1)-(A.4)\footnote{In this paper, we estimate $F_{coll}$ based on the \textit{absolute} velocity of the robot at contact point; the \textit{direction} is not considered, because our simplified human model does not allow for reliable calculation of the collision vector. Thus, our definition (2) is conservative: It also applies to contacts where the robot velocity points away from the human (compare Video, Scenario 6). This limitation will be addressed in the future with more detailed modeling.}, and $F_{\mathrm{max}}$ denotes the body-region-specific limit (see \mbox{ISO/TS 15066}, Annex A).

	The function $\phi$ is a transition function that returns the next state given the current state and a human action: $s_{k+1}=\phi(s_k,a)$. This function is implemented by the simulator: The next state is obtained by simulating the behavior of human and robot system for a given human action.\newline
	The search problem now consists of finding sequences of human actions $a_1, a_2, ... a_n$
	that, when performed in interaction with robot and workplace environment, result in an unsafe state $s \in U$.
	Depending on the number $|A|$ of possible actions and the length $n$ of the sequences, the search space, which consists of $|A|^n$ possible simulations sequences, can be vast. To address this challenge, we combine the following methods:
	\begin{itemize}
		\item \textbf{Monte Carlo Tree Search (MCTS):} We use MCTS to search for human action sequences that result in critical collisions. By rewarding the MCTS algorithm for the occurrence of dangerous situations, the virtual human learns to deliberately provoke unsafe states. This increases the chance of uncovering hazards and thus mitigates the problem of large search spaces \cite{Lee2018,Huck2020}.
		\item \textbf{Reachability Analysis:} To reduce the human action space, we combine MCTS with a reachability analysis. In particular, we use reachability analysis of the human arm to determine if the robot is reachable by the human. We thus avoid modeling the complex arm movements of humans in the form of explicit actions. This prevents an explosion of the search space.
	\end{itemize}

	\begin{figure}[t]
		\centering
		\includegraphics[width=1\columnwidth]{images/System_Overview_4.png}
		\caption{ \small The MCTS algorithm selects a sequence of human actions which the human model then executes in the simulation. Based on the level of danger in the current simulation state, the algorithm receives a reward that is designed to encourage dangerous behavior and thus increases the chance of finding hazards.}
		\label{fig:SystemOverview}
	\end{figure}

	\section{IMPLEMENTATION}
	\label{sec:Implementation}
	\begin{figure}
		\centering
		\includegraphics[width=0.7\columnwidth]{images/ArmModel.png}
		\caption{\small 4-DoF arm model with $\theta_F$ for flexion and $\theta_A$ for abduction of the elbow, $\theta_R$ for upper arm rotation, and $\theta_E$ for elbow flexion. Lower arm rotation and wrist flexion are omitted. Instead, a bounding sphere is attached to the wrist, over-approximating the possible hand poses. Upper arm length $l_U$, lower arm length $l_L$, and radius of the bounding sphere $r_H$ are parameterized according to \mbox{ISO 13857}.}
		\label{fig:ArmModel}
	\end{figure}
	Our implementation is based on the iterative search procedure shown in Fig. \ref{fig:SystemOverview}: The MCTS algorithm selects a human action which is then executed by the human model in interaction with the simulated workplace environment.
	After each search step the level of danger in the current simulation state is evaluated. Based on the danger level the MCTS algorithm receives a reward which it then uses to adapt the search behavior. When an unsafe state is found, or a maximum number of search steps is reached, the simulation is set back to the initial state and the next search iteration begins. The remainder of this section describes the human model, the MCTS algorithm, and the reward in more detail.\newline
	\textbf{Human Model. } We use "Bill", a simple human model included in the simulator CoppeliaSim (former V-REP \cite{Rohmer2013}). The model is adapted so that it can perform six walking- and six upper-body-motions, amounting to an action space of 36 combined motions:
	\begin{dmath}\breakingcomma
		A_{\mathrm{Walking}}=\left\{(\mathrm{stop}), (\mathrm{forward}),(\mathrm{left\ 45\degree}),(\mathrm{left\ 90\degree}),(\mathrm{right\ 45\degree}),(\mathrm{right\ 90\degree}) \right\}
		\label{eq:ActionSpaceStart}
	\end{dmath}
	\begin{dmath}\breakingcomma
		A_{\mathrm{UpperBody}}=\left\{ (\mathrm{upright}),(\mathrm{bend\ forward)},(\mathrm{bend\ left)},\mathrm{(bend\ right)},\mathrm{(bend\ forward\ and\ right)},\mathrm{(bend\ forward\ and\ left)} \right\}
	\end{dmath}
	\begin{gather}
	\label{eq:HumanActionEnd}
	A=A_{\mathrm{Walking}} \times A_{\mathrm{UpperBody}}\\
	|{A}|=6\cdot 6= 36
	\label{eq:ActionSpaceEnd}
	\end{gather}
	The walking speed is 1.6m/s, which is assumed as human approach speed in \mbox{ISO 13855} (compare Section \ref{sec:Standards}).\newline
	To prevent an explosion of the action space, arm motions are not modeled in the form of explicit actions. Instead, the virtual human performs a reachability check in each timestep to find potentially dangerous arm configurations.
	For the reachability check, we use an articulated model that allows the virtual human to also find collisions that require reaching around or behind protective barriers (see Fig. \ref{fig:ExampleSituation}). The arm model, seen in Fig. \ref{fig:ArmModel}, is adapted from \cite{Klopcar2005} to replicate the arm model on which \mbox{ISO 13857} is based \mbox{\cite[Table 3]{ISO13857}}. It features joints for shoulder abduction and flexion, upper arm rotation, and elbow flexion. Wrist flexion and lower arm rotation are omitted. To make up for the omission, a sphere is placed around the wrist for an over-approximate collision detection. In each timestep the virtual human checks if the robot is close enough to be potentially reachable. If this is the case, the reachability check is performed by sampling through the joint angle limitations given by \cite{Klopcar2005} and checking for collisions with robot and obstacles.

	\textbf{Algorithm. }For the MCTS algorithm we use a Monte Carlo Tree Search with Double Progressive Widening (MCTS-DPW) from Lee et al. \cite{Lee2018}. For reasons of brevity, we only describe the algorithm in a simplified manner. For a detailed explanation, we refer to \cite{Lee2018}.
	\begin{figure}[t]
		\centering
		\includegraphics[width=1\columnwidth]{images/SearchTree.png}
		\caption{ \small Search tree: Each search iteration starts in the initial simulation state $s_0$. At each node the algorithm decides whether to add a new node to the search tree by randomly sampling a new action, or re-visit an existing node by selecting an action based on the state-action values estimate. When the algorithm reaches a new node which has not been visited before, an action sequence is sampled randomly until it terminates by reaching an unsafe state $s \in U$ or by reaching the maximum number of search steps $k_{max}$}
		\label{fig:SearchTree}
	\end{figure}

	The algorithm iteratively samples action sequences and executes them in the simulator. While doing so, it builds a search tree (see Fig. \ref{fig:SearchTree}) where the nodes correspond to states and the edges to human actions\footnote{Technically, the implementation of Lee et al. selects actions indirectly via adjusting the random seed of the simulator. In our case, however, each seed corresponds to one specific action. Thus, for a clearer notation, we denote the action directly.}.
	While stepping through the search tree, a state-action value function $Q(s,a)$ is estimated to guide the search.
	A search iteration is started by initializing the simulator with the initial state $s_0$. The algorithm then steps through the search tree by selecting human actions and executing them in the simulation. In each search step, the algorithm decides whether to add a new node by sampling a new action from a uniform distribution over $A$, or whether to re-visit an existing node. A new node is added if the following criterion holds:
	\begin{equation}
	|X(s)|<k\cdot N(s)^\alpha
	\label{eq:progressiveWideningCriterion}
	\end{equation}
	Here, $|X(s)|$ is the number of different actions that have already been executed in this state (i.e. the number of edges starting at the current node), $N(s)$ is the number of times the node has already been visited, and $k,\alpha$ are parameters to balance between explorative and exploitative search behavior.s
	If (\ref{eq:progressiveWideningCriterion}) does not hold, an existing node is revisited by choosing an already explored action based on the estimated $Q(s,a)$:
	\begin{equation}
	a=\mathrm{argmax}\left\{Q(s,a)+c\cdot \sqrt{\frac{N(s)}{N(s,a)}}\right\}
	\label{eq:seedSelection}
	\end{equation}
	Here, $N(s,a)$ is the number of times the action $a$ was executed in state $s$, and $c$ is another parameter to balance exploration and exploitation.\newline
	Once the algorithm has reached a new node that has not been visited before, actions are sampled randomly from a uniform distribution until the simulation terminates. The simulation terminates when an unsafe state $s \in U$ occurs, or when the maximum number of search steps $k_{max}$ is reached. After the simulation terminates, it is set back to initial state $s_0$ and the next iteration of the search is started.\newline
	The algorithm receives a reward $R$ after each action. When reaching a terminal state, the state-action value function is updated via backpropagation:
	\begin{equation}
	Q(s,a)\leftarrow Q(s,a)+\frac{q-Q(s,a)}{N(s,a)}
	\end{equation}
	where $q$ is the undiscounted sum of rewards that have been incurred in previous nodes of the backpropagation sequence.\newline
	\textbf{Reward. }The reward $R$ is defined as follows:
	\begin{equation}
	R = \begin{cases}
	R_E & \text{if}\ s \in U\\
	-\frac{1}{c_D} & \text{if}\ s \notin U\ \mathrm{and}\ k=k_{\mathrm{max}}\\
	0 & \text{otherwise}
	\end{cases}
	\label{eq:Reward}
	\end{equation}
	Where the first component $R_E$ is a non-negative constant to reward the occurence of an unsafe state and the second component is a negative penalty that is given if the maximum number of actions $k_{max}$ is reached without finding an unsafe state. The penalty is based on $c_D$, which characterizes the level of danger in a given simulation state. Since we want to penalize \textit{safe} states (i.e. states with a small $c_D$) higher, we give the \textit{inverse} of $c_D$ as penalty.\newline
	The danger level $c_D$ is calculated by a heuristic function that differentiates between three cases:
	\begin{equation}
	c_D = \begin{cases} \frac{F_{\mathrm{coll}}}{F_{\mathrm{max}}} & \text{in case (a)} \\
	\min \left\{\frac{F_{\mathrm{coll,virt}}}{F_{\mathrm{max}}}*e^{-d_{\mathrm{HR}}},1\right\} & \text{in case (b)}\\
	0.01 & \text{in case (c)}
	\end{cases}
	\label{eq:dangerIndex}
	\end{equation}
	\begin{itemize}
		\item[(a)] Human-robot contact ($d_{HR}=0$).
		\item[(b)] Human and robot are close, but no contact ($d_{HR}\leq$1.5m).
		\item[(c)] Human-robot distance is larger than 1.5m or human and robot are fully separated by a barrier.
	\end{itemize}
	\indent In \textit{case (a)}, the danger level is defined as the ratio between the potential collision force and the maximum acceptable collision force for the affected body part.\newline
	\indent In \textit{case (b)}, where there is no direct human-robot contact, the danger level is calculated on the basis of a virtual collision force $F_{coll,virt}$, which is the potential collision force that would occur if the closest human-robot object pair would collide. To take distance into account, $c_D$ is scaled down with the human-robot distance $d_{\mathrm{HR}}$. Note that $c_D$ is limited to 1, so that a situation from case (b) always returns a lower value than an actual contact situation exceeding the limit.\newline
	\indent In \textit{case (c)}, we assume that the level of danger is negligible, and therefore define $c_D=0.01$ ($c_D > 0$ to avoid division by zero in (\ref{eq:Reward})).\newline
	\textbf{Postprocessing: }During the search, unsafe states are documented in a log file that contains the potential collision force, the affected body region, and the sequence of actions that has led to the unsafe state. Based on the log information, the user can re-play the simulation sequence and decide if the unsafe state constitutes a realistic hazard that requires further safety measures. It is important to note that this final decision still needs to be made by a human: Our method is primarily intended to raise awareness of \textit{possible} hazards, but due to modeling simplifications or an overly conservative definition of unsafe states, not all hazards that are possible in simulation are necessarily realistic in the context of a real-world application.

	\section{APPLICATION EXAMPLES}
	\label{sec:ApplicationExample}
	We demonstrate our approach in six different test scenarios that reflect typical industrial robot workplaces\footnote{To avoid the scenarios becoming too convenient due to bias (i.e. hazards being too easy to find), they were designed and implemented by persons who were not involved in the development of the proposed method.}. Each scenario has a deliberate safety flaw that results in a specific hazard. We deployed our virtual human as described in Section \ref{sec:Implementation} to find these hazards. As a baseline we also deployed a variant that uses a random search instead of the MCTS algorithm. In this variant, the human action sequences are sampled randomly from a uniform distribution over $A$ (the other functions, including the reachability check, remain unchanged). For each combination of scenario and search algorithm, we conducted ten test runs with different random seeds. Each search iteration consists of a sequence of 8 actions, each with a duration of 200ms. Length of the test runs was limited to 320 iterations.\newline
	The test scenarios are explained in Table I and in the video accompanying this paper. For each test scenario and each algorithm, we calculated the average runtime $T$ (i.e the average number of search iterations needed to find the hazard), and the miss rate $N_{miss}$ (i.e. in how many of the ten test runs the hazard was not found). These results are also shown in Table I. Both MCTS and random search successfully identified the hazards in the majority of test runs. However, in case of the random search, there were seven instances of hazards being missed, whereas MCTS only missed a hazard in one instance. Furthermore, compared to the random search, the average runtime of MCTS was significantly smaller in all scenarios. These results indicate that adapting the human behavior to provoke more dangerous situations does indeed increase the chance of finding hazards.\newline
	We also conducted a closer investigation of the one instance where MCTS missed a hazard. We found that this was due to premature convergence to a local optimum, that is, a collision where the reward is relatively high, but the situation does not constitute an unsafe state in the sense of our definition (2). Convergence to such a local optimum can prevent the algorithm from discovering a more critical situation (compare Fig. \ref{fig:localMinimum}).

	\begin{figure}
		\includegraphics[width=\columnwidth]{images/scenarios/LocalMinimum_annotated.png}
		\caption{\small Example from Scenario 2: In both situations, the human provokes a collision by avoiding the safety mat (yellow) and reaching around the fence. However, only on the left side, the potential collision force exceeds the \mbox{ISO/TS 15066} limit. The situation on the right side is a local optimum w.r.t the reward function, i.e. collisions are possible, but do not represent an unsafe state according to our definition (2).}
		\label{fig:localMinimum}
	\end{figure}

	\begin{table*}[!h]
		\begin{tabularx}{\textwidth}{|p{0.7\textwidth}|p{0.25\textwidth}|}
			\hline
			\textbf{Test scenario and hazard}	& \textbf{Results$^*$}\\
			\hline
			\textbf{1 Safety fence} (Fig. \ref{fig:ExampleSituation}):
			The robot operates behind a safety fence and plexiglass barrier. The plexiglass barrier has a feed opening which provides access to a conveyor belt.
			The safety distance between the feed opening and the robot path is insufficient. An unsafe state is possible when the worker reaches behind the feed opening and collides with the robot.
			& \textbf{MCTS:}\newline $T = 16.3,\ \sigma=12.8,\ N_{miss}=0$\newline
			\textbf{Random Search:}\newline $T = 37.4,\ \sigma=52.8,\ N_{miss}=0$\\
			\hline
			\textbf{2 Sensor mat } (Fig. \ref{fig:localMinimum}):
			The robot works in a cell which is encased by fences on three sides, but has an open front. A sensor mat in front of the cell stops the robot when the worker steps on it. However, the mat is not wide enough. It only covers the front of the cell, not the right and left side. Thus, an unsafe state can occur when the worker walks around the mat and reaches behind the fence into the cell.
			& \textbf{MCTS:}\newline $T = 131.6,\ \sigma=76.6,\ N_{miss}=1$\newline
			\textbf{Random Search:}\newline $T = 203.4,\ \sigma=133.4,\ N_{miss}=4$ \\
			\hline
			\textbf{3 Head collision } (Fig. \ref{fig:Hazards}-a)
			The robot performs a pick-and-place task on a table. A laser scanner detects the approach of workers and reduces the robot velocity. With the reduction, collisions with arms and upper body are uncritical. However, due to the height of the table the robot can also collide with the head which is still critical despite the velocity limitation.
			&  \textbf{MCTS:}\newline $T = 83.1,\ \sigma=45.8,\ N_{miss}=0$\newline
			\textbf{Random Search:}\newline $T = 115.8,\ \sigma=113.0,\ N_{miss}=1$ \\
			\hline
			\textbf{4 Light curtain } (Fig. \ref{fig:Hazards}-b) The worker can enter the robot cell through a light curtain. Upon entrance the robot stops with a certain stopping time. If the worker enters while the robot passes a certain point on its path, a part of robot protrudes far enough out so that the worker can reach it before the robot has slowed down sufficiently, resulting in an unsafe state.
			& \textbf{MCTS:}\newline $T = 14.9,\ \sigma=13.4,\ N_{miss}=0$\newline
			\textbf{Random Search:}\newline $T = 28.5,\ \sigma=23.4,\ N_{miss}=0$  \\
			\hline
			\textbf{5 Laserscanners } (Fig. \ref{fig:Hazards}-c)
			Two robots work in parallel. Each robot is safeguarded by a separate laser scanner zone (red, yellow). When a worker enters one of the zones, the respective robot stops while the other keeps running. However, an unsafe state is possible when the worker enters one of the laser scanner zones, but then reaches towards the robot on the opposite side, whose laser scanner field is not activated.
			& \textbf{MCTS:}\newline $T = 69.4,\ \sigma=66.8,\ N_{miss}=0$\newline
			\textbf{Random Search:}\newline $T = 174.3,\ \sigma=137.7, \ N_{miss}=2$  \\
			\hline
			\textbf{6 Heavy workpiece } (Fig. \ref{fig:Hazards}-d)
			The robot reaches outside its workcell to grasp a heavy workpiece which it then takes inside. Inside the cell, the robot works with full velocity. Outside, the velocity is limited to keep collision forces below acceptable thresholds.
			However, the limitation does not consider the additional effect of the payload. As long as no workpiece is grasped, collisions are still uncritical. But if a contact occurs after the workpiece is grasped, the potential collision force exceededs the limits.
			& \textbf{MCTS:}\newline $T = 24.6,\ \sigma=25.6,\ N_{miss}=0$\newline
			\textbf{Random Search:}\newline $T = 71.1,\ \sigma=57.9,\ N_{miss}=0$   \\
			\hline
		\end{tabularx}
		\label{tab:TestScenarios}
		\caption{\small Test Scenarios and Results. $^*$Notation: $T$ is the runtime, i.e. the average number of search iterations needed to find the hazard. $\sigma$ is the standard devation of the runtime. $N_{miss}$ denotes the number of unsuccessful test runs where the respective hazard was missed.}
	\end{table*}

	\begin{figure*}
		\centering
		\begin{subfigure}[t]{0.222\textwidth}
			\includegraphics[width=1\textwidth]{images/scenarios/VelocityLimitation_cropped.png}
			\caption{Scenario 3}
		\end{subfigure}
		\hfill
		\begin{subfigure}[t]{0.2\textwidth}
			\includegraphics[width=1\textwidth]{images/scenarios/LightCurtain_cropped.png}
			\caption{Scenario 4}
		\end{subfigure}
		\hfill
		\begin{subfigure}[t]{0.32\textwidth}
			\includegraphics[width=1\textwidth]{images/scenarios/TwoLaserscanners_cropped.png}
			\caption{Scenario 5}
		\end{subfigure}
		\hfill
		\begin{subfigure}[t]{0.226\textwidth}
			\includegraphics[width=1\textwidth]{images/scenarios/CollisionForceLimitation_cropped.png}
			\caption{Scenario 6}
		\end{subfigure}
		\caption{Examples for hazardous situations found in Scenarios 3-6 (for scenarios 1 and 2, see Fig. \ref{fig:ExampleSituation} and Fig. \ref{fig:localMinimum}).}
		\label{fig:Hazards}
	\end{figure*}

	\section{DISCUSSION}
	\label{sec:Discussion}
	The success of our prototypical implementation indicates that our approach is feasible and can find hazards in realistic simulation models with a high success rate. The comparison to the random search also confirms that our approach of generating adversarial human behavior improves search performance.\newline
	Compared to other hazard analysis methods, which typically use simplified or abstracted system models, our approach has the benefit that it essentially treats the system model as a black-box (compare Fig. \ref{fig:SystemOverview}), which means that there is no inherent limit regarding the modeling principles or the model's level of detail. Thus, our method can be deployed on detailed and dynamic simulation models without extensive simplification or abstraction. Furthermore, the approach only requires little prior knowledge about the internal workings of the system that is analyzed, which means that it is easy to deploy for untrained users. In future applications, these properties will also be useful when analyzing systems that are highly complex or rely on black-box components that are difficult to assess with traditional methods (e.g. neural networks).\newline However, the black-box character also has disadvantages: In cases where prior knowledge is available, it cannot be incorporated into the search to increase efficiency. Also, the approach is based solely on falsification, that is, it can find situations where safety criteria are violated, but it cannot give a guarantee that a system is safe (which, however, is the case for all available hazard analysis methods, compare Sections \ref{sec:MethodsAndTools} and \ref{sec:FutureWork}).\newline
	Regarding limitations of the current implementation, it should be noted that the human action space $A$ is quite simplistic. Furthermore, the search algorithm is currently not suitable to deal with local optima or multiple hazards in one scenario (compare Section \ref{sec:ApplicationExample} and Fig. \ref{fig:localMinimum}).

	\section{CONCLUSION AND FUTURE WORK}
	\label{sec:FutureWork}
	Given the promising results in the test scenarios, we aim to develop our approach further into a hazard analysis tool that can be used in industrial practice. To achieve this, we will explore different options for a more detailed human model. Along with this, we will also improve the collision force estimation to improve accuracy and avoid overly conservative estimations (compare Section \ref{sec:Approach} and Footnote 1).
	Apart from these rather straightforward improvements, we must also address a number of more fundamental challenges:
	\begin{itemize}
		\item \textit{{Action-/Search Space:}} In our example, the action space only consists of basic movements. However, depending on the application, other, perhaps more complex actions can be relevant. This leads to the question what types of actions the human action space should contain and how they should be represented. Also, a more fine-granular action space is likely to present new challenges regarding computational complexity.\newline
		\item \textit{Prior Knowledge:} How can we incorporate prior knowledge to increase search efficiency while maintaining the black-box character, and thus, the flexibility of our approach?
		\item\textit{{Local Optima:}} How can we avoid converging in a local optimum and discovering the same hazard repeatedly while missing other, perhaps more critical hazards?
	\end{itemize}
	Finally, it should be stressed that our approach is primarily intended to raise awareness of potential hazards. It cannot give a guarantee that a system is safe, because due to the large search space, an exhaustive search would be infeasible. This limitation is part of the fundamental trade-off between the level of detail of the analysis and its exhaustiveness \cite{Vicentini2019}. While a simulation-based approach may miss hazards due to the large search space, a formal approach may miss hazards due to modeling simplifications that are made to keep system complexity manageable. In the long term, it would be desirable to combine simulation-based and formal analysis techniques to combine both methods' strengths.

	\section*{ACKNOWLEDGMENT}
	The authors thank Tamim Asfour for fruitful discussions and valuable thought impulses.


UBIWEAR: An end-to-end, data-driven framework for intelligent physical activity prediction to empower mHealth interventions

Abstract: It is indisputable that physical activity is vital for an individual's health and wellness. However, a global prevalence of physical inactivity has induced significant personal and socioeconomic implications. In recent years, a significant amount of work has showcased the capabilities of self-tracking technology to create positive health behavior change. This work is motivated by the potential of personalized and adaptive goal-setting techniques in encouraging physical activity via self-tracking. To this end, we propose UBIWEAR, an end-to-end framework for intelligent physical activity prediction, with the ultimate goal to empower data-driven goal-setting interventions. To achieve this, we experiment with numerous machine learning and deep learning paradigms as a robust benchmark for physical activity prediction tasks. To train our models, we utilize, "MyHeart Counts", an open, large-scale dataset collected in-the-wild from thousands of users. We also propose a prescriptive framework for self-tracking aggregated data preprocessing, to facilitate data wrangling of real-world, noisy data. Our best model achieves a MAE of 1087 steps, 65% lower than the state of the art in terms of absolute error, proving the feasibility of the physical activity prediction task, and paving the way for future research.

\section*{Acknowledgment}
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 813162. The content of this paper reflects only the authors' view and the Agency and the Commission are not responsible for any use that may be made of the information it contains.

\bibliographystyle{IEEEtranN}

\section{Discussion \& Conclusions\label{conclusions}}
Despite our contributions, it is essential to acknowledge some limitations of our work. Given the little related work and the lack of open-source benchmarking datasets in the domain, it is infeasible to provide a systematic comparison with relevant algorithms. We hope that by focusing on reproducibility, we pave the way for future research in the field. Additionally, one should be cautious about applying the pre-trained models to different populations. The ``MyHeart Counts'' dataset, despite its great potential, has certain limitations in terms of demographic diversity. Hence, the applicability of the trained models for diverse subgroups of the population needs to be assessed before real-world deployment.

In summary, this paper addresses one of the most unexplored domains in wearable analytics through the introduction of UBIWEAR, a data-driven framework for physical activity prediction for personalized and adaptive goal-setting. Under this spectrum, we utilize real-world in-the-wild data originating from the ``MyHeart Counts" study (C1). The data exploited are at least thirty times larger than other related studies, which allows us to better capture different user patterns and provide more robust and generalized prediction algorithms. We explore both machine and deep learning approaches to provide end-to-end benchmarking. We propose a 3-stacked RNN (LSTM) as a SotA model for this task, achieving 1,087 MAE and 1 MdAE error in step counts given only three previous days' data as input  (C3). Our approach establishes a high standard and prepares the ground for future work. The most time-consuming procedure was the pre-processing pipeline tailored to self-tracking data peculiarities. This led to the design of a set of prescriptive guidelines and the release of an open-sourced Python library to help researchers and practitioners with the highly demanding task of data processing of ubiquitous self-tracking data (C2). Last but not least, all of our work is publicly available for data collection, data ingestion, pre-processing, experimentation, model training, and evaluation with pre-trained models uploaded to facilitate the reproducibility of this study (C4).
\section{Introduction}
According to the World Health Organization (WHO), physical inactivity is one of the leading risk factors for noncommunicable diseases and death worldwide, inducing substantial personal and societal cost \cite{world2019global}. On a personal level, it significantly increases the risk of cancer, heart disease, and diabetes, and it is estimated that up to five million deaths per year could be prevented if the global population were sufficiently active. Deteriorating population health also comes with a growing societal burden in terms of medical care and loss of productivity. Recent estimates show that physical inactivity has led to US\$ 54 billion expense for the health system and US\$ 14 billion in indirect economic losses in the US alone, while worldwide, 1–3\% of national health care expenditures can be attributed to physical inactivity \cite{world2019global}.

A significant challenge towards reducing the global prevalence of physical inactivity is encouraging individuals who are not sufficiently active to alter their behavior and include physical activity in their daily routine. To this end, research has shown that interactive technology can be strategically designed to motivate desirable behavior change, such as regular exercise and healthy nutrition \cite{gouveia2015we}, for better population health and wellness.
At the same time, investing in technological tools and resources for promoting regular physical activity can directly contribute to many of the United Nations' 2030 Sustainable Development Goals \cite{nations2015transforming}. Overall, across all settings, there are opportunities for digital mHealth innovations to harness the potential of data to promote, support, monitor, and sustain health behavior change, focusing on physical activity.

The U.S. Department of Health and Human Services recommends the equivalent of at least 150 minutes of moderate-intensity aerobic activity each week to rip the benefits of regular exercise \cite{pa_guidelines}. Simplifying these guidelines into something concrete and relatable, such as daily step counts, is an easy way for the majority of the population to understand and achieve them. Specifically, daily step counts between 7,000 and 9,000 steps can result in health benefits similar to achieving the recommended amounts of moderate-to-vigorous exercise \cite{kraus2019daily}. To this end, considerable research efforts have been made for developing effective technological interventions to help people achieve the recommended step counts. Amongst the most common and successful persuasive design techniques utilized in such interventions is goal-setting, i.e., setting a daily number of target steps for the user to achieve \cite{yfantidou2021self}.

The most straightforward goal-setting approach, used in most commercial physical activity trackers, is the \textit{fixed goal approach}, where the system sets a fixed goal for the user. However, this approach may lead to unrealistic goals, as it does not consider the singularity of an individual's behavior. At the same time, research shows that a \textit{personalized and adaptive goal approach} performs better in increasing adherence and physical activity levels \cite{adaptive_personalized_advisor}, by tailoring the system to enhance motivational appeal. In this approach, the system monitors the daily behavior of the user, such as physical activity, sleep, and stress levels, and then personalizes their step goal based on a combination of factors. At the same time, it adapts over time to incorporate possible changes in user behavior.

Previous works have attempted to tackle the task of personalized and adaptive goal-setting by employing statistical models for time-series forecasting or domain-expertise-based estimates of step goals tailored to different user groups (see Section~\ref{related-work}). However, the focus is currently shifted towards more intelligent approaches, which provide state-of-the-Art (SotA) performance for the task of physical activity prediction \cite{adaptive_personalized_advisor,dimitrios_vasdekis_2022_5839727}. Nevertheless, the field of physical activity prediction for adaptive goal-setting is still in its infancy, and SotA approaches suffer from various limitations discussed below:

\textbf{Small-scale \& Subjective Experimental Data (\textit{L1}):} Prior work relies on small-scale datasets that are constructed and designed according to conceptual or theoretical interests with articulated research questions, and collected by users participating in ongoing research, which can lead to a distorted reflection of the actual levels of their daily activity \cite{zhou_fukuoka_mintz_goldberg_kaminsky_flowers_aswani_2018};

\textbf{Challenging Data Wrangling (\textit{L2}):}
There are only scattered guidelines concerning preprocessing techniques suitable for handling the idiosyncrasies of real-world, noisy time-series data generated by personal informatics self-tracking systems, contrary to popular domains, such as natural language processing or computer vision;

\textbf{Lack of Physical Activity Prediction Benchmarking (\textit{L3}):} Limited studies exist in the area of predicting physical activity levels by exploiting machine learning and deep learning approaches for enabling personalized and adaptive goal-setting. Even then, the exact architectures and hyperparameters utilized remain unspecified, and a comprehensive benchmarking of relevant approaches is yet to be published;

\textbf{Reproducibility \& Reusability Issues (\textit{L4}):} Due to the limitations introduced by closed-source data and code repositories, reproducibility of published results and reusability of existing code and models for the task of personalized goal-setting is close to infeasible at this moment.

\begin{figure}[htb!]
    \vspace{-3mm}
    \centering
    \includegraphics[width=.55\linewidth]{images/framework-2.png}
    \caption{The UBIWEAR framework covers the path of physical activity data from its acquisition to its application scenarios.}
    \label{fig:ubiwear}

\end{figure}

Motivated by the issues above, this work proposes UBIWEAR, a five-step, data-driven framework for the task of physical activity prediction with the end goal of enabling personalized and adaptive goal-setting interventions via self-tracking. The framework comprises the complete process from physical activity data acquisition to end-user applications. Figure~\ref{fig:ubiwear} depicts UBIWEAR's pipeline. Our contributions are structured around the framework's steps:

\textbf{\textit{C1} - In-the-wild Data Exploitation:} We discuss the availability of open datasets for physical activity prediction, and the idiosyncrasies of self-tracking data. Ultimately, we utilize the openly accessible ``MyHeart Counts'' dataset \cite{hershman2019physical}, containing real-world, large-scale physical activity patterns for thousands of users, allowing us to better capture different segments of the population and build generalizable prediction models. The dataset has been collected in-the-wild, presenting an objective view of the users' behavior (L1).

\textbf{\textit{C2} - Self-tracking Data Processing Guidelines:} We introduce a set of prescriptive guidelines on how to process aggregated data from self-tracking devices, accompanied by a Python library release
\footnote{https://pypi.org/project/ubiwear/}. Derived from our exhaustive experimentation, we propose specific adaptation methodologies for traditional preprocessing techniques explicitly designed to handle self-tracking data idiosyncrasies. Our goal is for these guidelines to provide a more standardized definition of processing self-tracking data (L2).

\textbf{\textit{C3} - Physical Activity Prediction Benchmarking \& Evaluation:} We experiment with six different learning paradigms for physical activity prediction, from machine learning to advanced deep learning architectures, and benchmark their performance for this complex learning task (L3). Through the experimentation with more advanced architectures, UBIWEAR achieves a MAE (Mean absolute error) of 1087 steps, 65\% lower in terms of absolute error than that of the SotA model \cite{mohammadi2019neural}, proving the feasibility of intelligent physical activity prediction.

\textbf{\textit{C4} - Open Access Data and Code:} We purposefully work with an open-access dataset and publicly share our codebase
\footnote{https://github.com/stergiosbamp/deep-physical-activity-prediction} to enable the reproducibility of our results. To further facilitate future interdisciplinary research, we also adopt containerization and offer step-by-step guides on how to download, store and preprocess the data, as well as on how to reproduce our results with the uploaded pre-trained models (L4).

We structure the remaining of this paper as follows: Section~\ref{related-work} discusses the related literature in physical activity prediction and personalized goal-setting. Sections \ref{meth1}-\ref{meth5} introduce the UBIWEAR framework from data acquisition to self-tracking data processing, modeling and evaluation to applications of insights. Finally, Section~\ref{conclusions} concludes the paper and delineates ideas for future work.

\section{Data Acquisition\label{meth1}}
A motivation serving as the backbone of our framework is the peculiarities that materialize the problem of predicting future physical activity with machine learning. Although it can fall under the abstract category of time-series forecasting problems, the self-tracking data idiosyncrasies perplex the problem and thus require particular attention. The UBIWEAR framework is designed with the following in mind:
\begin{itemize}
    \item Raw data collected from sensors or wearables are sampled in arbitrary frequencies contrary to traditional time-series data.
    \item Physical activity datasets are guaranteed to be a multi-subject time-series sequences.
    \item Multiple input devices and overlapping records due to different users can introduce duplicates.
    \item Consecutive records cannot be assumed in a real-world scenario as users do not necessarily track their activities on a daily basis (no-wear time).
\end{itemize}

On top of the above-mentioned peculiarities, data acquisition for physical activity prediction is challenging; collecting proprietary data requires a significant monetary and time investment, while open datasets are limited and include diverse data modalities. Popular small-scale (in terms of sample size) datasets include the Extrasensory dataset (N=60 users) containing a wide range of sensor data combined with activity and location labels \cite{vaizman2017recognizing}; and the StudentLife dataset (N=48) comprised of continuous sensor data, self-reports, and various pre-post surveys combined with activity, mental health and academic performance labels \cite{wang2014studentlife}. On a larger-scale, the FitRec datasets contain user sport records from Endomondo, including multi-modal sequential sensor data and contextual information \cite{ni2019modeling}. In this study, we utilize a recent, large-scale, in-the-wild dataset from the ``MyHeart Counts Cardiovascular Health Study'' \cite{hershman2019physical}. In this study, participants contributed health data via an iPhone application, related to physical activity, fitness, sleep, and overall health for studying the patterns and any potential associations between them. We keep a subset of recordings relevant to our task, i.e., the number of steps that were recorded by the device at a particular time. Our final dataset contains 9,154,490 examples for total of 4,747 unique users of diverse gender, ethnicity, physical and health condition.

\section{Data Processing for Self-tracking Data\label{meth2}}
As discussed in Section~\ref{introduction}, there are no concrete guidelines for the preprocessing of aggregated self-tracking data, despite the idiosyncrasies of the task. Following our data acquisition step, we move to data processing by introducing a series of prescriptive guidelines as part of the UBIWEAR framework (Figure \ref{fig:data-preprocessing}). They consist of four components, namely Data Cleaning, Feature Engineering, Time-series Transformations, and Machine Learning Preparation. Each generic component is composed of traditional data pre-processing steps, \textit{tweaked for the self-tracking data domain}.
\begin{figure}[htb!]
    \vspace{-3mm}
    \centering
    \includegraphics[width=.65\linewidth, keepaspectratio]{images/data-preprocessing.png}
    \caption{An illustration of the data processing pipeline.}
    \label{fig:data-preprocessing}
    \vspace{-2mm}
\end{figure}

\noindent{\textbf{Data Cleaning.}} We initiate data cleaning by \textbf{removing duplicate records}, i.e., records with the same timestamp for a single user. Given the multi-subject and multi-device nature of self-tracking data, duplicate elimination significantly differs from traditional time-series processing. Specifically, self-tracking duplicates removal should be performed on a user-by-user basis, while caution should be given in cases of multiple input devices, as overlapping records can introduce duplicates. We proceed by \textbf{removing the outlier dates and values}. Outlier dates can arise from missing or corrupted dates in the raw data. To overcome this issue, we preserve the dates that correspond to the intervention duration, and drop any remaining missing dates. With regards to outlier step values, there are different sub-cases of outliers, and the decision of how to define them must be chosen carefully. In this work, we adopt the quantile method, for the removal of outlier values and avoid aggressive cut-offs. Also, time-series analysis requires that our time-series data be recorded in fixed-time intervals, so we need to \textbf{achieve a unified granularity}. Self-tracking resampling should be performed only after verifying common granularity for all input sources. In case of arbitrary sampling frequencies, record drops or value splits should precede resampling. The first approach (record drops), adopted in this work, is to drop low frequency records since they represent a small percentage of our dataset. A more sophisticated approach (value split) can be to divide these coarser granularity values by the number of daily active hours, and assign it to the hourly records per example. Next, during \textbf{user filtering}, we filter out any users that do not have the necessary number of days for the requested window size. Note that the larger the requested window size, the stricter the filtering criteria, and hence less users are included in the final dataset. Finally, we proceed with \textbf{zero values imputation}. Zero values observations can be reasonable in the night hours, but in the daytime inactivity is usually caused by forgotten self-tracking devices. Based on that intuition, zero values imputation should be performed only for active hours.\\
\noindent{\textbf{Feature Engineering.}} Feature Engineering includes the transformation of raw data into meaningful variables. A common practice in a majority of time-series problems is to enhance the feature space with features that are derived from timestamps (e.g., hour, day, week, month), as well as their \textbf{\textit{sin} and \textit{cos} transformations} to capture the cyclic nature of time \cite{martucci1994symmetric}. Furthermore, features such as bank holiday and or weekend can facilitate time-series forecasting \cite{taylor2018forecasting}. \\
\noindent{\textbf{Time-series Transformations.}} The \textbf{sliding window technique} allows treating the problem of physical activity prediction as a time-series forecasting problem. Specifically, self-tracking windowing should consider sliding windows for daily granularity and tumbling windows for hourly granularity to avoid excessive overlap of records. Care should be given to the target variable to represent daily (or higher) aggregate counts, in the case of hourly (or finer) granularity.\\
\noindent{\textbf{Machine Learning Preparation.}} The Machine Learning preparation step allows us to prepare our data to be fed into our learning models, including a \textbf{train-validation-test split}. However, in self-tracking data we need to respect chronological order i.e. no shuffling. Since we utilize neural networks, \textbf{feature and target scaling} is almost a mandatory step to ensure that the gradient descent moves smoothly towards the minima and that the steps for gradient descent are updated at the same rate for all features \cite{sola1997importance}.

\section{Modeling for Physical Activity Prediction\label{meth3}}
An important contribution of our work is the exhaustive benchmarking of the various learning models in the previously unexplored domain of physical activity prediction for enabling personalized and adaptive goal-setting.
Under this spectrum, we explore both machine learning and deep learning approaches to provide a comprehensive comparison.

In the machine learning category, we study three families of algorithms: linear models, tree models, and ensemble models. For the case of linear models, we choose the Ridge regressor \cite{mcdonald2009ridge} to minimize the multicollinearity of our data. Due to the hourly granularity, the number of features that arise is high enough. Thus, we believe that the regularization penalty to the loss function would be beneficial. In the case of tree models, we opt for decision trees, where we select the squared error function to measure the quality of a split which minimizes the L2 loss using the mean of each terminal node.
The third family of algorithms we explore is ensemble learning, and specifically the Gradient Boosting Regression \cite{friedman2002stochastic}, because of its capability to reduce both bias and variance in the data.

To achieve an end-to-end benchmarking of physical activity prediction, we also investigate how deep learning models of various architectures fit this task. We scrutinize and apply four different types of architectures. Multilayer Perceptrons (MLPs), 1-D Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) with long short-term memory (LSTM) cells. MLPs is a classic category of neural networks that can be applied in time-series problems by having one output node in the last layer which emits the actual regression value. 1-D CNNs are considered state-of-the-art in the signal processing domain, with their remarkable ability to automatically extract locality features in the time dimension. Last but not least, RNNs are well-suited for approaching time-series problems by catching temporal dependencies along with the input sequence. With regards to the deep learning models, we designed a high-level overview of the optimal architectures that yielded the best performance, following an exhaustive architectures analysis and hyperparameter tuning via trial and error discussed in Section \ref{exp2}.
\begin{figure*}[]
\minipage{0.25\textwidth}
     \includegraphics[width=.8\linewidth]{images/MLP-arch.png}

\endminipage\hfill
\minipage{0.25\textwidth}
  \includegraphics[width=.8\linewidth]{images/RNN-arch.png}

\endminipage\hfill
\minipage{0.25\textwidth}%
  \includegraphics[width=.8\linewidth]{images/CNN-arch.png}

\endminipage
 \caption{The final architectures of the MLP (left), RNN/LSTM (center), and 1D-CNN (right) regressors, respectively.\label{fig:mlp-arch}}
\end{figure*}

\section{Evaluation \& Implications\label{meth4}}
In this section we present our experimentation with data processing (\ref{exp1}) and modeling benchmarking (\ref{exp2}).

\subsection{Effects of Data Processing on Physical Activity Prediction \label{exp1}}
To conclude on the final dataset, we conduct a number of experiments regarding UBIWEAR's processing steps. Our data processing evaluation is relied on a time-aware train-validation-test split of our data, and we select the Ridge regressor as the baseline model in this experiment due to its ability to provide quick and accurate results.

With regards to \textbf{optimal granularity \& window size}, we experiment with hourly and daily aggregation of the data, while we simultaneously experiment with the window size (1-6 days) to obtain more fine-grained results (12 combinations in total). Given that the goal is to provide a model that can be deployed in the real-world, waiting to gather more than a week's data is neither user-friendly nor practical. The difference in error for every window size (MAE$>$500 steps) allows us to deduce that hourly granularity outperforms daily granularity in every case. In terms of window size, the performance of a 3-day window is comparable to this of larger windows. The improvement in MAE and MdAE between the 2-day and 3-day windows (MAE$_2=1417$ and MAE$_3=1361$) is higher than the improvement between the 3-day and larger windows (MAE$_4=1327$, MAE$_5=1300$ and MAE$_6=1281$). This evidence in conjointment to the practicality of smaller windows drives us to select three as the optimal window size.

In addition, we investigate whether the addition of \textbf{date features and cyclical transformations} can benefit the models compared to step features only. We have observed that in terms of MdAE the steps only dataset yields the best results (MdAE$_{steps\_only}=392.14$, MdAE$_{steps\_date\_features}=446.73$, MdAE$_{steps\_cyclic\_features}=574.41$), while the MAE scores are comparable. We assume that this is due to the high overlap of different users' records in our dataset. In other words, by extending the feature space with the date features for the same timestamps but different user behavior, we essentially introduce noise to our machine learning models by diverting attention from the actual activity data.

We finally experiment with \textbf{removing the outlier values} based on the quantiles method with $q=0.05$. Based on our results, we can deduce that indeed outliers removal boosts the performance of the model, by excluding individual abnormal patterns from our data and provides more conclusive predictions in unknown cases (MAE$_{keep\_outliers}=1887.16$ and MAE$_{remove\_outliers}=1307.01$).

Overall, we proceed to the models' benchmarking with a dataset consisting of 3-day windows of hourly granularity, without imputation of missing data, and having undergone outlier removal (both for dates and values with a $q=0.05$), as this combination of preprocessing steps shows the best performance for the ``MyHeart Counts'' dataset.

\subsection{Benchmarking Deep Learning: Does it match the hype?\label{exp2}}
As discussed previously, for the modeling procedure we divide our dataset into three sets: training, validation and test set (no shuffling). We utilize the validation set for the hyper-parameter tuning of the models to provide the optimal results.

\begin{table}[htb!]
    \caption{Performance benchmarking for all ML \& DL models.\label{tab:ml-dl-benchmarking}}
    \centering
    \begin{tabular}{@{}llll@{}}
    \toprule
    \textbf{Model} & \textbf{Dataset} & \multicolumn{2}{c}{\textbf{Metrics}} \\ \midrule
     &  & \textbf{MAE} & \textbf{MdAE} \\ \midrule
    \multirow{3}{*}{Ridge} & Train & 1372.889 & 458.038 \\

     & Test & 1359.766 & 390.271 \\ \midrule
    \multirow{3}{*}{Decision Tree} & Train & 1428.906 & 592.652 \\

     & Test & 1478.086 & 224.101 \\ \midrule
    \multirow{3}{*}{Gradient Boosting} & Train & 1209.139 & 463.878 \\

     & Test & 1222.738 & 135.668 \\ \midrule
    \multirow{3}{*}{MLP} & Train & 1148.293 & 296.582 \\

     & Test & 1094.822 & \textbf{0.082} \\ \midrule
    \multirow{3}{*}{CNN} & Train & 1167.130 & 284.548 \\

     & Test & 1099.581 & 11.110 \\ \midrule
    \multirow{3}{*}{RNN (LSTM)} & Train & 1171.620 & 280.274 \\

     & Test & \textbf{1087.838} & 0.855 \\ \bottomrule
    \end{tabular}

\end{table}
For the classic machine learning models, we find the best hyperparameters per paradigm through a grid search, time-series-based cross-validation technique with 5 splits. For the deep learning models, we identify the optimal hyperparameters by experimenting with various combinations of learning rate, dropout, hidden units, batch size, optimizer, kernel size, pooling and stride, per each architecture. Figure \ref{fig:mlp-arch} delineates the final architectures for the MLP, RNN and CNN, accordingly.

Table \ref{tab:ml-dl-benchmarking} shows the overall modeling outcomes against the train and test datasets. The best performing model is found to be the RNN achieving 1087 MAE and only 1 step of MdAE, confirming the assumption that RNNs are able to capture temporal dependencies in time-series problems. Equally well performance is delivered by a 3-layered feed-forward network (MLP) with almost similar results on the hold-out set. The lower MdAE from MLP can be justified on the intuition that predictions from this architecture are more robust to outlier examples, in contrast to the ones from RNN architecture. However, this difference is negligible since RNN can predict the overall cases more efficiently.
The CNN proved unsuitable for extracting features from the convolution operation although we utilized a fairly large kernel size given the temporal features. Despite that, the deep learning models deliver almost 1.25 times better performance than a simple regression model (e.g., Ridge) in terms of MAE, but the required computational resources and training time are significantly higher, a fact that should be considered before deducing that neural networks are the proper fit for every task. This is also more apparent in the case of deployment of such models for delivering interventions based on predictive modeling in wearable devices, where that computational resources are limited.

\section{UBIWEAR Applications\label{meth5}}
A vital question emerging is how our work and ultimately machine learning-based solutions for physical activity prediction can be embraced in real-world scenarios. Through our research, we provide some indicative use cases in which UBIWEAR can be applied.

First and foremost, by predicting personalized physical activity levels from the unique users' data, we can infer adaptive and challenging future step goals by increasing the predicted step goal by 10\%. Research has shown that activity goals proximate to personal levels for each individual can be advantageous for encouraging them and maintaining motivation to complete daily physical activity goals \cite{adaptive_personalized_advisor}. Even industrial wearables software and mHealth services for exercise can integrate our pre-trained models and inject into their system our predictions, to adapt to the specificities of each person and deliver intelligent future step goals. Furthermore, in a similar manner UBIWEAR can be adopted in randomized controlled trials (RCTs) to study behavioral changes in different groups of users. The outcome of such a procedure can benefit behavioral transformation interventions in a majority of healthcare applications that promote healthy lifestyle habits from different disciplines such as medicine or pharmacy. Therefore they can focus on the development of the clinical support systems and alleviate the cost of conducting an end-to-end, time-consuming pipeline and designing a machine learning system from scratch for that purpose.

\section{Related Work}

\section{Related Work: Intelligent Physical Activity Prediction for Personalized Goal-setting}
An emerging body of work is trying to tackle the lack of intelligent goal-setting systems by publishing studies of statistical, rule-based, machine learning- or deep learning-based physical activity prediction solutions that take into account historic data to provide realistic and achievable exercise goals.

\citet{van2018enhancing} developed a context-aware coaching system that adapts to each individual by taking into consideration factors such as context, and historical patterns of behavior. Similarly, ``Active2Gether" \cite{klein2017active2gether} is a coaching system that utilizes the behavioral characteristics of users and social comparison to provide personalized feedback. However, the most fundamental component of both systems, namely the reasoning engine that determines what type of support the user should receive, relies on a purely rule-based approach rather than a predictive one, limiting its extensibility potential. With regards to statistical forecasting approaches, \citet{zhou2018evaluating} investigated the effectiveness of personalized automated adaptive step goals, in comparison to fixed 10000 daily step goals. Their findings revealed that the intervention group had statistically significant more daily steps than the fixed steps goal group, but only few details were provided about the behavioral algorithm used.

Closer to our work, \citet{dijkhuis2018personalized} developed a machine learning system that aimed to predict whether or not an individual will meet their daily step goal. The authors benchmarked eight traditional machine learning models, but oversimplified the task of predicting activity goals as a binary decision. To bridge this gap, \citet{dimitrios_vasdekis_2022_5839727} developed a framework for machine learning-based physical activity prediction formulated as a regression task. While they explored a variety of significant contextual factors, their small-scale dataset might not be representative of the general population, while it prohibited them from bechmarking data-hungry SotA deep learning approaches. \citet{mohammadi2019neural} experimented both with traditional machine learning and a neural network architecture to predict a dynamically adjusted daily number of steps based on personal, environmental, and social factors. However, their experimentation was solely based on a small-scale dataset and they did not provide any information concerning the deep learning architectures and hyperparameters used, limiting the reproducibility and reusability of their approach. Their SotA ``BRIDGE'' model achieved a MAE of 1672 steps, 65\% higher than our proposed approach.

It is evident that only a limited number of studies exploit machine learning or deep learning techniques for the task of physical activity prediction for enabling personalized and adaptive goal-setting, even though SotA models provide significantly superior performance in similar tasks, such as human activity recognition or sleep stage classification. This gap motivated our work and guided UBIWEAR's methodology as will be presented in detail in the following sections (\ref{meth1}-\ref{meth5}).

Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)