This paper presents a formal definition and explicit representation of robot motion risk. Currently, robot motion risk has not been formally defined, but has already been used in motion and path planning. Risk is either implicitly represented as model uncertainty using probabilistic approaches, where the definition of risk is somewhat avoided, or explicitly modeled as a simple function of states, without a formal definition. In this work, we provide formal reasoning behind what risk is for robot motion and propose a formal definition of risk in terms of a sequence of motion, namely path. Mathematical approaches to represent motion risk are also presented, which is in accordance with our risk definition and properties. The definition and representation of risk provide a meaningful way to evaluate or construct robot motion or path plans. The understanding of risk is even of greater interest for the search and rescue community: the deconstructed environments cast extra risk onto the robot, since they are working under extreme conditions. A proper risk representation has the potential to reduce robot failure in the field.
Although researchers have investigated risk-aware planning from a variety of directions, including Partially Observable Markov Decision Process (POMDP) with negative reward as risk penalty @cite_5 , constrained POMDP @cite_8 , or risk allocation @cite_4 , the definition of risk was always taken for granted and only based on the ad hoc applications.
@cite_5 @cite_4 @cite_8
2952258289 2099495504 2123372395 2099873296
The partially observable Markov decision process (POMDP) provides a principled general framework for planning under uncertainty, but solving POMDPs optimally is computationally intractable, due to the "curse of dimensionality" and the "curse of history". To overcome these challenges, we introduce the Determinized Sparse Partially Observable Tree (DESPOT), a sparse approximation of the standard belief tree, for online planning under uncertainty. A DESPOT focuses online planning on a set of randomly sampled scenarios and compactly captures the "execution" of all policies under these scenarios. We show that the best policy obtained from a DESPOT is near-optimal, with a regret bound that depends on the representation size of the optimal policy. Leveraging this result, we give an anytime online planning algorithm, which searches a DESPOT for a policy that optimizes a regularized objective function. Regularization balances the estimated value of a policy under the sampled scenarios and the policy size, thus avoiding overfitting. The algorithm demonstrates strong experimental results, compared with some of the best online POMDP algorithms available. It has also been incorporated into an autonomous driving system for real-time vehicle control. The source code for the algorithm is available online. Partially Observable Markov Decision Processes (POMDPs) model sequential decision-making problems under uncertainty and partial observability. Unfortunately, some problems cannot be modeled with state-dependent reward functions, e.g., problems whose objective explicitly implies reducing the uncertainty on the state. To that end, we introduce ρPOMDPs, an extension of POMDPs where the reward function ρ depends on the belief state. We show that, under the common assumption that ρ is convex, the value function is also convex, what makes it possible to (1) approximate ρ arbitrarily well with a piecewise linear and convex (PWLC) function, and (2) use state-of-the-art exact or approximate solving algorithms with limited changes. The Partially Observable Markov Decision Process (POMDP) framework has proven useful in planning domains where agents must balance actions that provide knowledge and actions that provide reward. Unfortunately, most POMDPs are complex structures with a large number of parameters. In many real-world problems, both the structure and the parameters are difficult to specify from domain knowledge alone. Recent work in Bayesian reinforcement learning has made headway in learning POMDP models; however, this work has largely focused on learning the parameters of the POMDP model. We define an infinite POMDP (iPOMDP) model that does not require knowledge of the size of the state space; instead, it assumes that the number of visited states will grow as the agent explores its world and only models visited states explicitly. We demonstrate the iPOMDP on several standard problems. In this paper, we describe the partially observable Markov decision process (POMDP) approach to finding optimal or near-optimal control strategies for partially observable stochastic environments, given a complete model of the environment. The POMDP approach was originally developed in the operations research community and provides a formal basis for planning problems that have been of interest to the AI community. We found the existing algorithms for computing optimal control strategies to be highly computationally inefficient and have developed a new algorithm that is empirically more efficient. We sketch this algorithm and present preliminary results on several small problems that illustrate important properties of the POMDP approach.