This work presents an efficient method to solve a class of continuous-time, continuous-space stochastic optimal control problems of robot motion in a cluttered environment. The method builds upon a path integral representation of the stochastic optimal control problem that allows computation of the optimal solution through sampling and estimation process. As this sampling process often leads to a local minimum especially when the state space is highly non-convex due to the obstacle field, we present an efficient method to alleviate this issue by devising a proposed topological motion planning algorithm. Combined with a receding-horizon scheme in execution of the optimal control solution, the proposed method can generate a dynamically feasible and collision-free trajectory while reducing concern about local optima. Illustrative numerical examples are presented to demonstrate the applicability and validity of the proposed approach.
The proposed method in this work takes advantage of the architecture in @cite_20 : it first expands a goal-rooted backward tree over a topology-embedded state space without considering uncertainty in the dynamics. Then, instead of directly using the tree as a feedback policy, the control policy for LSCO is computed using the path integral control method.
@cite_20
2042204128 2400287765 2126000336 2119403022
We present an algorithm that probabilistically covers a bounded region of the state space of a nonlinear system with a sparse tree of feedback stabilized trajectories leading to a goal state. The generated tree serves as a lookup table control policy to get any reachable initial condition within that region to the goal. The approach combines motion planning with reasoning about the set of states around a trajectory for which the feedback policy of the trajectory is able to stabilize the system. The key idea is to use a random sample from the bounded region for both motion planning and approximation of the stabilizable sets by falsification; this keeps the number of samples and simulations needed to generate covering policies reasonably low. We simulate the nonlinear system to falsify the stabilizable sets, which allows enforcing input and state constraints. Compared to the algebraic verification using sums of squares optimization in our previous work, the simulation-based approximation of the stabilizable set is less exact, but considerably easier to implement and can be applied to a broader range of nonlinear systems. We show simulation results obtained with model systems and study the performance and robustness of the generated policies. A popular approach for online decision making in large MDPs is time-bounded tree search. The effectiveness of tree search, however, is largely influenced by the action branching factor, which limits the search depth given a time bound. An obvious way to reduce action branching is to consider only a subset of potentially good actions at each state as specified by a provided partial policy. In this work, we consider offline learning of such partial policies with the goal of speeding up search without significantly reducing decision-making quality. Our first contribution is to study learning algorithms based on reducing our learning problem to i.i.d. supervised learning. We give a reduction-style analysis of three such algorithms, each making different assumptions, which relates the supervised learning objectives to the sub-optimality of search using the learned partial policies. Our second contribution is to describe concrete implementations of the algorithms within the popular framework of Monte-Carlo tree search. Finally, the third contribution is to evaluate the learning algorithms in two challenging MDPs with large action branching factors, showing that the learned partial policies can significantly improve the anytime performance of Monte-Carlo tree search. In this paper, the state feedback control design problem for tracking control of a flexible link manipulator is considered. The calculation of state feedback control gains is conventionally handled by pole placement method or LQR method via Riccati equation. Unfortunately, they still possess trial and error approach of choosing some parameters. Particularly, choosing elements of Q and R matrices in the state feedback control using LQR method has to be done by trial. Therefore, an intelligent method to resolve this problem is proposed by adopting PSO algorithm. The experimental work is carried out to evaluate the effectiveness of the proposed method. In this paper, state feedback control design for tracking control of a flexible link manipulator is considered. The computation of state feedback control gains is conventionally handled by pole placement method or LQR method via Riccati equation. Unfortunately, they still possess trial-and-error approach in choosing their parameters. Particularly, choosing elements of Q and R matrices in the state feedback control design using LQR method has to be done by trial. Therefore, an intelligent method to resolve this problem is proposed by adopting PSO-based optimization. Experimental work is carried out to evaluate effectiveness of the proposed method.