Non-stationary domains, that change in unpredicted ways, are a challenge for agents searching for optimal policies in sequential decision-making problems. This paper presents a combination of Markov Decision Processes (MDP) with Answer Set Programming (ASP), named Online ASP for MDP (oASP(MDP)), which is a method capable of constructing the set of domain states while the agent interacts with a changing environment. oASP(MDP) updates previously obtained policies, learnt by means of Reinforcement Learning (RL), using rules that represent the domain changes observed by the agent. These rules represent a set of domain constraints that are processed as ASP programs reducing the search space. Results show that oASP(MDP) is capable of finding solutions for problems in non-stationary domains without interfering with the action-value function approximation process.
For mobile robots to perform complex missions, it may be necessary for them to plan with incomplete information and reason about the indirect effects of their actions. Answer Set Programming (ASP) provides an elegant way of formalizing domains which involve indirect effects of an action and recursively defined fluents. In this paper, we present an approach that uses ASP for robotic task planning, and demonstrate how ASP can be used to generate plans that acquire missing information necessary to achieve the goal. Action costs are also incorporated with planning to produce optimal plans, and we show how these costs can be estimated from experience making planning adaptive. We evaluate our approach using a realistic simulation of an indoor environment where a robot learns to complete its objective in the shortest time. The action language BC provides an elegant way of formalizing dynamic domains which involve indirect effects of actions and recursively defined fluents. In complex robot task planning domains, it may be necessary for robots to plan with incomplete information, and reason about indirect or recursive action effects. In this paper, we demonstrate how BC can be used for robot task planning to solve these issues. Additionally, action costs are incorporated with planning to produce optimal plans, and we estimate these costs from experience making planning adaptive. This paper presents the first application of BC on a real robot in a realistic domain, which involves human-robot interaction for knowledge acquisition, optimal plan generation to minimize navigation time, and learning for adaptive planning.