Recently, interesting results for translating navigation instructions to a high-level plan are discussed in [1]}, [2]}, where a graph is first built according to the preliminary knowledge of the environment. The detailed language text commands then guide the graph search algorithms to obtain the viable edges, which eventually form the navigation plan. However, this approach requires rather rich preliminary information, thus knowing the destination is enough to generate path plan. Hence the text commands appear to be redundant. In literature, a general frame of reinforcement learning (RL) has been investigated for the purpose of grounding the natural languages into robot behaviors [3]}, [4]}, [5]}. Normally the system description leverages a system model related to Markov Decision Process (MDP) such as object-orientated MPD (OO-MDP) or
linear temporal logic (LTL). Based on such robot behavior model, RL algorithms are explored in order to build a mapping from natural language to certain action. Nonetheless, applications of such framework is restricted, due to the limited number of robotic tasks that can be transferred into a MDP model. Combining vision and language for navigation has also attract much research attention [6]}, [7]}, [8]}. These approaches normally try to reveal the correspondence between the image and the text, based on which the next step action is selected among a finite set of pre-defined actions.
