Building a conversational AI system, like CIR, also imposes a challenge regarding the development of continual learning methods that allow the system to adapt in a dynamic environment. Most AI models, including the CIR modules described in this book, are developed using independent and identically distributed (IID) learning methods that assume that the world is static, and the data of the tasks that the models need to perform can be pre-collected. However, a conversational system has to work in a non-stationary world, where the tasks and settings change constantly. For example, XiaoIce has to release new dialog skills every week since its first release in 2014 to keep and grow its user base, and many of these skills are short lived [1]}. Ideally, the system should be able to adapt itself to the environment efficiently with little supervision of new tasks or settings without catastrophic forgetting, where the performance on old tasks degrades sharply when the system adapts to new tasks. Recent neuroscience studies show that humans can adapt much more efficiently than machines because humans are born with rich innate knowledge inherited from humans’ long evolutionary history and have the ability of transferring knowledge efficiently in two directions in our life-long learning: in forward transfer, previously learned tasks help improve the performance of new tasks, and in backward transfer, learning a new task improves the performance of old tasks [2]}. Inspired by these neuroscience studies, continual learning with neural networks becomes an active research field [3]}, [2]}.
