High-level semantic concepts of the image are effective to describe a unique situation and a relation between objects in an image<cite> [4,</cite> 10] . Extracting specific arXiv:1807.09434v1 [cs.CV] 25 Jul 2018 semantic concepts encoded in an image, and applying them into RNN network has improved the performance significantly<cite> [4]</cite> .