In our framework, each state vector \(s_t\)  represents a molecular structure at the \(t\) -th step, and each step is a sequence of three actions that define an augmentation of a fragment. Following the method from Hwang et al. [1]}, the current state molecule is represented as an undirected graph \(G\)  which consists of a node feature matrix, an adjacency matrix, and an attachment site masking vector.
A graph convolutional network (GCN) [2]}-based network provides node embeddings, \(H\) , which are then sum-aggregated to produce a graph embedding, \(h_g\) . Note that the same GCN encodes the current state molecular graph (for Action 1) and the fragments (for Action 3).
