Manipulating raw observation and constructing new features are widely used for deriving state representation. These methods range from simple approaches such as tile coding applied to linear function approximation methods for problems like n-state random walk [1]}, [2]}, to more complex methods like structure2vec [3]} and Pointer Networks [4]} for graph Combinatorial Optimization Problems (COPs) such as Vertex Cover [5]} and TSP [6]}. They are mainly employed for expanding the observation to more useful representations with or without taking the final policy into account. For problems like combinatorial optimization [7]}, robot navigation [8]} and real world business problems like train shunting [9]} and online advertising [10]}, raw observation of the environment may require processes to derive state representation.
