More recent work has adopted the idea of state augmentation to overcome this issue: by introducing
an extra state variable, an optimal policy can be sufficiently characterized as a Markov process
defined on this augmented state space. Broadly speaking, these studies develop CVaR MDP
frameworks using two kinds of state augmentation. The first kind introduces an extra state
variable that represents the running cost, and derives the dynamic programming principle from the
primal variational representation of CVaR, i.e.,
\(\operatorname{\mathrm {CVaR}}_q[C] = \min _{c \in \mathbb {R}}\lbrace  c + \frac{1}{q} \mathbb {E}[ (C-c)^+ ] \rbrace \)  [1]}.
This state augmentation scheme is adopted in, for example,
[2]}, [3]}, [4]}, [5]}, [6]}. The second kind of state augmentation
introduces an extra state variable that represents the quantile value, and derives the dynamic
programming principle from the dual variational representation of CVaR, i.e.,
\(\operatorname{\mathrm {CVaR}}_q[C] = \sup _{\mathbb {Q} : \mathbb {Q} \ll \mathbb {P}, \frac{d\mathbb {Q}}{d\mathbb {P}} \le q^{-1} }\mathbb {E}_{\mathbb {Q}}[ C ] \)  [7]}. The work of
[8]}, [9]}, [10]}, [11]} belongs to this category.
