Simulations are run with two sample sizes, \(n = 300\)  and 1000. Thus, we have in total 16 distinct factorial settings (four hazard/censoring scenarios, two propensity designs, and two sample sizes). For each setting, we have \(n_{rep} = 200\)  simulation replicates, and we estimate the optimal treatment rule based on both the \(\phi _\mu \)  and \(\phi _{\sigma ,t=5,\mu }\)  criteria. We compare the proposed method with the methods in [1]} and [2]}. We further compare the results with the observed policy characterized by (REF ) and the estimated optimal zero-order model, where all patients are given the same embedded treatment regime, which is the best on average over all embedded regimes. In particular, in implementing the [1]} method, the linear \(Q\) -function space and the random forest-based approximation space were separately used. The method of [4]} is not included in these simulations because [2]} can be viewed as a doubly robust extension of their method. After estimation, the values—either the expected truncated survival time or the survival probability at \(t=5\) —are evaluated by generating an uncensored sample of size \(n_{eval} = 10000\)  according to the estimated policies. The simulations were complete using R package dtrSurv. For reproducible research, the code is available at https://github.com/hunyong/survQlearn.
