Expression of IDO and PD-1/L1 has been found to be increased in endometrial cancer. Expression of these markers on the tumor cells leads to immunosuppression in the micro-environment of tumors that prevents immune system from attacking and killing tumor cells. The purpose of this trial is to target these antigens by concomitant administration of drugs epacadostat and pembrolizumab, thereby removing twin mechanisms of immune-suppression that may lead to better control of tumor growth.