More than 130 primary tumors of the central nervous system (CNS) have been identified. Most affect less than 1,000 people in the United States each year. Because these tumors are so rare, there are few proven therapies. This study will test whether the immunotherapy drug nivolumab is an effective treatment for people with rare CNS tumors.