This phase I trial studies the side effects and the best dose of everolimus when given together with brentuximab vedotin in treating patients with Hodgkin lymphoma that has come back (relapsed) or is not responding to treatment (refractory). Everolimus may stop the growth of cancer cells by blocking some of the enzymes needed for cell growth. Brentuximab vedotin may interfere with the ability of cancer cells to grow and spread by binding to a protein on the surface of cancer cells and then releasing a cancer-killing substance to them. Giving everolimus together with brentuximab vedotin may be a better treatment for Hodgkin lymphoma.