This phase II trial is studying how well giving sorafenib tosylate together with erlotinib hydrochloride works in treating patients with locally advanced, unresectable, or metastatic gallbladder cancer or cholangiocarcinoma. Sorafenib tosylate and erlotinib hydrochloride may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth or by blocking blood flow to the tumor.