 insecticide. Remember, before using any insecticide, read the label, and follow directions and safety precautions.
For carpenter bees, coating worn exterior surfaces with a polyurethane or oil-base paint will help. But the best bet is simply to plug the hole. Caulking can sag and shrink. Exterior-grade wood putty works better. Or drive a half-inch-diameter dowel into the hole with some glue, and cut to leave a flush surface.