1. A method for a robotic system, the method comprising: generating kinematics information for a robotic instrument within a field of view of a camera; capturing image information in the field of view of the camera; and adaptively fusing the kinematics information and the image information together to determine pose information of the robotic instrument; wherein the adaptive fusing of the kinematics information and the image information together includes synthesizing an image of a model of the robotic instrument in response to the kinematics information, matching the image of the model to the image information to generate match results, filtering the match results, and forming a plurality of states of the pose information for the robotic instrument in response to the filtered match results and kinematics information.