1. A robotic-method for performing surgery on a patient through a minimally invasive aperture, the method comprising: positioning, by a robotic surgical system, an elongate shaft of a tool through the minimally invasive aperture so that the shaft pivots at an actual pivotal center adjacent the minimally invasive aperture; acquiring, by an image capture device, an image of a surgical site within the patient, the surgical site image including an image of a distal portion of the tool within the field of view of the image capture device, the image of the distal portion including an image of markers, the image capture device being inserted into the patient; determining, by the robotic surgical system using the image of the markers, an image-based location of the actual pivotal center; determining image-based pivotal center correction information using the image-based location of the actual pivotal center; and determining, by the robotic surgical system, a position and/or orientation of the tool by imposing the image-based pivotal center correction information onto joint-based data, the joint-based data being used by a command module to calculate movements of a tool manipulator supporting the tool.