first of all , the theory on which the surgical probe of the invention is based will be discussed . endoscopes used in modem minimally invasive surgery typically utilize ccd cameras which record a two - dimensional projection of objects within the camera &# 39 ; s field of view . information about the third dimension is lost in such a projection . a methodology for reconstructing shape from a single two - dimensional image has been developed at the harvard robotics laboratory in cambridge , mass . this method uses a fluid - filled elastic membrane as a sensor for shape reconstruction . the fluid may be compressible or incompressible . black dots drawn in a matrix pattern on the inside surface of the fluid - filled membrane move as the membrane deforms upon encountering objects on its outer surface . the method is an example of biomimetics in that the shape reconstruction sensor is designed to mimic an actual fluid filled human finger tip . the algorithm for reconstructing shape relies on methods of projective geometry , differential equations and matrix algebra . see , n . j . ferrier , k . a . morgansen , d . hristu , &# 34 ; implementation of membrane shape reconstruction ,&# 34 ; technical report 97 - 1 , harvard robotics lab , harvard university ( 1997 ), the contents of which are incorporated by reference herein . as disclosed in this technical report , information about two dimensions x and y is recorded directly onto the ccd element of the camera . in order to determine the z component of each dot &# 39 ; s position , a constant volume constraint coupled with a minimum energy constraint and a fixed boundary condition are sufficient to limit the infinite number of solutions to one unique solution . the method depends on a fluid - filled , elastic membrane in which a fluid establishes a constant volume and the elasticity of the membrane allows for the application of an energy minimizing constraint . the algorithm is processed in a computer which generates an animated wire - frame model of the shape of the deformed membrane which thus provides information about the object deforming the membrane . the other major aspect of the probe relates to doppler velocimetry . doppler velocimetry is commonly used in medical applications to detect and diagnose blood flow in fetal monitoring and in arterial disease . doppler velocimetry is based on the well known doppler effect in which waves reflected from a moving object undergo a frequency shift . the shift is proportional to the speed of the moving object and is a result of the fact that the moving object reflects the waves that it encounters at the frequency at which it encounters them , not at the original frequency at which they were sent . thus , if an object is moving toward the wave source with a given velocity , it will encounter more waves per second then if it were standing still . the object will reflect this increased frequency to the source . the frequency shift can be measured between the reference wave which is propagated toward the object , and the frequency shifted wave that is reflected . doppler probes have been used for many years for fetal heart rate monitoring as well as non - invasive blood flow diagnosis on patients with heart disease . although the interpretation of the output of doppler probes is subjective , an experienced vascular specialist can accurately diagnose blood vessel characteristics from the sound of a doppler signal . for example , phasicity , which refers to the direction of blood flow , can be determined from a doppler signal . in addition , a vascular expert is able to detect an arterial obstruction by listening to the doppler signal and hearing the healthy &# 34 ; one - two - three &# 34 ; swishing sound deteriorate into a &# 34 ; one - two &# 34 ; swishing sound . these two sounds are referred to as a triphasic waveform and a biphasic waveform because they have either three or two distinct phases for each pulse of blood . similarly , a monophasic waveform in which only one swishing sound is heard per beat indicates more advanced arterial disease . as will be seen below , the output of the surgical probe of the invention is an audible signal from which motion can be inferred coupled with a visual display which provides shape reconstruction information . with reference now to fig1 a surgical probe 10 includes a casing 12 including a recessed portion 14 . the casing 12 is designed to the exact shape of an eyepiece of a two millimeter standard endoscope ( not shown ). an inner steel channel 16 is attached to the casing 12 and serves as a channel to guide an endoscope through the probe 10 . the inner channel 16 is approximately 255 mm long , 3 mm od and 2 . 2 mm id . it is preferred that this channel be made of 316 steel tube which is fda approved as body implantable . the inner steel channel 16 is disposed within an outer probe tubing 18 which is also preferably made of 316 stainless steel . it is preferred that the outside diameter of the outer probe tubing 18 be 10 mm with an inside diameter of 7 mm . a 10 mm outside diameter is preferred because trocars , which are tunnels through which minimally invasive tools enter the body , are made in sizes only up to 10 mm in diameter . the 7 mm inside diameter provides a suitable wall thickness for a rigid probe . a polystyrene window 20 resides in a shelf in the end of the outer probe tubing 18 . the polystyrene window 20 is approximately 0 . 259 inches in diameter and was machined from 0 . 05 inches thick clear polystyrene . as shown in fig2 ultrasound transducers 22 and 24 are disposed on the polystyrene window 20 . polystyrene is used for the window 20 because polystyrene has good acoustical coupling with 5 mhz ultrasound crystals such as the ultrasound transducers 22 and 24 . attached to the end of the outer probe tubing 18 is an elastic membrane 26 . a suitable elastic membrane 26 is a 3 / 8 inch diameter by 2 inches long pipe seal available from north american latex company of sullivan , ind . the preferred pipe seal was created in a special run with two dips in white and then two dips in black latex so that it is both opaque and yet provides good contrast for black dots 28 on a white inside surface . it is preferred that the elastic membrane 26 be part of a replaceable tip structure for replacement as needed . the black dots 28 were placed on the membrane 26 by mounting the latex membrane in the vice of a numerically controlled milling machine . a waterproof ink pen was placed in the collet of the milling machine instead of a normal milling bit . the pen applied dots to the membrane in a 3 × 3 grid according to commands given to the computer connected to the milling machine . the position of the dots 28 was recorded from the computer controlling the machine . it is preferred that the elastic membrane 26 be filled with an incompressible fluid such as an rtv gel 30 . with reference now to fig1 and 2 , a charge coupled device camera 32 is positioned within an opening in the polystyrene window 20 . alternatively , the camera 32 may be remotely located and light delivered to the camera 32 by , for example , a fiber optic bundle ( not shown ). the camera 32 forms a part of an endoscope ( not shown ) which slides through the inner steel channel 16 . the camera 32 is adapted to create an image of the dots 28 on the inner surface of the elastic membrane 26 . optical fibers 34 and 36 illuminate the inside surface of the membrane 26 allowing the camera 32 to form an image of the black dots 28 . more than two optical fibers may be provided if desired . the response of the ultrasound transducer crystals 22 and 24 were selected to be centered in the 5 mhz frequency range in order to penetrate tissue to a depth of 1 - 3 cm . lower frequencies do not penetrate as deeply and higher frequency result in greater signal attenuation . this frequency range also works well with electronics that drive them . those skilled in the art will appreciate that it is important to maximize the area of the crystals 22 and 24 since the depth of penetration is directly proportional to crystal area . suitable crystals 22 and 24 are pzt transducer material available from nicolet vascular inc . of golden , colo . the crystals normally come in half disk shapes and so a special design was ordered to provide the opening for the camera 32 . in this way , surface area was maximized while providing a line of sight through the center of the crystals 22 and 24 for the endoscope camera 32 . prism 401 glue available from the locktight corporation was used to adhere the crystals 22 and 24 to the polystyrene window 20 . this glue has good acoustical coupling properties . as will be appreciated by those skilled in the art , electrical leads for energizing the transducer crystals 22 and 24 are provided and access is obtained through an opening 38 in the casing 12 . the optical fibers 34 and 36 may also pass through the opening 38 . acrylic spacers 40 and 42 are provided for centering the inner steel channel 16 and to guide the optical fibers 34 and 36 and leads ( not shown ) from the transducer crystals 22 and 24 . care must be taken to minimize tensile stresses in the thin transducer leads . the electronics ( not shown ) for the doppler transducers were generated and processed by integrated circuits designed by nicolet vascular , inc . and used in the pocketdop ii handheld doppler device . it is also preferred that the electronics be battery powered in order to eliminate the risk of accidental electrical shock during a surgical procedure . it is also preferred that the camera 32 protrude slightly through the opening in the window 20 to minimize glare . it is also preferred that a multistrand fiber optic bundle be used for illuminating the inside surface of the membrane 26 . further details of a design prototype of the probe 10 are provided in thierman , jonathan , &# 34 ; surgical probe for the detection of pulsing blood vessels and embedded lesions ,&# 34 ; harvard university engineering senior design thesis ( 1998 ), the contents of which are incorporated herein by reference . with reference now to fig3 an output 50 from the doppler portion of the probe 10 is applied to an amplified speaker 52 which produces sounds characteristic of velocity of an object being interrogated . an output 54 from the shape reconstruction portion of the probe 10 is processed by a computer 56 and displayed on a video monitor 58 . the monitor 58 may display the raw image of the dots 28 in which case the surgeon infers shape directly from motion of the dots . alternatively , the computer 56 may process the signal 54 using the reconstruction algorithm discussed above to generate animated wire frame representations . fig4 a and 4b show shape reconstruction for an undeformed membrane ( fig4 a ) and for a deformed membrane ( fig4 b ). in use , the probe 10 is inserted into a surgical field through a trocar . the inner steel channel 16 , attached to the casing 12 , serves as a channel to guide the endoscope through the probe 10 . the surgeon watches the video display 58 which provides tactile feedback by showing the surface topography of an object encountered by the deformable tip of the probe . the surgeon also listens to the audio presented by the speaker 52 to ascertain the sounds consistent with the motion of , for example , flowing blood or a pulsing blood vessel . the combination of the video and audio information enables the surgeon to distinguish between the cystic duct and the cystic artery during the cholecystectomy procedure as one prime example of the use of the probe of the invention . the shape reconstruction and doppler velocimetry aspects of the invention work together exceptionally well in the surgical probe . the shape reconstruction portion relies on a fluid - filled membrane disposed on the probe tip . this fluid - filled membrane also provides an ideal acoustic window for propagation of the doppler signal through the latex membrane . neither aspect interferes with the other and can be packaged within the 7 mm space inside the probe tubing . a prototype of the probe was assembled and tested to establish that both the shape reconstruction and doppler blood vessel detection sensors were working as predicted . although a computer would normally be connected directly to the endoscopic video system , for testing purposes a vhs recording was made of the image obtained from the endoscope . this recording was then played through a vcr plugged directly into an epix video capture board on a dual - pentium 200 computer . in this way , the shape reconstruction software could access the video images in real time ( even though the images were actually recorded earlier , they were still played back in real time ) in order to process the images and reconstruct the shape of the membrane . in these experiments , it was necessary to calibrate the reconstruction software with the correct camera parameters , undeformed dot positions , and threshold values for determining which pixels were black dots and which were white latex . the shape reconstruction software used a 9 - by - 9 mesh of undeformed points even though there was actually a 3 - by - 3 mesh of dots drawn on the membrane . the additional dots were obtained by cubic interpolation in 3 - space using matlab finctions . in addition , modifications were made to the shape reconstruction software to change the pixel values of the image outside of a certain field of view to 255 ( white ) in order to eliminate noise that this unused part of the image was causing . the software was run while the video played at normal speed and the shape of the membrane was correctly reconstructed at approximately 7 hz . several sets of data points were taken from the reconstruction and plotted in matlab to check that the reconstruction that looked correct on the computer screen was in fact reasonable . examples of undeformed and deformed mesh plot is shown in fig4 a and 4b . the doppler aspect of the invention was tested in several ways to ensure that it was working properly . first , the probe was tapped and rubbed with a finger to determine if the transducers were actively sensing movements and hence shifts in the reflected acoustical wave . thereafter , the probe was dipped into a glass of water and it was able to pick up movements of particles suspended in the water . the amplified speaker produced loud rushing sounds even as the water was disturbed by gently blowing on its surface . finally , the probe was tested to evaluate its ability to detect blood flow in pulsing blood vessels . this testing was conducted by simply applying the probe to the radial artery at the wrist and the brachial artery at the crease of the elbow . because the skin on the outside of the body is not slick with water - based bodily fluids as surfaces found inside the body are during minimally invasive surgery , an aqueous gel was applied to the skin to prevent any air gaps between the tip of the probe and the skin . the result of this testing was a very strong and prominent pulsing sound reproduced by the amplified speaker 52 of the doppler system . the pulse of the blood vessels was clearly detectable when the probe was placed over the vessels and it stopped as soon as the probe was moved a few millimeters away from the vessels . these tests were conducted with the latex membrane 26 filled with distilled water rather than a preferred gel . the materials used in construction of the prototype probe were relatively expensive because they were of high quality and because some components such as the special &# 34 ; doughnut &# 34 ; cut transducers had to be specially ordered . nonetheless , the total material cost for one probe was approximately $ 250 , and construction time was approximately 6 hours . in production , the probe of the invention would be a small fraction of the cost of the endoscope which provides the visual images . modifications and variations of the invention will be apparent to those skilled in the art and all such modifications and variations are included within the scope of the appended claims .
surgical probe for use in minimally invasive surgery . the probe includes an elastic membrane tip portion which deforms upon contact with an object . the elastic membrane forms an enclosure containing fluid and an inside surface of the membrane includes a plurality of visible markings such as dots . a ccd camera forms an image of the dots and the image of the dots or a representation of the image of the dots is displayed on a video monitor . the probe also includes an ultrasound transducer which responds to frequency shifts to indicate motion of an object reflecting ultrasound energy . a speaker generates sounds related to the frequency shift as an indication of motion . the probe thus provides information about the surface of an object encountered during a minimally invasive procedure and also about motion of structures within the body .