an illustration of the method and apparatus of the invention is shown in the components of the apparatus and images derived therefrom in the figures . in the functional block diagram of fig1 is generally illustrated one embodiment of the invention . as shown in fig2 in one preferred embodiment an ultrasound imaging system 10 ( such as , a dornier urovision ultrasound imaging system ) is utilized employing a transrectal ultrasound probe 12 ( such as , a dornier model er - s ) operating at 5 . 6 , 7 . 5 , or 10 mhz . any conventional ultrasound probe can be used , provided the probe construction permits definition of at least two landmark points , or fiducials . the probe 12 ( best seen in fig1 , 8 and 9 ) includes a number of active transducer elements , such as ninety - six in the dornier er - s , with a display depth of 13 . 5 cm . display depth is a function of frequency . in order to precisely localize the position of the prostate for radiation therapy , the diagnostic ultrasound image is fused , or correlated , with the simulator x - ray image by using a matching technique employing at least two points . this methodology requires the unambiguous identification of two fiducial points on each image ( the ultrasound and x - ray images ). the transrectal ultrasound probe 12 in fig1 and 2 utilizes a curved phased array methodology for obtaining a fan shaped image 14 ( see fig2 ). arc 16 at the top of the ultrasound image in fig3 directly corresponds to the spatial position of a transducer array 18 inside the head of the probe 12 . this can be applied to planar as well as curved arrays . in the typical transrectal ultrasound image of the prostate in fig3 the end points ( also recited herein as “ fiducials ”) of the transducer arc are labeled “ 20 ” and “ 22 ” in the image . fig4 displays the lateral simulator film with the transrectal ultrasound probe 12 in place . the transducer array 18 of the probe 12 is visible in both the ultrasound and simulator images ( fig1 and 4 , respectively ). the ends of the transducer array 18 which correspond to the points on the ultrasound image are likewise labeled “ 20 ” and “ 22 .” by selectively enlarging sections of images , the points which correspond to the fiducial landmarks ( the end points 20 and 22 ) can be determined to within a single display pixel . once the fiducials have been identified in the image , a spatial transformation which maps the ultrasound image to the simulator image is determined . as shown in fig1 the electronic data from the probe 12 is output to an ultrasound imager 23 for processing through imaging processor 24 to computer 100 which executes selected computer software ( to be described hereinafter and / or set forth in appendix a ). the transformation consists of a two - dimensional rotation , translation and a global isotropic scaling of the appropriate data using computer software . this leads to a four parameter transformation , ( x s   i y s   i ) =  ( m x   x m x   y - m x   y m x   x )   ( x u   s y u   s ) + ( t x t y ) =  λ   ( cos   θ sin   θ - sin   θ cos   θ )   ( x u   s y u   s ) + ( t x t y ) ( 1 ) where the four parameters are the translations , t x and t y , the rotation angle , θ , and the global scaling , λ o . the vectors { right arrow over ( r )} us and { right arrow over ( r )} si represent the coordinates vector of any point in the ultrasound and simulator images respectively . if the two fiducial end points 20 and 22 on the ultrasound image are { right arrow over ( r )} us ( 1 ) and { right arrow over ( r )} us ( 2 ), and the two fiducial end points 20 ′ and 22 ′ on the simulator film are { right arrow over ( r )} si ( 1 ) and { right arrow over ( r )} si ( 2 ) then the matrix , m and translation { right arrow over ( t )}, which maps the ultrasound image to the simulator image is given by the solution to the linear system of equations given by , ( x u   s ( 1 ) y u   s ( 1 ) 1 0 - y u   s ( 1 ) x u   s ( 1 ) 0 1 x u   s ( 2 ) y u   s ( 2 ) 1 0 - y u   s ( 2 ) x u   s ( 2 ) 0 1 )   ( m x   x m x   y t x t y ) = ( x u   s ( 1 ) y u   s ( 1 ) x u   s ( 2 ) y u   s ( 2 ) ) ( 2 ) once the solution for m xx , m xy , t x and t y are known , every point in the ultrasound image can be mapped to the simulator image using equation ( 2 ). fig5 displays the result of transforming the ultrasound image to the simulator image . this will be referred to as the fusion image . fig3 displays the ultrasound image data along with other useful diagnostic information supplied by the imaging system 10 . however , only the ultrasound image data is transformed so that the details of the underlying radiographic image are not obscured . the accuracy of the rotational and translational component of the transformation is determined by the accuracy of localizing the two end points 20 and 22 on the ultrasound image and the end points 20 ′ and 22 ′ on the radiographic images . the end points 20 and 22 corresponding to the ultrasound image can be determined automatically since for a given setting of the field of view ( fov ) of the ultrasound imaging system 10 , the position of the end points 20 and 22 of the transducer array 18 always appear at the same points , or display pixels , in the digitized image . the automatic determination of the fiducials ( the end points 20 ′ and 22 ′) on the simulator image is more difficult since the probe 12 does not always appear exactly in the same position on the radiographic image as in the ultrasound image . to achieve automatically the locations of these end points , a conventional hough transform can be used to effectively and automatically locate the two points 20 ′ and 22 ′ on the simulator image , provided the outline of the transducer array 18 is used as the hough transform template . there will be two modes , manual and automatic , for the determination of the fiducials ( end points 20 ′ and 22 ′). for the manual mode , the image can be enlarged ( zoomed ) and the end points of the transducer array 18 can be determined to within a pixel ( the smallest image point ). therefore , the accuracy of the spatial position of the fusion is determined by the pixel size ( typically about 0 . 275 mm ) for the simulator image . the limit of the accuracy for automatic fiducial detection on the radiographic image using a hough transform is also determined by the pixel size . after the ultrasound image is transformed to the simulator image , anatomical volumes of interest can be manually contoured on the fusion image . for example , fig6 displays the fusion image where the prostate , bladder , and rectal volumes are contoured and color washed over the underlying image . a “ color wash ” is a graphics technique where a color image is “ washed ” over a gray scale image in such a way that the underlying gray scale image is still visible . if the pixels of the color image and the gray scale image are represented in hue - saturation - value ( hsv ) space , then the color washed image is obtained by setting the value to the value of the gray scale image and setting the hue to the hue of the color image . the saturation is set to unity . the resulting image looks like the original gray scale image but with transparent colors of the color image . the color washed representation of the volumes allow for the complete delineation of the volumes , while still allowing the structure of the underlying image , whether it be the ultrasound , treatment , or simulator image , to show through . in addition , the proposed radiation treatment field isocenter , width and height can be specified . fig7 displays the fusion image with the contoured volumes of interest and proposed radiation field 29 superimposed . in addition to the field outline , a block 30 , which appear as a white edge outlined square region , in the figure has also been drawn . the block 30 has been chosen to minimize rectal wall and bladder involvement in the field while ensuring that the tumor volume with a 0 . 5 to 0 . 75 centimeter margin receives 100 % of the dose . once the field 29 and the blocks 30 have been drawn , the percent area of prostate , bladder and rectum which is inside the radiation portal is automatically calculated and reported on the top of the fusion image . the area of the resultant radiation portal is also calculated and displayed . this quantitative information is available at the time of the simulation or treatment and may prove to be clinically valuable when assessing efficacy and morbidity . to aid in the development of an accurate treatment plan , options are available through use of the computer 100 ( see fig1 ) to implement ultrasound fusion software ( see attached appendix a ). the computer 100 interacts with the data from the probe 12 to produce the fusion image on a display 102 . a user can interact by interface 104 with the imaging system 10 to carry out the functionalities described herein . there are three basic modes of utilization : ( 1 ) the ultrasound mode , ( 2 ) the simulator mode , and ( 3 ) the fusion mode . all modes have the option of reading in new patient data , saving the present set of patient data , or restoring an old , previously saved set of patient data . all modes also have image enhancement functions such as windowing and leveling the image , edge detection , and enlargement or reduction of the image . treatment planning to define the radiation field size 29 , radiation blocks 30 and isocenter are available to each mode and include the ability to select and draw contours , activate a color wash of the contours and blocks , delete , add or delete points from a contour or block , measure absolute distance between points in the image specified by the mouse , and finally to fuse the images once all the point fiducials have been selected . there are additional functions of the computer software which are specific to each basic mode . in the ultrasound mode , there exist options to flip the image about the vertical axis , manual or automatic specification of the end point fiducials used in the fusion algorithm , and automatic or manual cropping of the ultrasound image . automatic cropping will crop the image using a special mask which will include only ultrasound image data for a specific mode of the imaging system 10 . the simulator mode has the options for manual or automatic specification of the end point fiducials used in the fusion algorithm . the automatic definition of the end points fiducials for the simulator image is based on a hough transform using the probe outline as a template . once the contours have been specified , the computer software automatically calculates the area of the contour in square centimeters . once the radiation field and the blocks 30 have been defined , the computer software can be used to determine the area of the contours which are directly involved in the radiation field . the size of the radiation field is also directly adjustable . the data necessary to carry out therapy on a new patient include digitized ultrasound image and a digitized simulator image . the file format of these images is comprised of a header and string . the header contains patient information ( name , age , etc .) followed by the image information ( dimensions , etc .). this header is followed by a string of bytes which comprise the total image . image compression is an option which can be selected by the user at the time the file is saved or restored . once a patient file has been saved , a new file is created which contains the details of the contours , blocks , radiation field , fiducials , and image information ( flipped status , cropped status , etc .). as shown in fig8 and 9 , the couch mount 32 functions to accurately and reproducibly position the ultrasound probe 12 relative to the radiation field isocenter . this results in the localization of the ultrasound imaging plane relative to the radiation field isocenter or beam delivery system . in addition , this particular design is intended to be simple , which aids in the positioning and insertion of the probe 12 into the patient . the isocenter is chosen to be the coordinate origin . the couch mount 32 consists of six basic parts : ( 1 ) probe holder 34 , ( 2 ) a first mechanism 36 for the angular positioning of the probe , ( 3 ) a second mechanism 38 for the vertical positioning of the probe 12 , ( 4 ) a third locking screw mechanism 40 for the fixing of position of the probe 12 along the holder 34 , ( 5 ) a machine 42 for rotational positioning of the probe 12 and , ( 6 ) a mounting plate 44 which mounts the holder 34 to the treatment table ( not shown ). the transrectal ultrasound probe 12 is mounted in the probe holder 34 . the design for the couch mount 32 has four degrees of freedom for the position of the probe 12 relative to the treatment table , and the standard radiation therapy treatment table has three degrees of freedom for the position of the couch mount 32 relative to the isocenter . the hardware specific geometry is displayed in fig9 . the four degrees of freedom for the couch mount 32 are referred to as l ,, φ , θ , while the degrees of freedom for the table are referred to as t height , t long , and t lat . due to the mathematical nature of the localization algorithm , three points on the ultrasound image must be localized relative to the coordinate origin ( isocenter ). these three points are arbitrary , insofar as the points must not be collinear , but rather they must define the imaging plane . these three points are referred to as a ( 1 ), a ( 2 ), and a ( 3 ). these three points are exactly determined by the specific values of l , , φ , θ , t height , t long , and t lat . the mode of the ultrasound imaging system also effects these three points since the pixel size of the image is adjustable . this variability is incorporated in the invention , however for the sake of simplicity , only one acquisition mode will be assumed . the positional information of the probe 12 in the four degrees of freedom are conveyed to the image localization software by any of four methods : manual readouts and key - in , digital readouts and manual key - in , digital encoding and automatic unit entry , and 3d ultrasound or optical ranging ( triangulation ) with automatic reading of spatial position with respect to the isocenter . with the latter scheme , 3d frameless system replaces the requirement for readouts for each degree of freedom of the probe 12 permitting greater positioning freedom and automatic movement compensation . the above design enables quantifying the ultrasound imaging plane relative to the isocenter . however , it is also necessary to quantify the radiation field relative to the isocenter . choosing the coordinate origin as the isocenter , any point in three dimensional space can be deemed and is described by the vector { right arrow over ( r )}. the radiation field is defined by the variables θ g , θ t , θ c , and sad , where θ g is the gantry angle , θ t is the couch mount 32 or table angle , θ c is the collimator angle , and sad is the distance from the radiation source to the isocenter . in addition , the field is further quantified by specifying the width and height , given by the symbols w and h . given these definitions , the points which lie within the radiation field are given by : - w · ( sad - z ′′′ ) / 2   sad ≤ x ′′′ ≤ w  ( sad - z ′′′ ) / 2   sad ( 3 ) - w · ( sad - z ′′′ ) / 2   sad ≤ y ′′′ ≤ h  ( sad - z ′′′ ) / 2   sad where , ( 4 ) ( x ′ y ′ z ′ ) =  ( cos   θ c sin   θ c 0 - sin   θ c cos   θ c 0 0 0 1 )   ( x y z ) = ( x   cos   θ c + y   sin   θ c - x   cos   θ c + y   cos   θ c z ) ( x ″ y ″ z ″ ) =  ( cos   θ g 0 sin   θ g 0 1 0 - sin   θ g 0 cos   θ g )   ( x ′ y ′ z ′ ) = ( x ′   cos   θ g + z ′   sin   θ g y ′ - x ′  sin   θ g + z ′   cos   θ g ) ( x ′′′ y ′′′ z ′′′ ) =  ( cos   θ t sin   θ t 0 - sin   θ t cos   θ t 0 0 0 1 )   ( x ″ y ″ z ″ ) = ( x ″   cos   θ t + y ″   sin   θ t - x ″  sin   θ t + y ″   cos   θ t z ″ ) ( 5 ) the radiation field can also be defined by the lines which go through the source point and any point on the field edge in the isocentric plane . the field edge can be mapped to the ultrasound imaging plane by determining the intersection of these lines with the plane of the ultrasound image . if the line is described by the two points { right arrow over ( r )} 1 and { right arrow over ( r )} 1 , and the ultrasound imaging plane is described by the above defined points a ( i ), then the point where this line intersects the imaging plane is given by : t =( d − a · x 1 − b · y 1 − c · z 1 ) / a · 1 + b · m + c · n ) ( 6 ) a =( y 2 − y 1 )( z 3 − z 1 ) −( z 2 − z 1 )( y 3 − y 1 ) ( 7 ) b =( z 2 − z 1 )( x 3 − x 1 ) −( x 2 − x 1 )( z 3 − z 1 ) ( 8 ) c =( x 2 − x 1 )( y 3 − y 1 ) −( y 2 − y 1 )( x 3 − x 1 ) ( 9 ) where t represents the parametric form of a line in three dimensions . the point of intersection is given by with these definitions , the cross sectional field outline as defined in the plane of the ultrasound image can be determined and graphically displayed on the real - time ultrasound image . these coordinates are obtained from the couch mount 32 , whereby the method of measurement of the coordinates are obtained by other manual or automatic means consisting of linear and angular vernier position readouts , digitally - encoded readouts , digitally - encoded data to computer , or automatic digital ultrasound triangulation of the four ultrasound probe coordinates . the ultrasound imaging system 10 is directly connected to the pc 50 ( see fig2 for example ) running the computer software of appendix a , so that the resultant displayed ultrasound image is in real time . while particular preferred embodiments of the invention have been shown and described , it will be obvious to those of skill in the art that changes and modifications can be made without departing from the spirit and scope of the invention as set forth in the claims .
a method and apparatus for localizing and treating a patient &# 39 ; s prostate by radiation therapy . the system includes an ultrasound probe for generating an ultrasound image of the prostate with the probe having at least two fiducial end points as landmarks . in addition , the system includes a ct apparatus to generate x - ray simulator images of the anatomical region near the prostate and further includes a computer executing computer software to transform the ultrasound image to the simulator image enabling accurate localization of the prostate relative to proximate anatomical features enabling accurate radiation therapy to be carried out on the prostate .