fam is defined as a collection of geometric models representing a patient &# 39 ; s anatomical structures including jaw bones , teeth , soft tissues and nerves . fig1 shows the components of a fam . the component for soft tissues is the main differentiator . the checkmarks indicate all the applicable components for various cases . for example , the fam of a “ fully edentulous lower jaw ” case will include the soft tissues , the bones and the nerves . with a fully edentulous case , there is no remaining tooth , so the components will be simply for the bones , soft tissues and nerves . for an upper jaw , there is no nerve model . the components of a fam are not necessarily separated geometric models . the actual embodiments can combine anatomical structures in one model or more . specifically , it is very common that the soft tissues and teeth are combined into one model , namely , “ tissue - tooth ” model ; the bone and teeth are combined as “ bone - tooth ” model . in fig2 , the basic embodiments of different cases are listed in a table . for example , “ partially edentulous upper jaw ” can have three components — a soft tissue model , a bone model and a tooth model , or two components — tissue - tooth and bone - tooth models . in the remainder of this document , the term “ tissue model ” will also indicate “ tissue - tooth model ”, and “ bone model ” will also indicate “ bone - tooth model ” unless otherwise stated . fig3 - 8 gives examples of bone models , bone - tooth models , soft tissue models , tooth models , nerve models , and tissue - tooth models . three models of bone or bone - tooth are shown in fig3 : a fully edentulous upper jaw , a partially edentulous lower jaw with ct scan scatters , and another one without scatters . in fig4 , two models are given . the first one is a soft tissue model of a fully edentulous upper jaw . the second one is that of a lower jaw . both of them are special embodiments of tissue models . it is important that the anatomy models in this invention are used for implant treatment planning purposes other than exact duplicates of the actual anatomy , therefore the tissue models are acceptable as long as they reflect the tissues in the tooth / implant areas . the models in fig4 are marked with non - anatomical areas resulted from the radiographic guides . such areas are not exactly the soft tissue ; they also reflect the space between the radiographic guide and the patient soft tissue or teeth . since a radiographic guide may not completely fit the patient &# 39 ; s anatomy , the gaps between them will be reflected by this tissue model . moreover , there are some clinical cases that the radiographic guides are not well placed onto the patients jaws when the patients are being scanned . the tissue models created by this approach will also reflect such gaps . this however does not affect the implant placement and restoration preview . as a matter of fact , such a “ tissue ” model help identify problems like this . in the second model , the soft tissue in the buccal side is missing , because the radiographic guide — the data source — does not extend to the soft tissue area at the buccal side . fig5 shows an assembly of a tissue model and a bone - tooth model . in fig6 a tooth model is illustrated , which does not have the tooth roots . fig7 is a nerve model shown with the jaw bone . fig8 gives two tissue - tooth models . the first one is a special embodiment created from ct scan of radiographic guide , and the second one from optical scan of a stone model or an impression . a fam is an assembly of these components . they can be represented as a single geometric model with multiple shells , each of which can be exported as an individual file . or , they can be a set of models . the data representation of the models can be of any format . usually , the triangulated model is used , which represents a model as a set of triangles . the stl file format is the standard format for it . the oral - dental anatomy modeling includes the approaches to acquire and create all of those components , as well as the approaches to represent the geometric data and to assemble the models together . fig9 lists all the possible data sources of the anatomy models , and the techniques to create the components . the patient ct scans are usually the data source for bones , teeth , nerves , as well as bone - tooth combinations . the term “ ct scan ” is used in this disclosure as a general term for all scan technologies that acquired 3d images as ct scan does , such as cone - beam ct ( cbct ), ultra - sound , etc . the ct scans of radiographic guide ( or scan template , scan prosthesis ) are the data source of teeth , soft tissues , and tissue - tooth combinations as well . the optical scans of radiographic guides or plaster / impression models can be used to create tissue and tooth models . so can the patient intra - oral scans . usually a bone model is created by contouring the ct scan of a patient . there is a need to remove x - ray scatters as shown in fig3 . x - ray scatters a the areas of high density , such as tooth or metal restorations . even though image processing techniques might be used to automate , at least partially , this process , the prevailing approach is just to let the software users manually circle some scatter areas of the bone model created by ct data thresholding , and then to remove the pixels within the areas . nerve models for a lower jaw are manually drawn . the users specify points on 2d ct slices , connect them with a spline , and then the software creates a tube model using it as the center . what actually makes difference for anatomy modeling is the inclusion of tissue models and the methods to acquire and create them . the technique to create a soft tissue model from the ct or optical scan is called “ virtual stone model ” in this invention , which is introduced in the next section . this approach has a special step to separate soft tissues from the teeth . a simpler option will be just to create tissue - tooth models as shown in fig8 in order to avoid such a step . because the data source of a soft tissue model is different from the ct scan of a patient , they are in different locations when put into same coordinate system . the soft tissue model needs to be registered with the bone model . the actual process can be either to register the source datasets first , or to register the models after they are created from the datasets . if a radiographic guide ct scan is used to create the soft tissue model , the guide model is registered with the bone first , and then the “ virtual stone model ” approach is used to create soft tissue model . the registration between the ct scan of the patient and that of the radiographic guide is based on the markers on the radiographic guide . fig1 illustrates this process . the thresholding of patient scan will output both the bones and the markers since the patient is scanned wearing the radiographic guide . the bone model and the radiographic guide are not aligned at the first place . in the middle of fig1 , the markers with the radiographic guide scan and the patient scan are shown in different windows . the figure shows a status that the makers have not been all recognized or specified yet . some more user interactions are needed to specify the markers in this case , because the simple thresholding technique cannot fully separate the markers from the bone structure or existing tooth restorations . image processing can be of course employed to automatically detect the markers . landmark based registration is applied once the markers are identified , and this finally puts the two models in the right locations as shown in the last picture of fig1 . iterative closest point ( icp ) is the most common method to register landmarks . another scenario is the registration between two surface models , specifically , two triangulated models . fig1 illustrates the surface model registration . approaches are needed to identify the corresponding points on the two models . improved icp can be used for this purpose ( zhengyou zhang , “ iterative point matching for registration of free - form curves and surfaces source ”). some implementations can be found in public domains such as the one in visualization toolkit ( www . vtk . org ). the possibility to perform registration determines what can be used to create the soft tissue model of a fam . for a fully edentulous case , an optical scan cannot be used for this purpose unless a radiographic guide model also exists , because for such a case the soft tissue model from an optical scan is completely different from the bone model , and there is nothing like a marker or an overlapping area to help align them . if the radiographic guide exists , one can register the guide with the patient scan by aligning the markers , and then to register the tissue to the guide as two surface models . the reason to have such a special embodiment is that optical scan can be used to create a more accurate surgical guide . if only the radiographic guide ct scan is used , its contouring surface can have different geometry due to the different threshold values used . it is unfortunately true that a contour surface from ct scan can barely reflect the actual geometry of a model because the thresholding is very subjective ( gao ser . no . 12 / 776 , 544 ). having an optical scan in the meantime will certainly eliminate this problem . for partially edentulous cases , the preferred embodiment is to create a bone - tooth model from the patient ct scan , and a tissue - tooth model from an optical scan , then to use the surface registration to put them together . assuming the surface model of the radiographic guide is already obtained by either ct data contouring or by an optical scan , the inward surface of the guide is treated as the surface of the soft tissues . a 3d model representing the soft tissues surrounding the bone structure is created , and combined with the bone model to form a “ virtual stone model ”. fig1 shows how the virtual stone model is created and used in treatment planning . the steps are listed below . 1 . the radiographic guide is registered with the bone structure . 2 . the surface model of the guide is created by contouring the ct scan of the radiographic guide , or any other imaging technique . 3 . a model view direction is defined . it is an imaginary extraction / insertion direction , along which the radiographic guide can be put into or pulled out of the patient &# 39 ; s mouth . this is shown in fig1 . 4 . next , the guide model is split into two portions as shown in fig1 . a view direction is defined as the opposite of the direction . all faces that are “ visible ” along this view direction are considered part of the tissue / tooth surface . for treatment planning purposes , this surface will serve as the tissue model , even though this model is more or less a “ wrapper ” of the tissue and tooth surfaces . 5 . along the defined view direction , some triangles of the inner surface may not be fully visible . they are called undercuts , which will leave holes on the model when the inner surface is extracted . the algorithm to fill the holes is a common technique in polygonal modeling . an example can be found in “ a piecewise hole filling algorithm in reverse engineering ” by y jun . the result is illustrated in fig1 . 6 . for fully edentulous cases , go to step 9 . for partially edentulous , output the resulted surface model as tissue - tooth model . 7 . a distance map between this model and the bone - tooth model is then evaluated . in a triangulated surface model , a vertex is represented as its coordinate and a scalar value . in order to represent the distance map , the scalar value is set to the distance between an extracted model vertex and the bone - tooth model . 8 . the tooth areas are identified through filtering the distance map . if a vertex is on the tooth area , its distance value should be close to zero within a tolerance . all the vertices and their triangles together can then be removed from the model . 9 . output the resulted model as the surface model of the soft tissues . 10 . this tissue model is then offset or extruded toward the bone structure . 11 . the result is then trimmed by the bone model . 12 . output this as 3d virtual soft tissue model . as shown in fig4 , the resulted solid model is actually the volume between the radiographic guide and the bone structure . when the guide is made with good accuracy and fits well with the patient anatomy , this model can well represent the patient &# 39 ; s soft tissues . together this tissue model and the patient bone structure including the remaining teeth make the so - called “ virtual stone model ”, a variation of fam as shown in fig5 . when the tissue - tooth model is from optical scan , the same procedure is applied to create the virtual stone model . the procedure is actually simpler . after the registration , the distance map is evaluated , and the rest is same as step 7 and after . the virtual stone model differentiates itself from the conventional plaster model or its optical scan because the anatomical components are separately represented , and they can be also manufactured with rapid prototyping or 3d printing . in an implant treatment planning system based on the anatomy modeling , the fam is in the center of the workflow . the system has the following major logical components . the anatomy modeling module takes ct and optical scans as inputs , and creates a fam with various modeling tools . the fam management and visualization modules manage all the models and their display . the implant placement module places implants with references to the fam . the restoration preview module performs aesthetics evaluations by previewing the implants and adding virtual teeth and the like . the surgical guide design module creates guides based on fam . fam serves as three roles . the first is called placement reference . the software system lets the users to place implants onto this model . in the prior art the implants are not placed with references to such a model reflecting the actual patient anatomy with soft tissues and remaining teeth . instead , the 2d slices of ct scan or 3d bone models are used for this purpose . fig1 illustrates how an implant is placed with a fam as reference . in 3d graphics window the components of the fam are displayed . each of them can be set visible or invisible . in the meantime , there are 2d views , which display the ct scan of the patient , and the cross sections of the fam components . when a panoramic view is desired , the 2d views will also display the reformatted cross sections defined by the specified arch curve . the implant placement is performed on all the 2d and 3d views . the implants are displayed as 3d models in 3d windows and cross sections in 2d . each view of an implant will have a placement widget attached . there are both 2d and 3d versions of the placement widgets . the users can use any placement widget in one view to adjust an implant &# 39 ; s position , and all the other views will be updated accordingly . there are also tools to change the display properties of individual components , such as opacity , color , lighting , etc . a model navigator accompanying the graphics views uses a tree structure to show the fam components and other objects created , and to provide individual controls to the models ( gao , us patent application ser . no . 12 / 795 , 045 ). secondly , fam serves as an aaesthetics reference , the reference to preview implants and restorations . abutments , virtual teeth , etc . are added to fam so that the users can better preview a case with the soft tissues simulated . fig1 shows a case that a virtual tooth is being added to the fam . its size and location are being adjusted . the essential difference between this process and existing approach is that the users get instant feedback or preview on how the implants and virtual teeth look like since the soft tissues are shown in the scene . this function to perform restoration preview with soft tissues is one of the goals of this invention . without fam , the software systems cannot provide an appropriate 3d preview tool to assess the aaesthetics of a treatment plan . using a radiographic guide model for preview , as in some implementations , is much more problematic , because it is made by offsetting the patients &# 39 ; anatomy . thirdly , fam is the base for designing and making surgical guides , and thus called the surgical guide reference . with prior art , there are two methods to create surgical guides . when a radiographic guide model is available , a surgical guide is created from this model by adding implant holes and additional form features to it . when there is no radiographic guide , i . e ., a bone level surgical guide is to be designed , and bone model or bone - tooth model is used as the base for surgical guide design . the areas surrounding the implants are clipped , offset and then modified with form features . fam , especially with soft tissue models from optical scans or bone level ( without tissue model ) cases , can lead to different surgical guide design and manufacturing methods in addition to the afore - mentioned options . in one specific embodiment , the virtual stone model is used as a base to design a surgical guide . the components of the virtual stone model are united together as a single solid model first , then the area surrounding the implants is clipped and extracted . this area is then offset by a given thickness . form features like implant holes , irrigation windows , fillets , etc . are added to the model . further model modifications can be also done using a cad system until the surgical guide model is as desired . this is a generic approach that works for all scenarios . fig1 illustrates a simple bone - borne surgical guide case . the implant areas are marked in the first picture , the extracted area is offset as shown in the second picture and a surgical guide is created afterwards . in fig1 , the tissue model and virtual stone model are also modified with implant holes . the surgical guide is automatically generated and shown in the third picture . alternatively , it is possible to create a “ master model ” using fam , with which conventional manufacturing method can be used to make a surgical guide . a simple embodiment of master model includes a virtual stone model with implant holes on it , and implant inserts that can be placed into these implant holes . the fam also enables the combination of digital treatment planning and physical model based planning . this is disclosed in “ a hybrid method for dental implant treatment planning ” ( gao , us application ser . no . 12 / 860 , 019 ). the logical components and the hierarchy of the treatment planning system based on fam are shown in fig1 . starting from the bottom of the figure is the system core level including the components of image data file i / o , image processing , visualization and registration . the center layer is fam creation and management . an optional fam navigator gives users an easy way to access and manage fam components . it can also control the display of the fam . in the anatomy modeling modules , fam components and virtual stone models are created . in the treatment planning layer are implant placement , restoration preview , and surgical guide design . a treatment planning system is considered based on anatomy modeling or fam if one of the following is true : a soft tissue model or virtual stone model is created before the treatment planning starts . the implant placement uses a soft tissue model as reference in addition to bone and nerve models . the restoration preview is also based on fam , essentially , on a tissue model . the models of implants , abutments and crowns are all placed onto a fam so that the preview is performed with a tissue model presenting . the system creates either surgical guide models or master models from fam .
this invention introduces an oral - dental anatomy modeling method and an implant treatment planning system based on a full anatomy model . dental implant treatment planning systems place implants on 2d slices of dicom files , and sometimes on 3d models of bones and remaining teeth . because the final look and feel of implants and restorations depend on how well they go along with the remaining teeth and soft tissues , treatment planning without tissue model presents safety and aesthetics risks . a fam consists of models for bones , teeth , soft tissues and nerves . they are created from ct and optical scans , and assembled together with model registration techniques . the tissue model is the real differentiator . a treatment planning system uses fam as a unique reference throughout the workflow . its implant placement , restoration preview and surgical guide design are all based on fam .