TECHNICAL FIELD

The present invention relates to a computer implemented method for modifying a digital three-dimensional model of a dentition comprising: displaying an image of the dentition on a display; receiving user input information modifying the shape of at least a portion of the dentition by markings input on the image on the display; and transferring the shape modifying information to the three-dimensional model and modifying the three dimensional model accordingly.

The three-dimensional model of the dentition has usually been obtained by either using a library of dentitions or by scanning and/or phototechnical acquisition of the oral cavity of a person, or by scanning the shape of the dentition taken as impressions in casting compound material in impression trays.

The invention may be used in a dental Augmented Reality application to preview a dental situation, which is the result of any modification of the dentition e.g., after a planned dental treatment, with teeth position correction devices in place or including any other modification of the dentition. The modified state of the dentition of the patient (e.g. after dental treatment) is referred to as the dental situation in the present application. The dental treatment can be planned using computer-implemented dental treatment design tools starting from the three-dimensional model of the dentition and creating a modified three-dimensional model of a dental situation after the treatment. Another option is to create a physical model of the dentition and to modify it by any dental alteration to obtain a physical model of the planned dental situation which is then scanned. The planned dental situation may include one or more new dental prostheses or other dental restorations, or a corrected teeth arrangement as a result of corrections of teeth positions, for example by use of dental braces.

For dentists, dental technicians and patients, it is of interest to get a visual impression of the appearance of a modified dental situation, i.e. to visualize the modified dental situation in an image of the face of the patient. For this purpose, a virtual preview (virtual mock-up) of the dentition modified by dental treatment is helpful for the dentist and may also be used in the course of interactively modifying the dental situation to get the most favorable aesthetic results.

BACKGROUND

U.S. Pat. No. 9,775,491 B2, which is hereby incorporated by reference in its entirety, discloses a computer implemented method for visualizing an image rendered from a three-dimensional model of a dental situation to an image of the face of the patient recorded by a camera. In this method a three-dimensional model of the oral cavity of the patient is obtained. This three-dimensional model is modified in a dentistry treatment plan by applying dental restorations to obtain a three-dimensional model of the dental situation of the patient dentition after application of the dental restorations. A two-dimensional image of the face of the patient including the mouth opening is obtained. Then the positioning of the camera that recorded the image relative to the dentition of the patient is estimated. In this application “positioning of the camera” meant to include the three-dimensional position x, y, z in space and the angular orientation of the camera with respect to the face of the patient. A virtual camera using the estimated positioning is processing the three-dimensional model of the dental situation to obtain a two-dimensional image, and a portion of the three-dimensional model of the dental situation is selected which is visible to the virtual camera. The image rendered by the virtual camera is overlaid and displayed in the image taken by the camera.

Similar methods are set forth in US 2011212420 directed to a method of forming a dental mold, US 20130060532 directed to a method of composing and designing teeth, US 2013158958 directed to a dental analysis method and system and US 20170319293 directed to a method of digitally designing a modified dental setup, all of which are hereby incorporated by reference in their entirety.

U.S. Pat. No. 7,717,708 B2, which is hereby incorporated by reference in its entirety, describes computer-based methods for supporting dentists during dental treatment planning for a patient. In such a method various sources of information on the physiological and anatomical state of the dentition of the patient are stored in the computing device. This includes for example a three-dimensional model of the dentition including the upper and lower teeth arches and gingiva, as well as two-dimensional images of the patient recorded by a camera, including the mouth opening and the visible parts of the dentition therein. The photo of the mouth opening may be displayed on a display of the computing device, and the dental practitioner may use software tools for marking certain features in the displayed photo, for example the occlusal plane. Furthermore, it is mentioned that software tools are provided for simulating changes in the anatomical position or shape of the craniofacial anatomical structures and their effect on the external visual appearance of the patient. In this connection it is also mentioned that any changes or modifications applied to the displayed photo of the dentition are automatically transferred to the three-dimensional model of the dentition which is modified accordingly. For marking certain features and lines on the displayed photo it is described that the user identifies points and lines with a user interface such as a mouse on the display. For marking a line or curve in the image of the dentition it is therefore necessary to input a plurality of points along the curve using the mouth. One important aspect when planning restorations for a part of or a complete dental arch is the positioning of the visible teeth arch contour curves, namely the incisal curve and the gingiva border curve of the upper and lower teeth arches, in the image of the mouth opening of the patient. Point by point input of such curves in the path of the upper or lower teeth arch to receive dental restorations is a slow and inefficient procedure.

In particular, when designing restorations for the upper or lower teeth arch the positioning of the visible teeth arch contour lines in the mouth opening is an important aspect. In this respect it would be particularly useful to get an immediate visualization of a modified incisal curve or a modified gingiva teeth border curve.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a method for modifying a digital three-dimensional model of a dentition of a person. The method allows for a quick intuitive input of a modification of the positioning and shape of a visible teeth arch contour curve of the upper or lower teeth arches.

These objects are achieved by the method comprising the features of the claims. Preferred embodiments of the invention are set out in the dependent claims.

According to the present invention the user can modify one of the visible teeth arch contour curves (incisal curve or gingiva border curve) by drawing or sketching a new line on the image of the dentition on the display, wherein the line drawn on the display determines up to where the new teeth arch contour curve should extend. It should be noted that the line drawn on the display is not shown on the display but only serves input information. The user can draw the line with his finger by sliding his finger tip along the new curve, or use an input tool such as pen to draw the new line on the display; again it should be noted that there is no physical line remaining or displayed in any way on the display. The display can be a touch screen which acquires the line drawn; alternatively the input tool can be a mouse, stylus or the like which can be used to draw a line by moving a cursor controlled by the mouse along the display or drawing a line with the stylus. In other words, along the line the teeth are stretched/compressed in the image in length direction (between tooth root and incisal edge) in the region of the line such that the teeth extend between the projected line and the dental arch contour curve opposite to the dental arch curve which is modified by the line drawn. This modification of the dental arch is done by deforming the three-dimensional model of the dentition in accordance with the line drawn.

In order to allow such design modifications by drawing virtual lines on the display displaying the dentition the following steps are performed. The image of the dentition is obtained by applying a virtual camera to the three-dimensional model of the dentition to render a two-dimensional image which is displayed on the display. The line drawn by the user on the image on the display is acquired and further processed. For this purpose the same virtual camera (i.e. in the same camera positioning) that was used to render the two-dimensional image of the dentition is used to project the acquired line to a three-dimensional projection surface which is approximating the shape of the upper and lower dental arches in the three-dimensional model of the dentition and which extends the dental arches into the adjoining gingiva regions and in the direction towards the other teeth arch to cover any opening between the upper and lower dental arches.

In the three-dimensional model of the dentition visible teeth arch contour curves of the upper and lower teeth arches are obtained, namely the incisal curve and the gingiva border curve for each of the upper and lower teeth arches. The teeth arch contour curves may be obtained by deriving them from the three-dimensional model of the dentition by automatic line detection in the three-dimensional model, or can be predefined by a user in advance. The teeth arch contour curves are then projected to the projection surface. One way of projection is to represent the curves as line segments where each point is projected by finding the closest point on the projection surface, respectively. Connecting these projected points results in the projected curves.

One of the teeth arch contour curves is selected as the one to be modified. This is done either based on a user selection in advance or can be done for example by selecting the one of the teeth arch contour curves which is closest to the projected line on the projection surface. The distance between two curves can for example be defined by calculating the integral over the projection surface area between the two curves.

All teeth along the projected line which belong to the selected teeth arch contour curve are deformed in the three-dimensional model of the dentition in a length direction of the teeth such that at least parts of the selected teeth arch contour curve, after deformation and projection to the projection surface, coincide with the projected line and any remaining parts of the selected teeth arch contour curve are located between the projected line and the baseline curve, whereas teeth regions on and beyond the baseline curve remain unaffected by the deformation.

That means that the acquired line projected to the projection surface becomes an enveloping or maximum line of the modified teeth arch contour curve, and, when the teeth arch contour curve to be modified is an incisal curve, the teeth in the affected area of the teeth arch are deformed by stretching or compressing them (stretching when the line drawn is further away from the gingiva border curve, and compressing when the line drawn for the new incisal curve is closer to the gingiva border curve than the current incisal curve). In this manner the teeth along the projected line are elongated or compressed in a manner defined by the line drawn on the image of the dentition on the display.

In the same manner the gingiva border curve can be redesigned by drawing a corresponding line, wherein in this case the opposite incisal curve of the same teeth arch is the baseline curve, and the teeth are elongated or compressed along the line drawn such that the gingiva border curve of the deformed teeth contacts the line drawn, whereas the opposite incisal curve remains unaffected.

As regards the deformation of the three-dimensional model of the dentition this includes methods in which a deformed three-dimensional model is generated and the original (undeformed) three-dimensional model is kept and remains stored.

The deformation of the three-dimensional model of the dentition can for example be carried out by applying scaling factors multiplying all coordinate values in the direction of deformation of all points of the three-dimensional model. For example, in case a polygon mesh is used for the three-dimensional model all vertices of the polygon mesh can be scaled using a scaling factor for the coordinate to be deformed, wherein the origin of the coordinate to be deformed is defined by the baseline curve such that no deformation takes place at the baseline curve itself, and the deformation in absolute terms is more pronounced the further the coordinate point is spaced apart from the zero line (baseline curve).

In a preferred embodiment the baseline curve is derived from the other teeth arch contour curve of the same teeth arch as the teeth contour curve to be modified by smoothing the other teeth arch contour curve. Smoothing of curves can be achieved as described above. Another possibility is to define a neighborhood range and assign an average or a maximum value within the neighborhood range.

In a preferred embodiment the deformation along the projected line is completed, and then an image of the deformed three-dimensional model is rendered by applying the virtual camera to the deformed three-dimensional model and the rendered image is displayed on the display instead of the image of the original undeformed three-dimensional model (the image thereof). In this manner the user can immediately get an impression of the visual appearance of the modified design of the dentition.

Alternatively new images are rendered continuously in real-time while the line is drawn on the display and the rendered images of the deformed portion of the selected teeth arch contour curve are displayed in real-time up to the current point to which the line has been drawn. In this case the image of the deformed teeth arch portion is following the line while this line is continued to be drawn by the user on the display. For example, the user can first draw a first line portion for defining a first part of a modified teeth arch contour curve, and the resulting modified portion of the teeth arch is already displayed by rendering the deformed three-dimensional model, and then the user can continue to draw the line to a further portion of the teeth arch to complete the design of the selected teeth arch contour curve. In principle, when the user first draws a short line portion this may have the effect that only a part of the teeth contour curve may be deformed by drawing a line only along a portion of a single tooth which then would already have the effect that only the incisal curve or gingiva border curve of this part of the tooth would be modified and the modified design be displayed on the display.

In a preferred embodiment the user may select the teeth arch contour curve to be modified by a line to be drawn in advance by explicitly identifying a particular teeth arch contour curve, or the teeth arch contour curve to be modified is automatically selected by selecting the teeth arch contour curve that is closest to the projected line when no other selection is present.

In a preferred embodiment the projected line is formed from the acquired line by using a sequence of points along the acquired line, and for each point by projecting, using the virtual camera, a straight line that intersects the point on the acquired line to the projection surface and by determining the intersection point of the straight line with the projection surface, wherein the projected line is formed by connecting the sequence of intersection points on the projection surface. In other words the acquired line is described by a polyline, and points along the polyline are projected by the virtual camera projecting a straight line through each point of the polyline on the display by the virtual camera to the projection surface to determine its position relative to the teeth arches. The sequence of intersection points with the projection surface formed in this manner describes the projected line as a polyline on the projection surface. The projection surface is needed in particular in the situation when the projected line would not be intersecting with any part of the dentition, e.g. the projected line is extending in an open area between the incisal curves of the upper and lower teeth arches.

In a preferred embodiment a vertical direction is defined for the three-dimensional model of the dentition by a direction perpendicular to an occlusal plane of the three-dimensional model of the dentition. The projection surface is then defined by a surface of a sector of a right cylinder or right prism around an axis parallel to the vertical direction, i.e. the longitudinal axis perpendicular to the end faces of the right cylinder or right prism is oriented in vertical direction. The projection surface is subdivided into a plurality of subsectors, each subsector extending over an incremental angular step with respect to the longitudinal axis of the cylinder or prism sector. For each subsector at least one point of the projected line is determined by projection or interpolation of projected points from neighboring sectors. For each subsector a corresponding dentition model subsector is determined by determining all coordinate points in the three-dimensional model contained in the subsector of the right cylinder or right prism sector. The deformation is performed for all coordinate points within the dentition model sector by applying a scaling factor scaling all coordinate points within the subsector in vertical direction, wherein the scaling factor is determined for each subsector separately. In this manner the projection surface and the scaling of the three-dimensional model of the dentition is discretized and performed in incremental subsectors along the projected line on the projection surface separately.

In a preferred embodiment the deformation is performed in the following manner. In each subsector on the projection surface a first point on the projected line and a second point vertically aligned with the first point are selected, which second point is the intersection point of a vertical line extending from the first point with the selected teeth arch contour curve projected to the projection surface. First and second vertical distances between the baseline curve of the three-dimensional model of the dentition projected to the projection surface and the first and the second point, respectively, are determined. The ratio between the first and second distance is calculated as the scaling factor for the subsector. If, for example, the point of the projected line has a vertical distance to the baseline curve of 11.0 mm, and the projected incisal curve of the selected teeth arch has a distance of 10.0 mm to the baseline curve, then the scaling factor for this subsector is 1.1.

For example, the first point of the subsector can be the one which has the maximal vertical distance of the projected line to the baseline curve in the subsector.

In particular, the deformation in each subsector can be performed using the scaling factor in the following manner. All coordinate points of the three-dimensional model of the dentition are multiplied by the scaling factor of the subsector by multiplying a z-coordinate, being defined in the vertical direction, of each coordinate point within the subsector with the scaling factor of the subsector, wherein the average value of the z-coordinate of the baseline curve defines the value zero of the z-coordinate in the subsector and only coordinate points with z>0 are scaled by multiplication with the scaling factor. In this manner coordinate points with z-coordinates closed to the baseline curve will be stretched/compressed by a small absolute amount only since the z-coordinate value itself is small, whereas coordinate points closed to the selected teeth arch curve will undergo a larger absolute deformation bringing them to or closed to the projected line.

In a preferred embodiment the selected teeth arch contour curve is smoothed by damping rapid changes along the teeth arch contour curve. Smoothing of the curves can for example be achieved by applying a filter, for example a low pass filter for damping rapid changes along the curve, or by fitting parameters included in a predetermined function so that the fitted function approximates the selected teeth arch contour curve by a smooth curve. In a preferred embodiment the baseline is derived from the opposite teeth arch contour curve of the same teeth arch by smoothing the curve to obtain the baseline curve.

In a preferred embodiment the selected teeth arch contour curve projected to the projection surface is used to select a teeth arch contour point for each subsector, for example a central point in the subsector. The teeth arch contour curve is then smoothed by defining in each subsector the teeth arch contour point by selecting it from a range of a predetermined number of neighboring subsectors as the teeth arch contour point having the largest vertical distance to the baseline curve. This smoothing by selecting the maximal distance point from a neighborhood of for example two or three neighboring subsectors is particularly useful for the incisal curve because in this manner the valleys or recesses of the incisal curve in the regions of adjoining neighboring teeth is at least partly preserved, i.e. also after deformation, for example by stretching teeth to elongate them there are still valleys in the resulting incisal curve in the adjoining region of a pair of neighboring teeth.

In a preferred embodiment, if the selected teeth arch contour curve that is to be modified is a gingiva border curve, the adjoining gingiva border region in the three-dimensional model of the dentition is deformed locally to adapt it to the deformation of the teeth determined by the projected line, wherein local deformation in vertical direction is performed in a manner such that movement of the coordinate points in vertical direction due to the deformation becomes smaller the further the coordinate points are spaced apart from the unmodified gingiva border curve.

In a preferred embodiment, in each subsector of the projection surface a first point of the projected line and second point vertically aligned with the first point are selected which second point is the intersection point of a vertical line extending from the first point with the smoothed teeth arch contour curve, first and second vertical distances between the baseline curve and the first point and the second point, respectively, are determined, and the ratio between the first and second distance is calculated as the scaling factor for the subsector.

Instead of the local deformation by a scaling factor a translation of the coordinates in vertical direction can be carried out, wherein a translation equal to the vertical distance between the old and the new curve is selected and is weighed with a weight factor which is close to 1 for coordinates close to the old curve and which is 0 close to the baseline curve.

