Dipole parameter estimation method and apparatus
BACKGROUND OF THE INVENTION 
     This invention relates to a dipole parameter estimation apparatus, which receives as input an electromagnetic field distribution generated on a living body or an object due to electric activity in the living body or the object and assumes the presence of one or more dipoles as sources of the electromagnetic field in the living body or the object to estimate the number, positions or moments of the assumed dipoles. 
     Heretofore, there has been the following method of estimating positions and moments of dipoles in the living body or the object from an electromagnetic field distribution generated on the living body or the object. 
     First, a dipole or dipoles are assumed in the living body or the object, and an electromagnetic field distribution generated by the dipole or dipoles at point of measurement set on the surface of the living body or the object is calculated. As an error function between the calculated value φ i .sup.(c) of the electromagnetic field distribution generated at an i-th measurement point, obtained as a result of calculation, and the measured value φ i .sup.(m) of the electromagnetic field distribution, for the squared-error r is calculated as: ##EQU1## If r is greater than a predetermined reference value, the positions and moments of the dipoles are corrected by using an optimizing method based on numerical analysis, typically a simplex method, so as to reduce the value of r. If positions and moments of the dipoles that correspond to an r value smaller than the reference value are obtained, they are made to be the estimated values of the positions and moments of the dipoles. This method is detailed in Bin He et al., Electric Dipole Tracing in the Brain by Means of the Boundary Element Method and Its Accuracy, IEEE Transactions on Biomedical Engineering, Vol. BME-34, No. 6, June 1987, hereinafter referred to as &#34;Literature 1&#34;. 
     In the above method, however, the method of setting the initial parameters such as the number, positions and moments of dipoles at the start of the calculation, is ambiguous, and depending on the initial values erroneous estimation of dipole positions result. In addition, in the above method it is necessary to repeatedly execute calculation many times until obtaining the final dipole positions and moments. Nevertheless, when the shape of the living body is taken into consideration, the calculation of the electromagnetic field distribution is complicated. Therefore, enormous time was required in obtaining the estimated values. Further, with a plurality of dipoles, the estimation requires double the time in the case of a single dipole, and also the error function has the increased number of minimum points. Therefore, in many cases the result of estimation was erroneous. 
     SUMMARY OF THE INVENTION 
     An object of the present invention is therefore to provide a dipole estimation apparatus, which is capable of high speed, highly accuracy dipole estimation by taking the shape of the living body or object into consideration even in cases where there are a plurality of dipoles that are assumed. 
     According to one aspect of the present invention, a trained neural network is used for estimating the number, positions or moments of one or more dipoles which are assumed as sources of the electromagnetic field distribution based upon an electromagnetic field distribution of a living body or an object. Hereinafter, either the dipole number, position, or moment, or any combination of these will be referred to as dipole parameters. 
     According to another aspect of the present invention, the neural network is preliminarily given training data describing the relationship between electromagnetic field distribution and dipole parameters of the dipole or dipoles as the source or sources of the electromagnetic field distribution. Hereafter, this process will be referred to as &#34;learning&#34; which means making corrections of weights of the neural network such that the network can correctly output, in response to the input of an electromagnetic field distribution, dipole parameters of the dipole or dipoles as the source or sources of that electromagnetic field distribution. Once the learning has been completed, its result is reflected on the weights of the neural network, and the calculations that are executed in the actual estimation are only forward calculations for obtaining the output of the neural network. Thus, according to the present invention, the estimation of dipole parameters can be done at high speed even where there are a plurality of dipoles that are assumed. Further, by using training data containing noise for the training by the neural network, dipole parameter estimation which is robust against noise is made possible. Further the dipole parameters that are estimated with the neural network may be combined as the initial parameters with the simplex method or similar optimization method based on numerical analysis. Doing so permits more accurate estimation of the dipole parameters by taking the outer shape of the living body into consideration. Using the estimation value obtained in the neural network the initial parameters, a value close to the true dipole parameters has already been obtained. Thus, calculation may be repeatedly executed a reduced number of times until obtaining the true dipole parameters. High speed dipole parameter estimation is thus possible. 
     Other objects and features will be clarified from the following description with reference to attached drawings. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a flow chart for explaining a dipole parameter estimation method embodying the first embodiment of the present invention; 
     FIG. 2 is a block diagram showing an apparatus embodying the second embodiment of the present invention; 
     FIG. 3 shows a neural network in the embodiment; 
     FIG. 4 shows a neural network having recurrent neural network structure; 
     FIGS. 5 to 7 show examples of providing target output signal data; and 
     FIG. 8 shows an example of the representation of the position of the dipole and the moment components. 
    
    
     PREFERRED EMBODIMENTS OF THE INVENTION 
     FIG. 1 is a flow chart for explaining a dipole parameter estimation method embodying the first invention. This embodiment is hereinafter referred to as first embodiment. In the following description it is assumed that the human brain has a single dipole produced therein and that it is intended to estimate, from the electromagnetic field distribution on the scalp, the number of dipoles, three position coordinate components of the dipole and three moment components of the dipole. Further, hereinafter the number, three position coordinate components and three moment components of dipole are collectively referred to as dipole parameters. 
     In step 1, input training data that was used for the learning by the neural network and measurement value of the electromagnetic field distribution on the scalp. The training data comprises the dipole parameters and the calculated value of the electromagnetic field distribution at each measurement point on the scalp when the dipole is produced in the brain. The electromagnetic field distribution is calculated from the dipole parameters as follows. In the following description, a head model is assumed, which is a conductive sphere with radius R and having a homogeneous conductivity. A method of calculating the potential generated on the surface of the sphere will be described. A coordinate is taken with the coordinate origin at the center of the sphere. When a dipole with its moment components (M t , O, M r ) is present at distance r from the coordinate origin of the z axis, the potential (R, θ, ψ) on the spherical surface at a specified point (R, θ, ψ) thereof is given as: ##EQU2## where σ is the conductivity of the sphere, Y n α m  (θ,ψ) is a spherical harmonic function defined as: ##EQU3## P n   m  (X) is the Legendre&#39;s associated function, and δ ij  is a Kronecker&#39;s symbol defined as: ##EQU4## The formula (2) represents the potential in cases when the dipole is on the z axis with zero on y axis component of moment. With a dipole at any position, the potential can be obtained by formula (2) through coordinate transformation. Where there is a plurality of dipoles, the potentials that are generated by the individual dipoles may be simply added together. 
     The calculated value of the electromagnetic field distribution may be used as the training data either directly or by adding noise. The electromagnetic field distribution φ i .sup.(L)&#39; with added noise is given, for instance, as: 
     
         φ.sub.i.sup.(L)&#39; =φ.sub.i.sup.(L) +η.sub.i     (5) 
    
     where φ i  is the calculated value of the electromagnetic field distribution at an i-th measurement point on the scalp, and η i  is the added noise, which is given by using a random number. The superscript L indicates that the electromagnetic field distribution is used for the training data. In many cases, the electromagnetic field distribution measured on the scalp contains noise. Thus, by having the neural network train the data containing noise, it is possible to realize a dipole parameter estimation apparatus which is robust against noise. 
     The training data will now be described. The training data is produced prior to the learning in the following way. As the head model, the above homogeneously conductive sphere, for instance, is set. It is assumed that n dipoles are present in the sphere, with the i-th dipole at a position given as (x i , y i , z i ) and with a moment given as (Mx i , My i , Mz i ). At this time, the electromagnetic field distribution produced by these dipoles on the spherical surface at a j-th point thereon is written as φ j . Where there are k electromagnetic field distribution measurement points, a single piece of training data comprises the number, positions and moments of these dipoles and electromagnetic field distributions (n, x 1 , y 1 , z 1  Mx 1 , My 1 , Mz 1 , . . . , x n , y n , z n , Mx 1 , My 1 , Mz n  φ 1 , . . . , φ k ). Such training data is prepared in an amount of, for instance 1,000 to 10,000 pieces for various numbers, positions and moments of dipoles, and it is stored in a magnetic fixed disk or similar storage device. While the above head model is a conductive sphere with a homogeneous conductivity, it is also possible to use a three layer concentric sphere model comprising three conductor layers having different conductivities in view of the fact that the brain, the cranium and the scalp have different conductivities or to use a model having the same shape as the shape of the actual head. The head model and the method of the electromagnetic field distribution calculation are detailed in James P. Ary et al., Location of Sources of Evoked Scalp Potentials: Corrections for Skull and Scalp Thicknesses, IEEE Transactions on Biomedical Engineering, Vol. BME-28, No. 6, June 1981, hereinafter referred to as &#34;Literature 2&#34;. 
     In step 2, the training data and also the measured electromagnetic field distribution value are normalized. The normalization is done, for instance, as follows. The calculated value of the electromagnetic field distribution at the i-th measurement point is written as φ i .sup.(L). At this time, the normalized electromagnetic field distribution data φ i .sup.(N) is defined as: ##EQU5## In this formula, max |φ| represents the electromagnetic field distribution value having the maximum absolute value among all the training data, and a is a normalization constant, which is appropriately 0.8, for instance but may be a different positive number. The input value is normalized in that if the input data has an excessive value, it makes the training difficult. Thus, a limitation is imposed on the maximum input value to permit more efficient execution of the training. Among the dipole parameters, the number of dipoles is normalized, for instance, as follows. ##EQU6## where N o  is the actual number, max N o  is the maximum available number, N is the normalized dipole number, and a is a real number satisfying 1&gt;a&gt;0. Of the dipole parameters, the position coordinates are normalized as follows. In the case of the x coordinate of the position, the normalized x coordinate x is defined as: ##EQU7## where x 0  is the actual x coordinate value, max |x 0  | is the maximum value of |x 0  |, and a is a real number satisfying 1&gt;a&gt;0. The other position coordinates and the three moment components are normalized likewise. The above formula defining the normalization is not restrictive, and any formula may be used so long as the value is less than unity. 
     In step 3, the weights of the neural network are initialized using random numbers or the like. 
     In step 4, the electromagnetic field distribution data in the training data is supplied to the input layer of the neural network. 
     In step 5, the dipole parameters are calculated using the neural network. FIG. 3 generally shows the neural network. As shown, it comprises three different kinds of layers, i.e., an input layer, an output layer and one or more hidden layers. In each layer, process unit systems called units 240 are provided. Each unit 240 receives input from units in the input layer side adjacent layer and provides output to units in the output layer side adjacent layer. The input/output relation of each unit can be defined as: ##EQU8## where h is the input to the unit, v is the output from the unit, and θ is a threshold value of the unit. The superscript represents the number of layers counted from the input layer. The subscript represents the unit number in each layer. W ij .sup.(1) shows the weights between the j-th unit in the (l-1)-th layer and the i-th unit in the l-th layer, and g(x) is an input/output response function. When calculations are executed orderly from the input layer to the output layer, an output can be finally obtained from the output layer. This output is the dipole parameter calculated by the neural network. As the response function, it is possible to use tanh(x) in lieu of the formula (11), and it is possible to use other functions. The above neural network in this embodiment shown in FIG. 3 has a structure which is free from feedback. However, the method in this embodiment can be used for dipole parameter estimation in cases where the neural network has a recurrent type network structure as shown in FIG. 4, in which the hidden or output layer provides feedback to layers existing in the input layer side or the own layer other than the output layer side adjacent layer. 
     In step 6, the difference between the dipole parameters calculated by the neural network and the dipole parameters in the training data is calculated. This difference is hereinafter referred to as error E. The error E is defined using, for instance, a squared-error as: ##EQU9## where σ i  is the output from the i-th unit in the output layer, and τ i  is the desired output from that unit. The error E need not be the squared-error so long as it is a non-negative value function to be zero when all σ i  and τ i  coincide. The desired output of the neural network is hereinafter referred to as a target output. As the target output, the number, positions or moments of dipoles corresponding to the supplied potential distribution data may be used, but it is also possible to use other data that represent dipole features. FIGS. 5 to 7 show examples of providing target output data. In the example of FIG. 5, electromagnetic field data φ j  (j=1, . . . , k) obtained by measurement at k measurement points is provided as input data, and there are n dipoles. Represented by x i , y i  and z i  (i=1, . . . , n) are normalized x, y and z coordinates at the i-th dipole position, and represented by Mx i , My i  and Mz i  (i=1, . . . , n) are x, y and z coordinates at the i-th dipole moment. In the example of FIG. 6, the target output is provided for dipole number estimation. FIG. 7 shows an example of the target output, in which the position and moment of dipole are represented by the on/off state of units. FIG. 7 shows only units in the output layer, and no other layer is shown. In this case, the position and moment may be expressed by making the position coordinates and moment components discrete and turning on the units corresponding to these values while turning off the other units. The on/off state of units may be expressed by numerical values &#34;1&#34; and &#34;0&#34;, but it is also possible to use other forms of expression. For example, the expression of the unit state as shown in FIG. 7 may be used for a dipole (shown by arrow mark) in FIG. 8 with the position coordinates given as (1, 1, 3) (cm) and the moment components given as (0, 1, 0) (μA·m). In FIG. 7, &#34;on&#34; units are shown by black circles, and &#34;off&#34; units are shown by white circles. While in the example of FIG. 7 there is only a single dipole, where there are a plurality of dipoles similar expression may be made by turning on units corresponding to the position coordinates and moment components of these dipoles. Further, where a plurality of dipoles are present as a group with a certain spread in an area, the expression is obtainable by turning on all the units corresponding to that area. In FIG. 7 the coordinates are made discrete for each 1 cm and also for each moment of 1 μA·m, but it is possible to make expression in a further sub-divided fashion. Further, while the example of FIG. 7 uses the Descartes coordinate system, it is possible to use as well other coordinate systems, such as a polar coordinate system or a cylindrical coordinate system. 
     In step 7, a check is done as to whether the error E is less than a predetermined reference value. If it is less than the reference value, the routine Goes to step 9. Otherwise, the routine Goes to step 8. 
     In step 8, the weights are corrected such as to reduce the error E. The weights are corrected by using error back propagation training and the like. A specific method of error correction using the error back propagation training is detailed in D. E. Rumelhart et al. Parallel Distributed Processing, Vol. 1, MIT press, pp. 318-362, 1986, hereinafter referred to as &#34;Literature 3&#34;. 
     In step 9, the three-dimensional shape of the head is measured, and shape information is stored. The shape information means the coordinates of the three-dimensional shape of the head surface and the coordinates of the measurement point. The head shape is measured by, for instance, a method, in which two-dimensional position coordinates of the head profile are measured from Xray CT pictures of the head picked up in a plurality of sections or MRI pictures and combining the two-dimensional coordinates of the head profile measured in each section. In this way, the three-dimensional shape of the head is measured. It is possible to directly measure the three-dimensional coordinates of the head by using a three-dimensional position measurement apparatus utilizing a magnetic sensor. 
     In step 10, the measured value of electromagnetic field distribution on the scalp at each point thereof is supplied to the neural network. 
     In step 11, the dipole parameters are calculated from the measured value of electromagnetic field distribution by using the neural network. The method of calculation is the same as in step 5. 
     In step 12, the dipole parameters calculated in the neural network are converted. This is done in that the dipole parameters that are estimated in the neural network has been normalized by the method described in step 2 so that it is necessary to multiply the data by a normalization constant to obtain the actual dipole parameters. For example, in the case of the x coordinate of the dipole position, the actual x coordinate value x 0  is obtained from the calculated value x from the neural network by the following calculation. 
     
         x.sub.0 =max|x|×x/a.               (13) 
    
     Similar calculation is done for other dipole parameters. 
     In step 13, the electromagnetic field distribution produced by dipole on the scalp at each measurement point thereof, is calculated using the shape information input in step 9 and the dipole parameters obtained in step 12. 
     In step 14, the difference between the measured value of the electromagnetic field distribution and the calculated value of the electromagnetic field distribution obtained from the estimated value of dipole parameters are calculated. This difference is referred to as error r. The error r is defined using the squared-error as: ##EQU10## where φ i   m  is the electromagnetic field distribution measured at the i-th measurement point on the scalp, and φ i .sup.(c) is the value of the electromagnetic field distribution at the i-th measurement point calculated from the estimated value of the dipole parameters. The error r need not be the squared-error so long as it is a function taking a non-negative value such as zero when all φ i .sup.(m) and φ i .sup.(c) coincide. 
     In step 15, a check is done as to whether the error r is less than a predetermined reference value. If it is less than the reference value, the routine goes to step 17. If it is greater than the reference value, the routine goes to step 16. 
     In step 16, the dipole parameters are corrected such as reducing the value of r between the actual measurement of the electromagnetic field distribution and the value thereof calculated from the estimated value of the dipole parameters. The correction of the dipole parameters may be made by using a squared-error minimizing method or a simplex method. The squared-error minimizing method and simplex method are detailed in, for instance, Hiroshi Konno &amp; Hirosi Yamashita, Non-linear Planning Processes, Nikka Giren Publishing Inc., hereinafter referred to as &#34;Literature 4&#34;. 
     In step 17, the dipole parameters are displayed. As the method of the dipole parameter display, there are one, in which the number, three position coordinate components and three moment components of dipoles are numerically displayed, and one, in which the dipole is displayed with a directed line segment together with three-dimensional coordinate axes. Of course, it is possible to use other methods. 
     Further, if the training by the neural network has been preliminarily completed and adequate weights are stored in a fixed magnetic disk or a similar storage device, it is possible to dispense with the above steps 3 to 8. 
     FIG. 2 is a block diagram showing an apparatus embodying the second invention. This embodiment is referred to as a second embodiment. 
     In this second embodiment, an electromagnetic field distribution measurement unit 100 measures the electromagnetic field distribution Generated on the scalp at a plurality of measurement points thereon and feeds data thus obtained to a data normalization unit 110. The electromagnetic field distribution measurement unit 100 may be realized by using, for instance, an electroencephalogram meter manufactured by NEC San-ei Instruments Ltd. or a magnetic flux measurement apparatus using a SQUID sensor. 
     The data normalization unit 110 normalizes the supplied training data and the electromagnetic field distribution data from the electromagnetic field distribution measurement unit 100 and provides the resultant normalized data to an input buffer 120 to be held. The normalization is executed by the method described before in connection with step 2 in the previous first embodiment. The data normalization unit 110 may be realized by using, for instance, a personal computer in &#34;PC-9800&#34; series manufactured by NEC Corporation, an engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, etc. 
     The input buffer 120 receives and stores the training data and the measured value of electromagnetic field distribution. The input buffer 120 is required to have a large capacity and be capable of high speed reading and writing of data. It may be realized by using a magnetic disk device, a semiconductor memory device, an optical magnetic disk device, etc. 
     A unit output memory unit 130 receives and stores the measured value of the electromagnetic field distribution and the training data from the input buffer 120. Further, the unit output memory unit 130 receives and stores an unit output from an unit output updating unit 160. When the unit output memory unit 130 receives a weight correction end signal from a weight correction unit 140, the unit output memory unit 130 supplies dipole parameters calculated by the neural network to a dipole parameter conversion unit 170. The unit output memory unit 130 may be realized by using the magnetic disk device, the semiconductor memory device, etc. 
     The weight correction unit 140 receives the target output from the input buffer 120, the unit output from the unit output memory unit 130 and the weight data from a weight memory unit 150 and derives the difference between the output value of the neural network and the target output. Then the weight correction unit 140 calculates corrected values of the weights by error back propagation training and updates the weights that are stored in the weight memory unit 150. 
     When the error between the output value of the neural network and the target output becomes less than a reference value, for instance 10 -3 , the weight correction unit 140 feeds a weight correction end signal to the unit output memory unit 130. A specific method of updating is detailed in Literature 3. The weight correction unit 140 may be realized by using a personal neuroboard &#34;Neuro-07&#34; in &#34;PC-9800&#34; series manufactured by NEC Corporation, the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, etc. 
     The weight memory unit 150 stores the weights among the individual units of the neural network. The weight memory unit 150 is referred by the unit output updating unit 160 and weight correction unit 140 and used for the calculation of the sum of the inputs to the units of the neural network and also the calculation of weight correction amounts at the time of the training with the teacher. Further, the weight correction unit 140 updates the stored contents in the weight memory unit 150. The weight memory unit 150 may be realized by using, for instance, the magnetic disk device, the semiconductor memory device, etc. 
     The unit output updating unit 160 receives the training data from the input buffer 120, the weights from the weight memory unit 150 and the individual unit outputs from the unit output memory unit 130 and calculates the output value of each unit by the method described before in connection with step 5 in the first embodiment. The calculated output values are supplied to and stored in the unit output memory unit 130. The unit output updating unit 160 may be realized by using, for instance, the personal neuroboard &#34;Neuro-07&#34; in &#34;PC-9800&#34; series manufactured by NEC Corporation, the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, etc. 
     The dipole parameter conversion unit 170 receives normalized dipole parameters calculated in the neural network from the unit output memory unit 130 and converts the received parameters into actual dipole parameters by using the method described earlier in connection with step 12 in the first embodiment. The dipole parameter conversion unit 170 may be realized by using, for instance, the personal computer in &#34;PC-9800&#34; series manufactured by NEC Corporation, the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, etc. 
     The dipole parameter memory unit 180 receives and stores dipole parameters from the dipole parameter conversion unit 170. Also, when the dipole parameter memory unit 180 receives a dipole parameter correction end signal from a dipole parameter correction unit 200, it feeds dipole parameters to a dipole parameter display unit 190. The dipole parameter memory unit 180 may be realized by using the personal computer in &#34;PC-9800&#34; series manufactured by NEC Corporation, the magnetic disk device, the semiconductor memory device, etc. 
     The dipole parameter display unit 190 receives and displays the dipole parameters stored in the dipole parameter memory unit 180. As a specific method of display may be used one described before in connection with step 17 in the first embodiment. The dipole parameter display unit 190 may be realized by using, for instance, the personal computer in &#34;PC-9800&#34; series manufactured by NEC Corporation, the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, a printer &#34;PC-PR602PS&#34; manufactured by NEC Corporation, etc. 
     The dipole parameter correction unit 200 receives the actually measured value of the electromagnetic field distribution from the electromagnetic distribution measurement unit 100, and the electromagnetic field distribution value calculated on the basis of the dipole parameters calculated in the neural network from an electromagnetic field distribution calculation unit 210, calculates the error between the actually measured value and calculated value of the electromagnetic field distribution and corrects the dipole parameters. As a specific method of dipole parameter correction may be used one described before in connection with step 16 in the first embodiment. When the error becomes less than a predetermined reference value, the dipole parameter correction unit 200 feeds a dipole parameter correction end signal to the dipole parameter memory unit 180. The dipole parameter correction unit 200 may be realized by using the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation etc. 
     The electromagnetic field distribution calculation unit 210 receives the dipole parameters from the dipole parameter memory unit 180 and shape information from the shape information memory unit 220, and calculates the electromagnetic field distribution generated on the individual points of the head surface on the basis of the dipole parameters. The electromagnetic field distribution calculation unit 210 may be realized by using the personal computer in &#34;PC-9800&#34; series manufactured by NEC Corporation, the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, etc. 
     A shape information memory unit 220 receives and stores shape information from a shape information measuring unit 230. The shape information memory unit 220 may be realized by using, for instance, the magnetic disk device, the semiconductor memory device, etc. 
     The shape information measuring unit 230 obtains measured head shape information and feeds the measured shape information to the shape information memory unit 220. A specific method of obtaining the measured head shape is one as described before in connection with step 9 in the first embodiment. The shape information measuring unit 230 may be realized by using, for instance, the digitizer, the personal computer in &#34;PC-9800&#34; series manufactured by NEC Corporation, the engineering work station &#34;EWS4800&#34; manufactured by NEC Corporation, a three-dimensional position measurement apparatus utilizing a magnetic conversion technique, provided by POLHEMUS Inc. in U.S.A. 
     While the above embodiments were described in connection with a case of outputting dipole parameters about a single dipole from on-scalp electromagnetic field distribution data, but dipole parameter estimation is possible in a similar way in case where a plurality of dipoles are present. In addition, not only from the electromagnetic field distribution on the scalp, but also from the electromagnetic distribution on any object of dipole parameters is obtainable in a similar way. 
     According to the present invention, high speed estimation of the number, positions and moments of source dipole is obtainable from the electromagnetic field distribution on a living body or an object. In addition, where there are a plurality of dipoles, high speed estimation of the number, positions and moments of the dipoles is obtainable in the same way as in the case where a single dipole is present.