the present invention provides an approach for segmenting foreground from background based on integrated cues . this approach integrates three measurements , focus , intensity , and motion , using a two - layered neural network , to segment complex scenes . its advantage is that it combines simple segmentation measurements to increase robustness for segmenting a variety of scenes . by forming three separate segment maps of the image sequence , the present invention then chooses the best map based on the training of a neural network . the neural network used in the present invention is shown in fig1 along with the optimum weights determined from the training of this network using a variety of images . each map is an n × m image consisting of n × m pixels . input to the neural network is corresponding pixels , i ( i , j ), m ( i , j ), f ( i , j ), where i = 1 , . . . , n and j = 1 , . . . , m , one at a time in a left - to - right top - to - bottom order . the pixel values for the motion map are either 0 or 255 ( where 0 indicates no motion and 255 indicates motion ). the pixels values for the focus map and intensity map range from 0 to 255 , inclusive . now , once each pixel is input to the neural network , the network calculates an output value , o ( i , j ) for the ( i , j ) inputs . the final output result is an n × m image where 0 = background and 255 = foreground . so , we can think of the processing of one image as a loop that runs n × m times , i . e ., the neural network is accessed n × m times . similarly , for a sequence of images , if one image loops n × m times , then for k images the neural network is accessed k × n × m times . according to the present invention , a two - layered neural network integrates three measurements for segmentation : focus , intensity , and motion . it is worthy to note that any technique for detecting focus , intensity or motion , respectively , will suffice , as long as it provides a segmentation map based on the same information . the training of the neural network will then determine the appropriate weights to apply to the various inputs using the different segmentation techniques . two assumptions are made about the scene . first , it is assumed that the scene foreground is focused and the background blurred , i . e . closer objects are focused . second it is assumed that objects to be segmented are in motion . the focus detection technique used in the present invention is a known technique , therefore a full detailed description is not necessary to describe the present invention . a brief description , however , will be helpful . focus is a function of depth . the farther away an edge is from the point of focus , the more blurred it becomes . this measurement indicates different depths . if an object point e is not in focus , the resulting image is a blurred image e called a blur circle . the blur circle size , hence the amount of focus , is a function of the depth u of point n . image focus is easily measured from high frequency components , such as image edges . the less blurred an edge , the higher the image focus , measured from edge strength . the focus measurement d over an n × n neighborhood in an image is d = ∑ x  ∑ y   s  ( x , y )  2 w where | s ( x , y )| 2 is the magnitude of sobel edge detection on image g ( x , y ) and w is the edge width in g ( x , y ). then , within the n × n neighborhood , ƒ ( x + i , y + j )= d , whereƒ ( x , y ) is the focus measurement image , i = 0 , . . . , n , and j = 0 , . . . , n . the output of this detector is a map showing the pixels in the current image that are in focus and those that are blurred , i . e ., the pixels that are part of the foreground and the pixels that are part of the background . this map is then input to the neural network as discussed below . as in the focus detection , the motion detection technique used in the present invention is a known technique , thus a detailed description of this technique is not necessary to describe the present invention . a brief description , however , will be helpful . where md ( x , y ) is the motion detected image and g i and g i + 1 are the ith and ( i + 1 )- th frame in the sequence . motion between successive frames is indicated by pixel differences greater than threshold t . if the pixel difference is greater than the threshold , the pixel in the current image is set to a gray level of 255 , otherwise it is set to a gray level of 0 . in this case , a gray level of 255 represents black and a gray level of 0 represents white . this threshold is determined experimentally in a known way . if the object has not moved , then the result is a blank image . m  ( x , y ) = { 255 if   md  ( x , y ) & gt ; t , 0 otherwise } the output from this motion detector is a motion map indicating the pixels that are in motion and those that are not , which represent the pixels that are part of the foreground and the pixels that are part of the background , respectively . as in the focus and motion detection , the intensity detection technique used in the present invention is a known technique , thus a detailed description of this technique is not necessary to describe the present invention . a brief description , however , will be helpful . intensity j ( x , y ) is simply gray level from 0 to 255 . the importance of foreground intensity data is that it assists the neural network in segmenting object interiors . focus and motion are measured from object edges . therefore , a third measurement is needed for object interiors . in our work , this measurement is intensity , where large regions are input to the neural network . the output of this detector is an intensity map , which indicates those pixels belonging to the foreground and those to the background . neural network a two - layered back propagation network is trained to segment a sequence . fig1 shows the network architecture . the neural network 10 includes a hidden layer 11 and an output layer 12 . the hidden layer 11 contains two neurons 13 , 14 , and the output layer contains one neuron 15 . the neurons 13 - 15 use sigmoid functions with weighted inputs . essentially , these are summing amplifiers with weighted inputs . the inputs to the network are the motion , focus , and intensity measurements , or segmentation maps . the output is the segmented foreground image o ( x , y ), o  ( x , y ) = { 255 if   forground , 0 otherwise } . the network is trained using the initial two frames of a sequence and its hand - segmented result . it is possible to speed the training with an adaptive learning rate , according to the rule , where w is a layer &# 39 ; s weights , b is its bias , lr is the adaptive learning rate , d is the layer &# 39 ; s delta vectors , and p is its input vector and t indicates that vector p is first transposed before being multiplied . the present invention provides an integrated segmentation approach to coding of images . foreground and background features are segmented and background features discarded . the network is first trained using the first two frames of a sequence to get focus , motion , intensity , and segmented data . see fig2 which shows the four step training algorithm 20 for training the neural network . in the first step of the process , the first images in the sequence are acquired 21 . next , the images are segmented 22 by hand . next , motion , focus and intensity are calculated 23 . finally , the neural network is trained 24 using the speed up process discussed above . fig3 also shows the four - step segmentation algorithm 30 . first , two successive images are acquired 31 . next , focus , motion , and intensity are measured 32 . the measurements are input to the trained neural network 33 . the network outputs the segmented foreground . the segmented foreground is then outlined on the original image 34 , which indicates the capability of the process of the present invention . as can be seen in fig5 ( c ), the output of the process resulted in properly segmenting the man from the background . the segmentation map is used as a mask for separating the foreground from the background in the image coding process . the neural network applies the weights in table 1 below to calculate the following equation : o  ( x , y ) = w 1 w 2 [ v 11  i  ( x , y ) + v 21  f  ( x , y ) + v 31  m  ( x , y ) + v 41  b  ( x , y ) v 12  i  ( x , y ) + v 22  f  ( x , y ) + v 32  m  ( x , y ) + v 42  b  ( x , y ) ] + + w 3 b  ( x , y ) where o ( x , y ) is the segmentation map , i ( x , y ) is the intensity segmentation map , m ( x , y ) is the motion segmentation map , f ( x , y ) is the focus segmentation map , b ( x , y ) is the bias information , and v11 , v21 , v31 , v41 , v12 , v22 , v32 , v42 , w1 , w2 , and w3 are the weights indicated in table 1 . these weights have been determined to work over some particular images . modifications to the exact weights will occur depending upon the exact images being used . these weights are merely indicative of those determined by the inventors . fig4 shows a plot 40 of the error rate ( sum - squared error ) 41 versus epochs ( i . e ., training cycles , which for an n × m image is n × m bytes of processing ) 42 during training of a test sequence ( see fig5 ). with good training , error reduces as training time increases , until a minimum error is reached . in our training session , the sum squared error reached a minimum at 4000 . this translates into an average intensity difference between the neural network - generated segmentation map and the actual segmentation map for our 176 × 144 images of 0 . 0025 / pixel . an advantage of the present invention is that it segments without any post - processing operation to fill the segmented object interior . prior techniques required a fill operation to create the segmentation mask shown in fig5 ( b ). this filling operation is non - trivial , especially with regard to an image without line segments that are closed . as a result of the present invention , the shape of the object is preserved by the intensity measurement in the neural network . as the focus and motion detectors operate on edge effects , which are high frequency components , they provide little information regarding the interior of the image . thus , without the intensity measurement , a filling operation is necessary . since the intensity measurement provides information regarding the interior of the image , using this information in the neural network eliminates the need for filling the interior of the image , thus making the post - processing filling step unnecessary . in addition , the intensity measurements are easily calculated . fig5 shows the segmentation results for a frame in an image sequence . as shown , the neural network segmentation is accurate for the fiftieth frame of this sequence , which was trained on the first and second frames of the sequence . fig5 ( a ) depicts the output from the camera 61 , which is input to the three detectors . fig5 ( b ) shows the final segmentation map output from the neural network , which as is evident corresponds well to the outlined figure . fig5 ( c ) shows the segmentation foreground outlined , which shows the boundary of the foreground and background . this is shown to indicate the success of the segmentation approach , but is never actually created for the next step in the coding process . fig6 shows the apparatus 60 for implementing the method of the present invention . two successive images are first acquired using a digital camera 61 , for example . next , the digitized images are input to three detectors 63 , 64 , 65 , which calculate motion segmentation maps , focus segmentation maps and intensity segmentation maps , respectively . these maps are then input to the neural network 66 , which outputs the final segmentation map , which is used to outline the foreground from the background . thus , the present invention discloses an approach to foreground and background segmentation using integrated measurements . this approach is advantageous for two reasons . one , it is computationally simple . two , combined measurements increase robustness in segmenting complex scenes . other possible modifications include comparing the use of intensity versus color measurements as a basis for segmentation . while a neural network is used to perform the integration of the multiple maps and the assignment of weight , a fizzy logic circuit could also be employed . this invention could also be implemented on a sun sparc workstation with an image acquisition device , such as a digital camera and a video board . one could also modify the method of the application and use a known disparity detector as an additional input to the neural network or as a replacement for one of the focus or intensity measurements . this is accomplished by simply by replacing one of the focus or intensity detectors with the disparity detector , which outputs its version of the segmentation map , which is then weighted by the neural network .