this invention resides in a real time computer vision system capable of tracking the motion of objects , including the human body or portions thereof . the system is capable of tracking the gestures and behaviors through an unstructured and possibly cluttered environment , then outputs the position of the tracked features in each observed scene . to determine position in an immersive environment , a user is preferably outfitted with active infrared emitters which are tracked by custom linear cameras . a set of design specifications associated with an implemented system are shown in table 1 : the implemented system is capable of determining the location of 30 points , 30 times a second with a resolution of 2 . 5 mm within 5 meters of the tracking system . the field of view , range and accuracy have been specified to provide a reasonably large working volume to accommodate a variety of applications . the number of sensors was selected to allow for placement of multiple sensors on desired tracking points to allow the same point to be located irrespective of orientation to reduce the adverse effects of line - of - sight occlusion . virtual reality applications such as head tracking for head / helmet mounted display ( hmd ) generation dictate the high accuracy , sensor scan rate ( same as display update rate ), and low latency , all of which are desirable to help combat simulator sickness . the invention relies on an infrared - based , non - contact motion measurement system . referring to fig1 , small infrared ( ir ) light emitting diodes ( leds ) called tags ( 102 ) attached to the person or object are flashed in sequence using a controller 104 and tracked with a set of three linear optical sensors 106 . optical filters shown in fig6 are used to reduce background ir emissions and highlight the ir leds , thereby reducing the complexity of the image processing algorithms and improving system performance . the system works well in indoor conditions where diffuse incandescent or fluorescent light is present . the presence of direct incandescent light or sunlight can be tolerated somewhat . the absolute 3d position of each ir led ( tag ) is computed from the angle of arrival detected by the optical sensors using triangulation methods shown in fig2 . the ir led tags are button - sized devices ( preferably no greater than 0 . 25 inch diameter ) that are attached to the objects / points to be tracked as applicable to the object / point ( velcro ®, double sided surgical tape , etc .). the tags preferably use 890 nm low directivity leds . the relative intensity of the ir radiation is 80 percent at 90 degrees off axis , allowing the tag to be readily imaged when the camera is in the half - plane field of view . each tag is preferably constructed by encapsulating the backside of the led in plastic both for a smooth mounting surface as well as to provide strain relief for the electrical connections . the total tag package is small , and so light that it may be unobtrusively affixed to a persons face and be used to resolve facial features . the wires from the tags are then run to the tag controller 104 , which is a walkman sized , untethered , battery powered device that may be attached to a person &# 39 ; s belt . the tag controller also has a rs - 232 serial port for local ( on the person ) communication , and an infrared data access ( irda ) compliant serial port for external communication and programming with a maximum baud rate of 115 . 2 kbps . the tag controller 104 turns the ir led tags on and off in sequence with precise timing to allow the position sensor array to view only one tag per camera exposure . fig3 is a block diagram of the ir led tag controller 104 . the controller allows for the tag illumination sequence to be initiated based on an external electrical signal ( which can be generated from the camera array controller ). if so connected , the controller synchronizes the tag sequence which the sync signal . if not , the tag controller cycles the tags based on its internal crystal clock timing circuits . the controller provides an incrementing output to decode circuits that directly drive the tag leds . the default mode of the tag controller is to scan 30 tags at 30 hz , but it can be programmed to scan fewer tags at higher rates or more tags at lower scan rates . thirty leds are sequenced in 33 . 333 milliseconds . if fewer than 32 leds are programmed , the sequence complete more quickly . the capabilities of the tag controller could be expanded to include more sensors at lower scan rates provided that the aggregate frame rate of 900 hz is not exceeded . a few alternate sensor scan rates are given in table 2 : fig4 shows the tracking system timing diagram assuming 30 leds are active and tracked . sync is the sync signal either generated electrically by the camera array controller or detected via the ir optical sync detector that is a part of the camera array controller . note that first led in the sequence is shorter in duration and brighter in intensity . in the preferred embodiment , this led is also modulated with a 200 khz signal which helps makes detection of the pulse easier against the constant background radiation presented to the optical sync detector photodiode by ambient lights ( overhead fluorescent and incandescent lights ). the optical sync detector shown in fig5 detects the first ( and all other led ) ir pulses using a photodiode 502 . because the signal from the diode is very low level , it is amplified by a high gain front - end circuit 510 . then the signal is filtered at 512 to remove all high frequency noise ( frequencies greater than the 200 khz modulation frequency ). then the signal is filtered by a narrow bandpass filter 514 set at 200 khz . because led 0 is modulated at this frequency and all ambient light and light from other higher numbered leds are not , only when led 0 is lit is there an output to the envelope detector 516 . this signal appears when led 0 is lit , or when the tag sequence begins . the start signal is conditioned by an isolation amplified - schmitt trigger pair 520 and present to the camera array controller ( fig7 ) as a signal to initiate frame capture of target 0 . the position sensor consists of three near infrared linear ccd cameras mounted on a 1 meter bar that views each tag from three separate locations , as shown in fig1 and 2 . in the preferred embodiment , two cameras are oriented horizontally and one vertically to provide a complete basis vector for computing three - dimensional location of the tags . each connects to the camera array controller through a serial digital interface . the camera system itself is controlled via a dsp that accepts commands from the array controller and send data back to the array controller via the serial digital interface . the dsp operates the linear ccd through a ccd controller circuit that handles all ccd timing and control and provides for digitizing the analog ccd circuit outputs for read into the dsp ( through a fifo buffer circuit ). the current implementation uses a 2048 element linear ccd circuit . analog outputs from the ccd bucket brigade are digitized to eight - bit accuracy . as shown in fig6 , each tag image is presented to the ccd active area 606 through a high pass optical filter 606 ( which moves a substantial portion of the visible band from the input light energy spectra ) and a cylindrical lens 604 which elongates the tag spot image perpendicular to the ccd linear extent . using cylindrical optics 604 and ir - pass filter 606 , the linear cameras measure the angular position of the tags in one dimension only . the dsp detects the bright area projected from a tag using a spot fitting algorithm so that the localization of spot position is not limited to the resolution set by the linear camera pixel density ( 2048 in this implementation ). rather , resolution along the ccd linear extent of nominally ten times better is achieved by the subpixel - processing algorithm . the ccd array 602 interfaces to a specialized linear ccd processor 610 . the processor 610 controls timing of the ccd readout , has variable gain amplifiers and an 8 - bit a / d converter and can support pixel scan rates of up to 2 . 5 megapixels / second . the image is processed in real time in the camera itself by digital signal processor ( dsp , 620 ) to determine the angle of arrival . the horizontal ( or vertical ) resolution of the proposed system can be adjusted by varying the field of view and the number of pixels in the ccd , as set forth in table 3 : the resolution in the depth dimension can be adjusted by varying the distance between the two horizontal resolution cameras . a 1 - meter separation of two 2048 pixel linear ccd cameras with a field of view of 45 degrees , results in a resolution of 4 . 56 mm in the depth dimension . at this point , it is important to note that the aforementioned resolution numbers assume that the location of the ir tag can be resolved to one pixel . this is a worst case resolution number since image processing algorithms that can easily achieve sub - pixel location and image registration are readily available . the camera array controller depicted in fig6 generates an electrical sync signal at the start of each target capture cycle that be directly connected to the tag controller . in this mode , the camera systems and tag controller are electrically synchronized and not subject to any ambient lighting noise or other effects . alternatively , the camera array controller accepts a sync signal at the beginning of each tag controller tag illumination sequence derived from the detected output of led 0 . in either case , the camera array controller signals the dsp to initial frame capture simultaneously on the three linear ccd imaging systems ( through the ccd controller integrated circuits that provide control and timing of the ccd circuits ). each camera subsystem produces digital output that locates the bright spot ( from one of the tags ) along the ccd linear extent . this location is read by the dsp from each camera and then used to compute the tag three - dimensional location based on factory calibration parameters . ( each camera system is placed on a fixed calibration frame at the factory . leds located at known places on the frame are lit in sequence so that where they project onto the linear cameras is determined . from this data it is possible to compute the transform which converts locations along each camera linear extent to three - dimensional points in the system field of interest . once the angles of arrival have been determined by the individual cameras , the three angles are transmitted to another dsp in the camera array . this dsp computes the three dimensional calibrated position of the infrared tag in real time using the relationships shown in fig2 , and transmits the result in the form of an output position value in x , y , and z via a serial rs - 232 interface . the output may be delivered to a workstation or pc which captures the motion tracking data set for display or use in computer animation or gesture control applications . in addition to position values , each output includes a tag detection confidence factor . this is necessary because tags can be blocked from view so that no valid x , y , and z value can be computed . the output and camera data input interfaces could be any other type of digital transmission interface including firewire , usb , ethernet , or other parallel or serial digital interfaces . the overall approach of this system is very cost effective due the reduced cost of the required hardware . this is accomplished in at least two ways : 1 ) by decoupling the horizontal dimension from the vertical using cylindrical optics , and 2 ) through the use parallel processing to speed up the image processing . each camera needs only to compute the angle of arrival , which is based on the location of the brightest spot on the ccd . an advantage of the invention over systems that use one or more two - dimensional ccd cameras is that high speed linear cameras are not as costly , and produce smaller raw images ( three images of 2048 pixels as compared to two or more images of 1024 × 1024 pixels ), which can be processed with simpler algorithms faster . this , combined with processing of each 2048 pixel image separately is the key to minimizing the system &# 39 ; s latency . the system also has the advantage that 3d tracking may be accomplished in a noisy environment without interfering with the user &# 39 ; s experience . in table 3 , the accuracies quoted exceed the desired 1 centimeter resolution at 8 meters without the use of subpixel resolution algorithms . to meet the field of view specifications , it may be desirable to adjust the optical components of the linear cameras to widen the field of view , however , that would still provide a 1 centimeter resolution . this invention finds utility in a variety of more comprehensive systems , including human body tracking and gesture recognition . although different apparatus may be used , the optical body tracker described herein may be interfaced to the gesture recognition system disclosed in u . s . pat . no . 6 , 681 , 031 , or to the systems described in u . s . provisional patent application ser . nos . 60 / 183 , 995 ; 60 / 186 , 474 ; or 60 / 245 , 034 , all of which were incorporated herein by reference above . u . s . pat . no . 6 , 681 , 031 , for example , describes a system engineered to control a device such as a self - service machine , regardless of whether the gestures originated from a live or inanimate source . the system not only recognizes static symbols , but dynamic gestures as well , since motion gestures are typically able to convey more information . a gesture is defined as motions and kinematic poses generated by humans , animals , or machines . specific body features are tracked , and static and motion gestures are interpreted . motion gestures are defined as a family of parametrically delimited oscillatory motions , modeled as a linear - in - parameters dynamic system with added geometric constraints to allow for real - time recognition using a small amount of memory and processing time . a linear least squares method is preferably used to determine the parameters which represent each gesture . feature position measure is used in conjunction with a bank of predictor bins seeded with the gesture parameters , and the system determines which bin best fits the observed motion . recognizing static pose gestures is preferably performed by localizing the body / object from the rest of the image , describing that object , and identifying that description . further details regarding this and the other systems incorporated herein by reference may be obtained directly from the respective applications . in addition to the applications already described , the technology disclosed herein may also be used to detect and localize bright flashes of ir illumination over a longer distance . this could be useful for detecting the launch of man - portable air defense systems ( manpads ) or rocket propelled grenades ( rpg &# 39 ; s ). detection of these devices is currently very difficult . however , the capability is necessary to protect both commercial and military assets ( including aircraft ). firefly already localizes bright ir illumination and produces a 3d position for the light . the projectile tracking scenario extends these capabilities to work over a larger range . however the general calculations and tracking principles are the same . the system is therefore applicable to both tracking and detection . another application area is tracking head motions in a virtual reality simulator . such simulators are well known to those of skill in the art .
an optical system tracks the motion of objects , including the human body or portions thereof using a plurality of three - dimensional active markers based upon triangulation from data read via multiple linear ccds through cylindrical lenses . each marker is lit in sequence so that it is in sync with a frame capture using the imaging system positioned and oriented so as to provide a basis for computing three - dimensional location . in the preferred embodiment , the imaging system detects an infrared signal which is sent out by the tag controller as part of the tag / marker illumination sequence at the beginning of the first tag position capture time . the controller then traverses through the tags in time sync with each imaging system frame capture cycle . thus , only one unique tag will be lit during each image capture of the cameras , thereby simplifying identification . using linear ccd sensors , the frame time is very short , allowing very many markers to be sampled and located sequentially in real time .