1. A muon tomography system that detects threat in an object, cargo or vehicle under inspection, comprising: a first set of position sensitive muon detectors located on a first side of an object holding area to measure positions and directions of incident muons towards the object holding area; a second set of position sensitive muon detectors located on a second side of the object holding area opposite to the first side to measure positions and directions of outgoing muons exiting the object holding area; wherein each of the first and second sets of position sensitive detectors is structured and arranged to allow at least three charged particle positional measurements in a first direction and at least three charged particle positional measurements in a second direction different from the first direction, and a signal processing unit to receive data of measured signals of the incoming muons from the first set of position sensitive muon detectors and measured signals of the outgoing muons from the second set of position sensitive muon detectors, the signal processing unit configured to analyze scattering behaviors of the muons caused by scattering of the muons in the materials within the object holding area based on the measured incoming and outgoing positions and directions of muons to obtain a tomographic profile or the spatial distribution of scattering centers within the object holding area, and to generate an obtained cosmic muon image of the object, cargo or vehicle under inspection, wherein the signal processing unit is operable to subtract a reference cosmic muon image that represents a background of an object, cargo or vehicle under inspection in the object holding area from the obtained cosmic muon image of the object, cargo or vehicle under inspection to obtain a subtracted image; and process the subtracted image to determine whether the object, cargo or vehicle under inspection contains a target object that is absent from the reference cosmic muon image, and wherein the signal processing unit is configured to generate the subtracted image after a short exposure time of data collection by the first and second sets of position sensitive detectors for generating the obtained cosmic muon image of the object, cargo or vehicle under inspection, wherein the short exposure time is insufficient for using the obtained cosmic muon image to fully isolate and identify the target object from the background of the object, cargo or vehicle under inspection in the object holding area in absence of the subtracted image based on the reference cosmic muon image, and the signal processing unit is configured to indicate a threat when the subtracted image is at or exceeds a threshold for a threat.