1. An object localization method using multiple cameras comprising the steps of: obtaining a plurality of two-dimensional (2D) line samples based on foreground pixels in a camera view for a scene of interest, wherein the 2D line samples are foreground line segments along a plurality of 2D sampling rays that are originated from a pre-specified and fixed vanishing point in the camera view and established, regardless of any image feature, for pre-specified angular positions for representing the projections of non-horizontal three-dimensional (3D) planes containing a camera center to the camera view, and each of the 2D line samples corresponds to an angle on the non-horizontal 3D plane with a vertex of the angle being the camera center and two sides of the angle defined by the two ends of the foreground line segment, for representing foreground video objects for the scene of interest; constructing a plurality of 3D line samples that stand on at least one 3D reference plane for the scene of interest from the 2D line samples in multiple camera views, wherein the 3D line samples are obtained by intersecting the angles associated with the 2D line samples; and determining 3D locations of foreground objects by clustering the 3D line samples into object groups.