1. A dual-aperture digital camera with autofocus (AF) for imaging an object or scene, comprising: a) a first sub-camera that includes a first optics bloc and a color image sensor with a first number of pixels, the first camera operative to provide a color image of the object or scene; b) a second sub-camera that includes a second optics bloc and a luminance image sensor having a second number of pixels, the second sub-camera operative to provide a luminance image of the object or scene, wherein the first and second sub-cameras have substantially the same field of view; c) an AF mechanism coupled mechanically at least to the first optics bloc and used to perform an AF action on at least the first optics bloc; and d) a camera controller coupled to the AF mechanism and to the two image sensors and configured to control the AF mechanism, to calculate a sharpness difference between the color and luminance images, the sharpness difference being due to the AF action, and to process the color and luminance images into a fused color image using the calculated sharpness difference.