Q:

How to detect a quadrilateral shape in an image in Java?

I want to detect a quadrilateral shape in an image in Java. I can use the HoughLinesP method in OpenCV (using javaCV for opencv java binding) to detect line segments. But I can't figure out how to detect a quadrilateral shape - is there another method for that or some way to use the hough lines? Also once the corners of the quadrilateral are detected, I want it to return a rectangle just like this class does - http://www.aforgenet.com/framework/docs/html/7039a71d-a87d-47ef-7907-ad873118e374.htm  - is there an equivalent library in openCV?

A:

How do your input images look like? If you detect lots of line segments with the Hough transformation, you could maybe try using RANSAC to generate a number of quadrilateral shape hypothesis, find a way to rate their fitness and return the best hypothesis. 
One hypothesis could be generated like this:

choose four random line segments from the detected line segment set
find the four corners by looking for intersections of the lines the chosen line segments lie on
rate the fitness of the quadrilateral shape defined by these four points

The fitness could maybe be the area of the hypothetical quadrilateral (see the Bretschneider's formula for calculating the area of a convex quadrilateral), the distance of the quadrilateral edges from the other line segments in the detected set, or something that better fits your application.
This is just an idea, I haven't tried using this approach yet (but I'm planing on implementing something similar). Let me know if you think this could work, or why it won't! :)

