Completely crazy idea for an image classification system?
frankus: Check out the [SIFT Algorithm](http://en.wikipedia.org/wiki/Scale-invariant_feature_transfo...). It's not exactly what you describe but might give you some ideas.