Visualizing classification of natural video sequences using sparse, hierarchical models of cortex.
References [1] Hubel, D.H. Wiesel, T.N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. (Lond. 195, 215 243 (1968) [2] K. Fukushima: Neocognitron: A self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biological Cybernetics, 36(4) pp. 193 202 (April 1980) [3] Riesenhuber, M. Poggio, T. (1999) Hierarchical Models of Object Recognition in Cortex, Nature Neuroscience 2: 1019 1025. [4] T. Serre, A. Oliva and T. Poggio. A feedforward architecture accounts for rapid categorization. Proceedings of the National Academy of Science, 104(15) pp. 6424 6429, April 2007. [5] T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber and T. Poggio. Object recognition with cortex like mechanisms. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 29 (3) pp. 411 426 2007. [6] Paul Henning and Andrew White, Trailblazing with Roadrunner, Computing in Science Engineering, July/August 2009, pp. 91 95. [7] Hans Moravec, When will computer hardware match the human brain? J. Evolution &Technology, 1998. [8] S.P. Brumby, G. Kenyon, W. Landecker, C. Rasmussen, S. Swaminarayan, and L.M.A. Bettencourt, "Large scale functional models of! visual cortex for remote sensing" Proc. 38th IEEE Applied Imagery Pattern Recognition, Vision: Humans, Animals, and Machines, 2009. [9] Pinto N, Cox DD, and DiCarlo JJ. Why is Real World Visual Object Recognition Hard? PLoS Comp. Biology, 4(1):e27 (2008) [10] Pinto N, Cox DD, and DiCarlo JJ. A High Throughput Screening Approach to Discovering Good Forms of Biologically Inspired Visual Representation, GTC 2009. [11] Olshausen, B. A. and Field, D. J. (1996) Emergence of simple cell receptive field properties by learning a sparse code for natural images. Nature, 381:607 609. [12] Jim Mutch and David G. Lowe. Object class recognition and localization using sparse features with limited receptive fields. International Journal of Computer Vision (IJCV) 80(1) pp. 45 57, October 2008. [13] William K. Coulter, Cristopher J. Hillar, Guy Isley, Friedrich T. Sommer. Adaptive compressed sensing A new class of self organizing coding models for neuroscience. ICASSP 2010, 5494 5497, Dallas, TX, March 2010. [14] Chang, C. C. and C. J. Lin (2001) LIBSVM: a library for support vector machines. (http:/www.csie.ntu.edu.tw/~cjlin/libsvm) [15] Oja, Erkki, Simplified neuron model as a principal component analyzer, Journal of Mathematical Biology 15 (3) 267 273, Nov 1982. [16] S. Brumby, L. Bettencourt, M. Ham, R. Bennett, and G. Kenyon, "Quantifying the difficulty of object recognition tasks via scaling of accuracy versus training set size" Computational and Systems Neuroscience (COSYNE) 2010, 25 28 Feb 2010, Salt Lake City, UT. [17] Will Landecker, Steven Brumby, Mick Thomure, Garrett Kenyon, Luis Bettencourt, and Melanie Mitchell, "Visualizing Classification Decisions of Hierarchical Models of Cortex" Computational and Systems Neuroscience (COSYNE) 2010, 25 28 Feb 2010, Salt Lake City, UT. [18] H. Jhuang, E. Garrote, X. Yu, V. Khilnani, T. Poggio and A. Steele and T. Serre, Automated home cage behavioral phenotyping of mice, Nature Communications, 1(1) doi:10.1038/ncomms1064, 2010. [19] J. Mairal, F. Bach and J. Ponce. Task Driven Dictionary Learning. submitted to IEEE PAMI. Sep 2010. (http:/arxiv.org/abs/1009.5358) [20] S. G. Mallat and Z. Zhang, Matching Pursuits with Time Frequency Dictionaries, IEEE Transactions on Signal Processing, pp. 3397 3415, Dec 1993. Abstract Recent work on hierarchical models of visual cortex has reported state of the art accuracy on whole scene labeling using natural still imagery. This raises the question whether the reported accuracy may be due to the sophisticated, nonbiological back end supervised classifiers typically used (support vector machines) and/or the limited number of images used in these experiments. In particular, is the model classifying features from the object or the background? Previous work (Landecker, Brumby, et al. COSYNE 2010) proposed tracing the spatial support of a classifier's decision back through a hierarchical cortical model to determine which parts of the image contributed to the classification, compared to the positions of objects in the scene. In this way, we can go beyond standard measures of accuracy to provide tools for visualizing and analyzing high level object classification. We now describe new work exploring the extension of these ideas to detection of objects in video sequences of natural scenes.