@inproceedings{
stretcu2023agile,
title={Agile Modeling: From Concept to Classifier in Minutes},
author={Otilia Stretcu and Edward Vendrow and Kenji Hata and Krishnamurthy Viswanathan and Vittorio Ferrari and Sasan Tavakkol and Wenlei Zhou and Aditya Avinash and Enming Luo and Neil Gordon Alldrin and Mohammadhossein Bateni and Gabriel Berger and Andrew Bunner and Chun-Ta Lu and Javier A Rey and Giulia DeSalvo and Ranjay Krishna and Ariel Fuxman},
booktitle={NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World},
year={2023},
url={https://openreview.net/forum?id=6hkY6dYtBA}
}
The application of computer vision methods to nuanced, subjective concepts is growing. While crowdsourcing has served the vision community well for most objective tasks (such as labeling a "zebra"), it now falters on tasks where there is substantial subjectivity in the concept (such as identifying "gourmet tuna"). However, empowering any user to develop a classifier for their concept is technically difficult: users are neither machine learning experts nor have the patience to label thousands of examples. In reaction, we introduce the problem of Agile Modeling: the process of turning any subjective visual concept into a computer vision model through real-time user-in-the-loop interactions. We instantiate an Agile Modeling prototype for image classification and show through a user study (N=14) that users can create classifiers with minimal effort in under 30 minutes. We compare this user driven process with the traditional crowdsourcing paradigm and find that the crowd's notion often differs from that of the user's, especially as the concepts become more subjective. Finally, we scale our experiments with simulations of users training classifiers for ImageNet21k categories to further demonstrate the efficacy of the approach.
Agile Modeling: From Concept to Classifier in Minutes
Otilia Stretcu Edward Vendrow Kenji Hata Krishnamurthy Viswanathan Vittorio Ferrari Sasan Tavakkol Wenlei Zhou Aditya Avinash Enming Luo Neil Gordon Alldrin Mohammadhossein Bateni Gabriel Berger Andrew Bunner Chun-Ta Lu Javier A Rey Giulia DeSalvo Ranjay Krishna Ariel Fuxman