@inproceedings{kotian-etal-2024-taal,
    title = "{TAAL}: Target-Aware Active Learning",
    author = "Kotian, Kunal  and
      Bhattacharya, Indranil  and
      Gupta, Shikhar  and
      Pavani, Kaushik  and
      Bhandari, Naval  and
      Dasgupta, Sunny",
    editor = "Malmasi, Shervin  and
      Fetahu, Besnik  and
      Ueffing, Nicola  and
      Rokhlenko, Oleg  and
      Agichtein, Eugene  and
      Guy, Ido",
    booktitle = "Proceedings of the Seventh Workshop on e-Commerce and NLP @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.ecnlp-1.14",
    pages = "136--144",
    abstract = "Pool-based active learning techniques have had success producing multi-class classifiers that achieve high accuracy with fewer labels com- pared to random labeling. However, in an industrial setting where we often have class-level business targets to achieve (e.g., 95{\%} recall at 95{\%} precision for each class), active learning techniques continue to acquire labels for classes that have already met their targets, thus consuming unnecessary manual annotations. We address this problem by proposing a framework called Target-Aware Active Learning that converts any active learning query strategy into its target-aware variant by leveraging the gap between each class{'} current estimated accuracy and its corresponding business target. We show empirically that target-aware variants of state-of-the-art active learning techniques achieve business targets faster on 2 open-source image classification datasets and 2 proprietary product classification datasets.",
}

Pool-based active learning techniques have had success producing multi-class classifiers that achieve high accuracy with fewer labels com- pared to random labeling. However, in an industrial setting where we often have class-level business targets to achieve (e.g., 95{\%} recall at 95{\%} precision for each class), active learning techniques continue to acquire labels for classes that have already met their targets, thus consuming unnecessary manual annotations. We address this problem by proposing a framework called Target-Aware Active Learning that converts any active learning query strategy into its target-aware variant by leveraging the gap between each class{'} current estimated accuracy and its corresponding business target. We show empirically that target-aware variants of state-of-the-art active learning techniques achieve business targets faster on 2 open-source image classification datasets and 2 proprietary product classification datasets.
Kotian, Kunal Bhattacharya, Indranil Gupta, Shikhar Pavani, Kaushik Bh ari, Naval Dasgupta, Sunny
TAAL: Target-Aware Active Learning