metadata
tags:
- roberta
- adapterhub:nli/multinli
- adapter-transformers
license: apache-2.0
language:
- en
library_name: adapter-transformers
Adapter yoh/distilroberta-base-sept-adapter for distilroberta-base
An adapter for the distilroberta-base model that was trained on the AllNLI, Sentence compression and Stackexchange duplicate question datasets (see information here).
This adapter was created for usage with the adapter-transformers library. See this paper and repository for more information on the tasks.
Usage
First, install adapter-transformers and sentence-transformers:
pip install -U adapter-transformers sentence-transformers
Note: adapter-transformers is a fork of transformers that acts as a drop-in replacement with adapter support. More
Now, the adapter can be loaded and activated like this:
from sentence_transformers import SentenceTransformer, models
# Load pre-trained model
word_embedding_model = models.Transformer("distilroberta-base")
# Load and activate adapter
word_embedding_model.auto_model.load_adapter("yoh/distilroberta-base-sept-adapter", source="hf", set_active=True)
# Create sentence transformer
pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension(), pooling_mode='mean')
model = SentenceTransformer(modules=[word_embedding_model, pooling_model])
Architecture & Training
See this paper
Evaluation results
See this paper
Citation
@article{huang2023adasent,
title={AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification},
author={Yongxin Huang and Kexin Wang and Sourav Dutta and Raj Nath Patel and Goran Glavaš and Iryna Gurevych},
journal = {ArXiv preprint},
url = {https://arxiv.org/abs/2311.00408},
volume = {abs/2311.00408},
year={2023},
}