license: mit
language:
- en
pipeline_tag: text-classification
arxiv: 2305.03695
Model Card for Vera
Vera is a commonsense statement verification model. See our paper at: https://arxiv.org/abs/2305.03695.
Model Details
Model Description
- Developed by: Jiacheng Liu, Wenya Wang, Dianzhuo Wang, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi
- Shared by [optional]: Jiacheng Liu
- Model type: Transformers
- Language(s) (NLP): English
- License: MIT
- Finetuned from model [optional]: T5-v1.1-XXL
Model Sources [optional]
- Repository: https://github.com/liujch1998/vera (Coming soon!)
- Paper [optional]: https://arxiv.org/abs/2305.03695
- Demo [optional]: https://huggingface.co/spaces/liujch1998/vera
Uses
Direct Use
Vera is intended to predict the correctness of commonsense statements.
Downstream Use [optional]
Vera can be used to detect commonsense errors made by generative LMs (e.g., ChatGPT), or filter noisy commonsense knowledge generated by other LMs (e.g., Rainier).
Out-of-Scope Use
Vera is a research prototype and may make mistakes. Do not use for making critical decisions. It is intended to predict the correctness of commonsense statements, and may be unreliable when taking input out of this scope.
Bias, Risks, and Limitations
See the Limitations and Ethics Statement section of our paper.
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Use the code below to get started with the model.
Please refer to https://huggingface.co/spaces/liujch1998/vera/blob/main/app.py#L27-L98
Training Details
Training Data
See the Data Construction section of our paper.
Training Procedure
See the Model Training section of our paper.
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime: [More Information Needed]
Speeds, Sizes, Times [optional]
[More Information Needed]
Evaluation
See the Evaluation Results section of our paper.
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: V100
- Hours used: 2560
- Cloud Provider: Private
- Compute Region: US
- Carbon Emitted: 331 kg
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
Jiacheng Liu