--- language: - en license: apache-2.0 tags: - roberta - classification - dialog state tracking - natural language understanding - uncertainty - conversational system - task-oriented dialog datasets: - ConvLab/multiwoz21 metrics: - Joint Goal Accuracy - Slot F1 - Joint Goal Expected Calibration Error model-index: - name: setsumbt-dst-nlu-multiwoz21 results: - task: type: classification name: dialog state tracking dataset: type: ConvLab/multiwoz21 name: MultiWOZ21 split: test metrics: - type: Joint Goal Accuracy value: 51.8 name: JGA - type: Slot F1 value: 91.1 name: Slot F1 - type: Joint Goal Expected Calibration Error value: 12.7 name: JECE --- # SetSUMBT-dst-nlu-multiwoz21 This model is a fine-tuned version [SetSUMBT](https://github.com/ConvLab/ConvLab-3/tree/master/convlab/dst/setsumbt) of [roberta-base](https://huggingface.co/roberta-base) on [MultiWOZ2.1](https://huggingface.co/datasets/ConvLab/multiwoz21). This model is a combined DST and NLU model and is a distribution distilled version of a ensemble of 5 models. This model should be used to produce uncertainty estimates for the dialogue belief state. Refer to [ConvLab-3](https://github.com/ConvLab/ConvLab-3) for model description and usage. ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.00001 - train_batch_size: 3 - eval_batch_size: 16 - seed: 0 - gradient_accumulation_steps: 1 - optimizer: AdamW - loss: Ensemble Distribution Distillation Loss - lr_scheduler_type: linear - num_epochs: 50.0 ### Framework versions - Transformers 4.17.0 - Pytorch 1.8.0+cu110 - Datasets 2.3.2 - Tokenizers 0.12.1