Edit model card

YAML Metadata Warning: The pipeline tag "conversational" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, text2text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, any-to-any, other

Xwinter 120B

A Goliath-120b style frankenmerge of Xwin-LM-70b-v0.1 and WinterGoddess-1.4x-70b. Meant as a slight update to Goliath by using the successor model to Euryale.

There's a similar merge called WinterGoliath-123b by @ChuckMcSneed.

Prompting Format

Vicuna and Alpaca.

Merge process

The models used in the merge are Xwin-LM-70b-v0.1 and WinterGoddess-1.4x-70b.

The layer mix:

- range 0, 16
  Xwin
- range 8, 24
  WinterGoddess
- range 17, 32
  Xwin
- range 25, 40
  WinterGoddess
- range 33, 48
  Xwin
- range 41, 56
  WinterGoddess
- range 49, 64
  Xwin
- range 57, 72
  WinterGoddess
- range 65, 80
  Xwin

Acknowledgements

@Xwin-LM For creating Xwin

@Sao10K For creating WinterGoddess

@alpindale For creating the original Goliath

@chargoddard For developing mergekit.

Downloads last month: 7

Safetensors

Model size

118B params

Tensor type

FP16

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for llmixer/Xwinter-120b

Quantizations

2 models