Upload folder using huggingface_hub

eaf84a6 verified 19 days ago

6.36 kB

	---
	license: apache-2.0
	library_name: transformers
	tags:
	- general-purpose
	- roleplay
	- storywriting
	- merge
	- finetune
	base_model: mistralai/Mistral-Nemo-Base-2407
	model-index:
	- name: Chronos-Gold-12B-1.0
	results:
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: IFEval (0-Shot)
	type: HuggingFaceH4/ifeval
	args:
	num_few_shot: 0
	metrics:
	- type: inst_level_strict_acc and prompt_level_strict_acc
	value: 31.66
	name: strict accuracy
	source:
	url: >-
	https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: BBH (3-Shot)
	type: BBH
	args:
	num_few_shot: 3
	metrics:
	- type: acc_norm
	value: 35.91
	name: normalized accuracy
	source:
	url: >-
	https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: MATH Lvl 5 (4-Shot)
	type: hendrycks/competition_math
	args:
	num_few_shot: 4
	metrics:
	- type: exact_match
	value: 4.38
	name: exact match
	source:
	url: >-
	https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: GPQA (0-shot)
	type: Idavidrein/gpqa
	args:
	num_few_shot: 0
	metrics:
	- type: acc_norm
	value: 9.06
	name: acc_norm
	source:
	url: >-
	https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: MuSR (0-shot)
	type: TAUR-Lab/MuSR
	args:
	num_few_shot: 0
	metrics:
	- type: acc_norm
	value: 19.42
	name: acc_norm
	source:
	url: >-
	https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: MMLU-PRO (5-shot)
	type: TIGER-Lab/MMLU-Pro
	config: main
	split: test
	args:
	num_few_shot: 5
	metrics:
	- type: acc
	value: 27.98
	name: accuracy
	source:
	url: >-
	https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
	name: Open LLM Leaderboard
	---

	# Chronos Gold 12B-1.0

	![image/webp](https://cdn-uploads.huggingface.co/production/uploads/630417380907b9a115c6aa9f/3hc8zt8fzKdO3qHK1p1mW.webp)

	Chronos Gold 12B 1.0 is a very unique model that applies to domain areas such as
	general chatbot functionatliy, roleplay, and storywriting. The model has been observed to write up to 2250 tokens in a single sequence. The model was trained at a
	sequence length of 16384 (16k) and will still retain the apparent 128k context length from Mistral-Nemo, though it deteriorates over time like regular Nemo does based on
	the [RULER Test](https://github.com/hsiehjackson/RULER?tab=readme-ov-file#-ruler-whats-the-real-context-size-of-your-long-context-language-models)

	As a result, is recommended to keep your sequence length max at 16384, or you will experience performance degredation.

	The base model is `mistralai/Mistral-Nemo-Base-2407` which was heavily modified to produce a more coherent model, comparable to much larger models.

	Chronos Gold 12B-1.0 re-creates the uniqueness of the original Chronos with significiantly enhanced prompt adherence (following), coherence, a modern dataset, as well as supporting a majority of "character card" formats in applications like SillyTavern.

	It went through an iterative and objective merge process as my previous models and was further finetuned on a dataset curated for it.

	The specifics of the model will not be disclosed at the time due to dataset ownership.

	## Instruct Template

	This model uses `ChatML` - below is an example. It is a preset in many frontends.

	```
	<\|im_start\|>system
	A system prompt describing how you'd like your bot to act.<\|im_end\|>
	<\|im_start\|>user
	Hello there!<\|im_end\|>
	<\|im_start\|>assistant
	I can assist you or we can discuss other things?<\|im_end\|>
	<\|im_start\|>user
	I was wondering how transformers work?<\|im_end\|>
	<\|im_start\|>assistant
	```

	## Quantization

	#### LlamaCPP
	[GGUFs from @bartowski](https://huggingface.co/bartowski/Chronos-Gold-12B-1.0-GGUF)

	#### Exllama2
	[4.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-4.5bpw-h6-exl2)

	[5.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-5.5bpw-h6-exl2)

	[6.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-6.5bpw-h6-exl2)

	[8.0bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-8.0bpw-h8-exl2)

	#### FP8
	[FP8 Quant by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-FP8)

	## Sampling Settings
	Nemo is a bit sensitive to high temperatures, so I use lower. Here are my settings:
	```
	Temp - 0.7 (0.9 max)
	Presence Penalty - 1.0
	Repetition Penalty range - 2800
	Min P - 0.10
	```

	## Additional Details
	This model was created by elinas on discord. Thank you to [@kalomaze](https://huggingface.co/kalomaze) for providing a model that made this merge possible.

	This is one of multiple models to come out in the series by size and model architecture, so look forward to it!

	Contact me on Discord for inquiries.
	# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
	Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_elinas__Chronos-Gold-12B-1.0)

	\| Metric \|Value\|
	\|-------------------\|----:\|
	\|Avg. \|21.40\|
	\|IFEval (0-Shot) \|31.66\|
	\|BBH (3-Shot) \|35.91\|
	\|MATH Lvl 5 (4-Shot)\| 4.38\|
	\|GPQA (0-shot) \| 9.06\|
	\|MuSR (0-shot) \|19.42\|
	\|MMLU-PRO (5-shot) \|27.98\|