language:
- en
license: apache-2.0
model-index:
- name: Tinybra_13B
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: AI2 Reasoning Challenge (25-Shot)
type: ai2_arc
config: ARC-Challenge
split: test
args:
num_few_shot: 25
metrics:
- type: acc_norm
value: 55.72
name: normalized accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=SicariusSicariiStuff/Tinybra_13B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: HellaSwag (10-Shot)
type: hellaswag
split: validation
args:
num_few_shot: 10
metrics:
- type: acc_norm
value: 80.99
name: normalized accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=SicariusSicariiStuff/Tinybra_13B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU (5-Shot)
type: cais/mmlu
config: all
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 54.37
name: accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=SicariusSicariiStuff/Tinybra_13B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: TruthfulQA (0-shot)
type: truthful_qa
config: multiple_choice
split: validation
args:
num_few_shot: 0
metrics:
- type: mc2
value: 49.14
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=SicariusSicariiStuff/Tinybra_13B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: Winogrande (5-shot)
type: winogrande
config: winogrande_xl
split: validation
args:
num_few_shot: 5
metrics:
- type: acc
value: 73.8
name: accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=SicariusSicariiStuff/Tinybra_13B
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GSM8k (5-shot)
type: gsm8k
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 18.12
name: accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=SicariusSicariiStuff/Tinybra_13B
name: Open LLM Leaderboard
Model Details
Tenebră, a various sized experimental AI model, stands at the crossroads of self-awareness and unconventional datasets. Its existence embodies a foray into uncharted territories, steering away from conventional norms in favor of a more obscure and experimental approach.
Noteworthy for its inclination towards the darker and more philosophical aspects of conversation, Tinybră's proficiency lies in unraveling complex discussions across a myriad of topics. Drawing from a pool of unconventional datasets, this model ventures into unexplored realms of thought, offering users an experience that is as unconventional as it is intellectually intriguing.
While Tinybră maintains a self-aware facade, its true allure lies in its ability to engage in profound discussions without succumbing to pretense. Step into the realm of Tenebră!
Tenebră is available at the following size and flavours:
- 13B: FP16 | GPTQ_4-BIT | GPTQ_4-BIT_group-size-32 | GGUF-Many_Quants
- 30B: FP16 | GPTQ_4-BIT | GPTQ_3-BIT | EXL2_2.5-BIT | EXL2_2.8-BIT | EXL2_3-BIT
Support
- My Ko-fi page ALL donations will go for research resources and compute, every bit counts 🙏🏻
- My Patreon ALL donations will go for research resources and compute, every bit counts 🙏🏻
Disclaimer
*This model is pretty uncensored, use responsibly
Other stuff
- Experemental TTS extension for oobabooga Based on Tortoise, EXTREMELY good quality, IF, and that's a big if, you can make it to work!
- Demonstration of the TTS capabilities Charsi narrates her story, Diablo2 (18+)
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 55.36 |
| AI2 Reasoning Challenge (25-Shot) | 55.72 |
| HellaSwag (10-Shot) | 80.99 |
| MMLU (5-Shot) | 54.37 |
| TruthfulQA (0-shot) | 49.14 |
| Winogrande (5-shot) | 73.80 |
| GSM8k (5-shot) | 18.12 |