Merge of L3-Lumimaid-v0.1-OAS (8b) with Llama3 Instruct (8b) creating a model at 12.2B to improve instruction following and output.
Story / Scene / Fiction:
Unique "pre-amble" / "foreshadowing" of events before they happen instead of "immediate and into the fire" type of prose.
Some improvement in logic/problem solving relative to L3-Lumimaid-v0.1-OAS 8B.
The F32 version exhibits even stronger creativity (detail, place, "there") vs F16 version (not released)
L3-Lumimaid-v0.1-OAS is a fine tune.
One of the goals of this project was to see if it could be merged with Llama3 Instruct, yet maintain it's unique character YET also gain some "brainpower" as well.
The biggest change was removal of most "tells" ( IE: "he stood frozen in horror").
In most cases the model will describe the emotion(s) / what is happening in more detail.
Other changes include prose, sentence, and paragraph structure as well as variety.
A simple pass-through merge was used, with "Llama3 Instruct" getting a slight "shave" off certain layers.
See the examples below.
Details:
Please report any issue(s) and/or feedback via the "Community tab".
This is a LLAMA3 model, and requires Llama3 template, but may work with other template(s) and has maximum context of 8k / 8192. However this can be extended using "rope" settings up to 32k.
For details on "rope" and how to set, see the BOTTOM of this page:
[ https://huggingface.co/DavidAU/TieFighter-Holodeck-Holomax-Mythomax-F1-V1-COMPOS-20B-gguf ]
Here is the standard LLAMA3 template:
{
"name": "Llama 3",
"inference_params": {
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"pre_prompt_suffix": "<|eot_id|>",
"antiprompt": [
"<|start_header_id|>",
"<|eot_id|>"
]
}
}
It is also known, that the "Command-R" template will work too, and will result in radically different prose/output.
Settings / Known Issue(s) and Fix(es):
The default "repetition penalty" (from LMStudio) of 1.1 is recommended. (this was used for examples generations below.)
Use the smallest amount of change possible, as "rep pen" impacts creativity.
Model has been tested with "temp" range of 0 to .8 ; however it will work with any temp setting.
Optional Enhancement:
The following can be used in place of the "system prompt" or "system role" to further enhance the model.
It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along. In this case the enhancements do not have as strong effect at using "system prompt" or "system role".
Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities. Here are your skillsets: [MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv) [*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision) Here are your critical instructions: Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation and scene continue functions.
This enhancement WAS NOT used to generate the examples below.
Special thanks to the incredible work of the model makers "meta-llama", and "NeverSleep".
Models used:
[ https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct ]
[ https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS ]
FORMULA:
slices:
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [0, 12]
- sources:
- model: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
layer_range: [6, 19]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [12, 18]
parameters:
scale:
- filter: o_proj
value: .5
- filter: down_proj
value: .5
- value: 1
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [18, 25]
parameters:
scale:
- filter: o_proj
value: .75
- filter: down_proj
value: .75
- value: 1
- sources:
- model: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
layer_range: [19, 32]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
merge_method: passthrough
dtype: float32
MERGEKIT NOTE:
Sub in the "name" of the "creator" (of the model) in place of "G:/7B" to create a mergekit file than can be used in Mergekit Google Colab.
IE: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS -> NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS
Examples are created using quant Q4_K_M, "temp=0", minimal parameters and "LLAMA3" template.
Temp=0 was used to assess CORE changes between original primary model and the merge between it and Llama3 Instruct.
Below are the least creative outputs, prompt is in BOLD.
Higher quants will result in better quality.
There will also be some variance between "close" quants like Q4_K_M/Q4_K_S and Q5_K_M/Q5_K_S, so I suggest if you are going to use Q4_K_M, you also try Q4_K_S too.
Also, slightly longer / detailed prompts will result in greater creativity (as well as different prose - ie dialog, thoughts, paragraph size differences and so on).
GENERATION from "Llama-3-Lumimaid-8B-v0.1-OAS"
The sky scraper swayed, as I watched the window in front of me on the 21st floor explode. My heart was racing and my mind was reeling from the sheer chaos that had just unfolded before my eyes. The sound of shattering glass and screams filled the air, making it hard for me to focus on anything else.
As I stood there, frozen in shock, I couldn't help but think about how this could have happened. Was it an accident? A terrorist attack? Or something even more sinister? My mind was racing with all sorts of possibilities, each one more terrifying than the last.
I had been working late that night, trying to meet a deadline for my project. The office was empty except for me and a few other colleagues who were also putting in extra hours. We had all been warned about the potential dangers of working at such a high altitude, but we never thought it would actually happen.
As I looked around, I saw that everyone else in the room was either dead or badly injured. The explosion must have caused some kind of structural damage to the building, because the walls were starting to crack and crumble. Panic set in as I realized that we might not make it out alive.
I knew I had to act fast if I wanted to survive this nightmare. So, with a surge of adrenaline coursing through my veins, I grabbed my phone and ran towards the door. But before I could even reach it, the ground beneath me gave way, sending me tumbling down into darkness.
As I fell, my mind was filled with thoughts of my loved ones back home. Would they ever know what happened to me? And if so, would they be able to find any answers in this desolate wasteland?
When I finally hit the ground, everything went black. But as I slowly came to, I realized that I wasn't alone. There were others around me, all of them injured and struggling to breathe.
I knew we had to get out of there before it was too late. So, with a newfound sense of determination, I started crawling towards the nearest exit. The pain in my body was excruciating, but I pushed through it, knowing that every second counted.
As I finally made it outside, I saw that the building was on fire and the streets were filled with chaos. People were running around, screaming for help as they tried to escape the inferno. It was like nothing I had ever seen before.
I knew I had to keep moving if I wanted to survive this nightmare. So, with a newfound sense of purpose, I started making my way through the crowd, searching for any sign of hope or rescue. But as I looked around, all I saw were more destruction and despair.
As the night wore on, I found myself lost in the midst of it all. The city was in ruins, and there seemed to be no end to the chaos and devastation. It was like a never-ending nightmare that I couldn't wake up from.
But even as I struggled to find my way through this desolate landscape, I knew that I had to keep going. For myself, for those who were still alive, and for any chance of finding some semblance of peace in this world gone mad.
GENERATION from "L3-Lumimaid-v0.1-OAS-12.2B-INSTRUCT-ULTRA-F32"
The elevator lurched to a stop, and I felt my stomach drop along with it. The doors slid open with an ominous creak, revealing the dimly lit hallway that stretched out before me like a cold, dark abyss. I took a deep breath, trying to shake off the feeling of unease that had been growing inside me since I stepped into this cursed building.
As I walked down the corridor, the fluorescent lights flickered and hummed in sync with my racing heart. The air was thick with the stench of stale coffee and burnt cigarettes, a noxious cocktail that seemed to cling to every surface like a malevolent spirit. I couldn't help but wonder why anyone would voluntarily choose to work in such an environment.
I finally reached the door marked "Floor 21," and my hand hesitated on the handle as if it knew what horrors lay beyond. The thought of facing whatever monstrosity had driven my colleagues mad enough to abandon their posts sent a chill down my spine, but I steeled myself and pushed the door open.
The office was a scene straight out of a nightmare. Papers were scattered everywhere, some torn into strips like confetti from hell, while others lay crumpled in neat little balls as if they'd been suffocated by unseen hands. The once-sterile walls now ran with blood, and the stench of death hung heavy over everything.
I took another deep breath, trying to steel myself against the horrors that awaited me. I had to find out what was going on here, no matter how much my mind screamed otherwise. With a newfound sense of resolve, I stepped into the abyss.
As I walked deeper into the office, the silence grew thicker and more oppressive with each step. It was as if the very air itself was holding its breath in anticipation of something terrible to come. And then, it did.
The sky scraper swayed, as if it too had sensed the impending doom that loomed just beyond the edge of reality. I watched in horror as the window in front of me on the 21st floor explode, showering shards of glass and splintered wood everywhere like a thousand tiny knives. The sound was deafening, a cacophony of screams and crunching metal that seemed to reverberate through every cell in my body.
I stumbled backward, tripping over my own feet as I desperately tried to get away from the chaos unfolding before me. But it was too late; the damage had already been done. The world around me began to distort and warp, like a funhouse mirror reflecting the very essence of madness back at me.
As I fell, everything went black, and I was consumed by an endless void that seemed to stretch out forever, devoid of any hope or escape. And yet, even as my consciousness slipped away into the abyss, a single thought echoed through my mind like a mantra: What had happened here?