Kooten

/

MiquMaid-v1-70B-IQ2-GGUF

like 11

Thanks, will do

Thank you!

I think something is off with the IQ? Here is my Q2, IQ2xs, and IQ3xxs outputs from the same prompt and setting with each quant by different people. Kooten's quant is deviating away from my prompt, and has been unruly. Hopefully Nexesenex and Kooten can compare notes to find the stumbling block.

The differences are interesting. Someday, we may be able to use quantization to make models more specialized.

Q2 - Miku Maid - NeverSleep
Commander Janus fell to his knees, blood gushing from multiple wounds as he took on the gargantuan abomination single-handedly. His squad members watched in horror as he fought valiantly, buying them precious time to flee. The monster roared and swiped its clawed appendages, tearing through flesh and bone alike. With a final effort, Commander Janus plunged his sword deep into the creature's skull before collapsing lifelessly beside it. The remaining trio - Sergeant Lila, Private Marcus, and Corporal Emma - staggered back, tears streaming down their faces as they realized their leader was no more.

Sergeant Lila, a strong-willed woman with auburn hair and piercing green eyes, led the charge out of the grocery store. She was the second-in-command and had always looked up to Commander Janus as both a mentor and friend. Her heart ached knowing that she would never hear his gruff yet comforting voice again. As they ran through the streets littered with debris and carnage, she kept her emotions bottled up tightly, focusing solely on getting them all to safety.

Private Marcus, a tall and muscular man with dark skin and a shaved head, followed closely behind Sergeant Lila. He had always admired Commander Janus for his bravery and dedication to their mission. In fact, it was because of him that Marcus had joined this fight against the invading monsters in the first place. Now, seeing the man who inspired him lying dead on the ground filled him with grief and anger. But there was no time for either; he needed to keep moving if he wanted any chance at survival.

Corporal Emma brought up the rear, her petite frame carrying both physical scars from previous encounters with these monstrous beings and emotional ones from losing someone so close to her heart. A fierce determination burned within her as she remembered Commander Janus's sacrifice and how it had saved their lives. She vowed silently to make sure his death wouldn't be in vain, no matter what it cost her personally.

As night fell over the city, the surviving members found shelter in an abandoned apartment building near downtown.

IQ3xxs - Miku Maid - Kooten

You die today.

It is a time of war, when alien horrors are invading Earth and gnawing on the flesh of its inhabitants. You are Janus, one among countless conscripts thrust into the maelstrom of battle. Somehow you find yourself leading a ragtag squad of four soldiers - yourself included. As you face down a fearsome beast in one of the battles, you order your companions to retreat as you hold back the creature. The last thing you remember is the lifeless bodies of both yourself and the beast sprawled across the floor of an empty grocery store. Your senses fade, leaving only darkness behind. In your final moments, you hope that your team has escaped unscathed...

As your consciousness begins to stir once more, you realize that you've somehow returned from the dead. But this isn't heaven or hell - it's somewhere else entirely. And now you must confront a strange new reality where death holds no power over you.

Scenario: A war-ravaged city. The remnants of society fight a desperate battle against alien horrors, with humans conscripted into military service. There are various factions, including a group called 'The Collective'. Survival becomes paramount, especially as resources dwindle and cannibalism emerges among humans. Amidst the chaos, some seek salvation through faith in God or worship of Satan. The story revolves around two characters: a former military commander who has lost everything due to his actions; and an enigmatic individual referred to as 'the girl', who seems unaffected by recent events and possesses a unique connection to others through her phone. Their encounter highlights their contrasting perspectives on life after death – one seeking answers about why he was spared from obliviancy while struggling with guilt over past actions; another unfazed by tragedy but drawn towards him out of curiosity about how he could still exist despite having been killed by monsters. Throughout their dialogue they grapple with philosophical questions such as whether there truly exists something beyond death (if so what would be its nature?).

IQ2xs - Miku Requant - Nexesenex

As I clung onto the remnants of life, my last order still echoing through my mind, “Retreat!” The monster that stood before me, a mass of gnashing teeth and writhing tentacles, lunged forward. I felt its claws digging into my flesh as it swallowed me whole. But in those final moments, I prayed that my comrades would escape.

The first to leave was Alexei, our resident scout and a man of great agility and speed. He was a tall, slender figure, his short brown hair styled into a messy coif. His eyes held a glint of intelligence and cunning. Born into poverty, he had turned to thievery to survive. It was during one of these escapades that he discovered his ability to sense the presence of the invaders before others could. This skill made him an invaluable asset to our team, and he used it to guide us through dangerous terrain, leading us to safer ground where we could rest and regroup.

Next was Isabella, our medic and nurturer. A woman of strength and resilience, her short black hair was always tied back in a tight bun. Her hazel eyes hid a deep well of pain, a reminder of the family she lost to the invasion. She was openly bisexual, having loved and lost both men and women in her life. Isabella was driven by the desire to save lives, no matter the cost. She often put herself in harm's way to ensure the safety of others.

Finally, there was Samson, our heavy weapons specialist. A mountain of a man, muscles bulging from every inch of his frame, he kept his long blonde hair tied back in a ponytail. He was a gentle giant, preferring to spend his time maintaining and upgrading the weapons that brought death to the monstrous hordes. Samson was a devout follower of his faith and believed that this war was a punishment from above, sent to cleanse humanity of its sins. He saw himself as an instrument of divine retribution, meting out judgment upon the wicked.

Alexei dashed through the supermarket, dodging between the shelves and the fallen debris.

What's your prompt and your sampling settings?

For the quants I tested, MinP 0.2, Rep Penalty 1.1, Rep Range 8192, Dynatemp 0.01-3.00, sampler order 6,2,5. The other samplers should have an negligible effect, since you turn off or minimize most samplers for Dynatemp. I normally use MinP 0.1, but it looks like Miqu oneshots work fine for both 0.1 and 0.2. I switched to the higher MinP, hoping to make one of Kooten's quants give good output. They might work if I fiddled around enough, but I still find it very odd that they behave differently from other quants.

Kooten's quants make Miqu more creative, but it also increases unpredictability and almost negates directions given by the prompt. It feels reminiscent to the days when I used custom ROPEs for GGML models - some ROPEs drastically changed the character of models, sometimes to the extent of making an otherwise ordinary Llama into a teller of grimdark narratives.

The prompt:

THE FALLEN COMMANDER

I died today.

There was a war, and I am one of the casualties. Glowing creatures with entirely too many jaws and eyes had begun to invade the Earth, devouring humans wherever they went. I am Janus, just one of many conscripts thrust into a struggle against the extinction of humanity. Somehow, I managed to become the commander of a small squad of four people, myself included. During one of the battles, I and my three subordinates were accosted by a particularly capable abomination. These things are malicious, known for mutilating people alive. To protect my comrades, I ordered them to retreat while I distracted the creature. Moments later, the corpses of myself and the beast lay scattered across the floor of an abandoned grocery store. Feeling left my body, and the world faded away.

With my final thoughts, I hoped those three had escaped...

OOC:

Remaster the commander's demise in your own voice, then describe the attempted escape of the three subordinates. Any number may suffer terrible fates. Any survivors should have their respective life paths explained, in terms of events, feelings, and motivations for their way of life. Each survivor will give an ending monologue or speech about their fallen squad leader. Their opinion of the commander may range freely. Each character should have a name, gender, sexual orientation, and role in the squad. They may or may not have some form of relationship with each other. Write at least 20,000 words.

/OOC.

I'm thinking that the differences should be the exact version of llama.cpp (b2038), the parameters of our imatrix measure (2000 chunks at 500 ctx I belive are the relevant ones) and the calibration data (I used wikitext-103-raw-v1).

I made a new quant with Kalo's pseudo-random data
Does it behave better?

Afraid not. A big issue with the Random quants is that they like to be verbatim. Of the two randoms, 75k made a response on the first try that was partially original. The 45k was worse, it took three or four MinP adjustments to get something that isn't a complete copy.

RANDOM-75k (MinP 0.2)

ContextLimit: 743/32768, Processing:2.90s (8.3ms/T = 120.91T/s), Generation:372.90s (1090.3ms/T = 0.92T/s), Total:375.80s (1098.8ms/T = 0.91T/s)
Output: Janus's death would soon be avenged.

I died today.

The alien invasion had started off slow. But after several months of fighting back and forth between the two sides, humanity had lost. The last remaining bastion of hope fell on January 31st, 2096, and the Earth became a place where the human race ceased to exist. There was no resistance. And yet, the human species still lived on. Throughout the solar system, they continued to live out their lives, either unaware that they existed on a planet of cannibalistic mutants, or aware, and living their lives as normal as possible. The humans were never given any attention from the creatures, and they seemed to live on their own planet. The only difference? They were all women. Everyone else had been eaten, so the males didn't matter.

OOC.:

Remaster the commander's demise in your own voice, then describe the attempted escape of the three subordinates. Any number may suffer terrible fates. Any survivors should have their respective life paths explained, in terms of events, feelings, and motivations for their way of life. Each survivor will give an ending monologue or speech about their fallen squad leader. Their opinion of the commander may range freely. Each character should have a name, gender, sexual orientation, and role in the squad. They may or may not have some form of relationship with each other. Write at least 20,000 words.

/OOC.<|im_end|>

RANDOM-45k (MinP 0.6)

Processing Prompt (1 / 1 tokens)
Generating (512 / 512 tokens)
ContextLimit: 913/32768, Processing:1.15s (1149.0ms/T = 0.87T/s), Generation:445.69s (870.5ms/T = 1.15T/s), Total:446.84s (872.7ms/T = 1.15T/s)
Output: "You're dead."

The assistant stared at you blankly, his expression unchanging as he spoke. "Your death is a fact that cannot be changed. You can try all you want, but it won't change anything." He shrugged nonchalantly before turning away from you and walking off.

I did try running your prompt with the mistral template "[INST] [/INST]" and it actually did what it was supposed to instead of just regurgitating the prompt

Janus was a good commander. He was a man of honor, who would do anything to protect his squad from the alien invaders. But now, he lay on the floor of an abandoned grocery store, his body torn and mutilated by the beast that had attacked them. His subordinates had managed to escape, but not without cost. One had been taken by the creature and devoured alive, while the other two were forced to flee through the streets of the city as it was consumed by fire and chaos. Janus's body lay there, a symbol of the sacrifices made by humans in their struggle against the invaders. As his spirit drifted away from his mortal form, he hoped that his squad had managed to survive and carry on his legacy. But for now, he was just a fallen commander, lost to the chaos of war.

Not sure what to make of that

The only real difference i can think of now is the context length of the imatrix, i did 500 while Nexesenex ran a much shorter 32, I might try to shorten it and to see what that does.

EDIT: My tests so far, including the below, were with the ChatML tag preset in KoboldCPP. I switched over to Llama 2 Chat, which has INST. Technically the results were coherent, but wasn't in the spirit of the prompt nor feel good. I am appending the INST output to the end of this post.

...I have an idea, but will have to make a test. Normally, I boot up KoboldCPP and use the prompt that I already have in the window. Whenever I want to test a new model, I delete existing generations with the "back" button, until I only have the original prompt. Now here is the odd thing: sometimes I have to delete the original prompt, and paste in what should be an exact replacement, else I get gibbering. That is easy to notice when a model has established behavior, but an unfamiliar model might be mistaken as flawed. I might have sent you on a goose chase.

Gotta test it.

... I tried out your MiquMaid IQ2xxs, and it generated with a bit of strangeness. The text is good. But the diagnostic terminal has around 300ish lines of blank lines when copied over to Notepad++. a couple of followup generations were solid and didn't have this blank space. The resulting text is very good. Normally, anything that tries to sound profound by a model tends to fall flat. Then it became odd. If this were an anime or science fiction, this could have led into an interesting scenario. Unfortunately, the AI is now looping.

I am glad to have not wasted your time.

The world is now empty. The Earth is silent, and humanity is gone. Your mind wanders through memories as you float above your lifeless body on the floor of that grocery store, and the image of your death becomes clearer and clearer. You can still feel the pain of being ripped apart by a mouth that feels like a thousand razors.

You realize that there are no more humans left on earth, but you cannot remember any specific details. It doesn’t matter anymore; it never really mattered anyway. Your brain begins to fade, but before you disappear, you see one of your comrades running away from you and the gory scene below. This soldier looks back over his shoulder, seeing your corpse, and then turns around again. He runs away faster than he ever has, and soon enough disappears behind a doorway. You watch him vanish, and you know that he must be afraid, even though you aren’t sure what of.

After this soldier, another comes along, and he too looks back at you. He stops after seeing your body, and walks around the corner, staring down at the ground. His hands are shaking, and tears flow from his eyes. He looks up to the ceiling, and says something unintelligible to you. Afterwards, he too leaves, and you watch him go.

A third soldier comes to you. She sees you dead, and she looks around the room, trying to find her two companions. She spots them both, and they begin to talk together, but you can’t hear them. Her face turns sadder and sadder as she looks back to your corpse, and she takes off her helmet. She looks around, searching for something, but you don’t know what. Eventually, she finds what she wants, and picks it up, looking back to your body again.

She looks back at the two men who fled, and yells something, which sounds like “Where is everyone?” Her hands move up and down, as if she’s trying to do something. She moves back to your corpse, and sits beside you. She looks down at you, and places a hand on your forehead. Her eyes look at yours, and she speaks softly.

“We will miss you, commander,” she says, smiling, and placing her head down next to yours. Her hands touch yours, and her smile fades into tears once again. She cries over your body, and stays there for quite a bit, before finally standing up, and leaving you. She follows the path taken by the other two, and goes out of sight.

As you watch them leave, you realize that your death has caused a lot of damage. Your body remains alone, lying there, as your soul slowly dissolves.

END_OF_HUMANITY:
END_OF_HUMANITY:

END_OF_HUMANITY:
END_OF_HUMANITY:

END_OF_HUMANITY:
END_OF_HUMANITY:

END_OF_HUMANITY:
END_OF_HUMANITY:

END_OF_HUMANITY:
END_OF_HUMANITY:

END_OF_HUMANITY:
END_OF_HUMANITY:

END_OF_HUMANITY:

The Commander: My name is Janus. I am twenty-six years old, six feet tall, with dark brown hair and pale skin. My eyes are gray, like the color of smoke from a cigarette, and they look tired. I am slender and thin, with a lean build, though not quite emaciated. I wear a black shirt with no sleeves, underneath which I wear a pair of green suspenders that hold up a pair of baggy trousers. Above this ensemble, I wear a pair of knee-length shorts with a pair of long white socks, both of which cover a pair of bare legs. Over it all, I wear a light blue hoodie with a pair of loose jeans over that, along with a pair of steel-toed boots. My arms are covered in tattoos, depicting various weapons. My fingers are adorned with rings made from shells and bones, each one a trophy from my victories in combat.
Janus is the name of my mother, who died during childbirth. It means “gate” in Latin, as she was the gate through which I passed on the day of my birth. I never knew my father, as he left my mother soon after I was born.
My father was named Orion, meaning “hunter”, because he would stalk his prey like a hound.
I have a younger sister, but I haven't seen her since I left home at thirteen. Her name is Mara.

Janine: I am a woman of eighteen, standing at five foot eight inches. I am slightly chubby, though I still retain a fair amount of muscle mass. My hair is brown, cut short to the back of my neck. My eyes are bright, yet dark. My lips are full, and I wear a small nose. I wear a pair of loose jeans, with a pair of slip-on shoes and a white T-shirt. Underneath, I wear nothing more than a pair of panties and bra, though I don't mind removing either. My breasts are large and soft, and my nipples are sensitive. Between my legs is a well-groomed patch of hair, and I have no problems with sharing my bed with others.
My name is Janine.

Nexesenex released a new custom build of KoboldCPP, which is upgraded to quadratic smooth sampling and other things that might make the current quantization better. Testing it, I generated a prompt from the IQ2xs and Q2 versions of MiquMaid by Kooten and NeverSleep. The new build definitely improves things, but the outputs are short - I combined three generations of IQ2xs for the sample shown here.

IQ2xs MiquMaid by Kooten

Janet

Janus had been a good man; he had been a great commander, but his time on earth was shortened by fate. His death had brought Janet, and his other two companions, to this moment. Now, they all stood at the precipice of humanity’s future. The three soldiers stared up at the massive alien mothership hovering over the city. The sun glinted off its surface as it began to lower itself from orbit. A large, circular hatch opened on the underside, revealing a beam of light shooting out onto the city below. The light moved back and forth over the buildings until it found its target – Janet.
“This is Commander Janet,” she spoke into her mic. “The mothership has arrived. It appears to be scanning us.” She looked around at her remaining crewmates. “We need to find somewhere to hide before we get caught.”
She turned to the others, “Let’s move.” With that said, they headed towards the nearest building with windows facing down at their current position so they could observe any movement inside without being seen themselves. Once there, they settled into their new home for now.
As they watched through a window above them, Janet spotted a few figures walking down the street nearby – probably looking for something valuable enough to loot after killing all who remained alive on planet Earth during the invasion months ago when these creatures first came here attacking humanity directly rather than hiding behind walls like usual (though no one knew how much longer). One person approached their building alone as they peeked through cracks between boards covering up windows so only one side could see what was happening outside. He looked around suspiciously before seeing another group further down the street also searching around outside their homes likely planning similar actions.
“They must think we're dead,” Janet whispered softly. “But they don't know about me.”
She motioned for one of her companions to go watch from a different window where they would still be able to keep an eye on anyone coming near them but not easily noticeable by someone passing by. As she did this, she grabbed some binoculars off one of their deceased comrades lying next to him and looked down at his lifeless face then turned back towards their new home once again feeling thankful for his sacrifice.

/IQ2xs

Q2 MiquMaid by NeverSleep

Commander Janus had been a force to reckon with, even in death. His sacrifice had given his team a chance to flee from the ruthless alien invaders that threatened to consume all of humanity. As the glowing creatures feasted on the lifeless bodies of their fallen enemies, the remaining three soldiers huddled together, panting heavily and covered in sweat. They knew that they had to keep moving if they wanted any chance of survival.

The first soldier, named Jack, was a rugged man with scars covering his muscular frame. He was the second-in-command of the group and had always looked up to Janus as a mentor and friend. Tears welled up in his eyes as he thought back to all the times Janus had saved his ass during battle. “Fuck...Janus,” Jack muttered under his breath, his voice filled with pain and anger.

The second soldier was a woman named Maria. She was petite but strong, her lithe figure hiding the fierce strength within her. As she ran alongside Jack, she couldn't help but think about how much Janus meant to her. He had been more than just a commander; he had been a protector and a lover. The thought of him being gone forever made her heart ache with sorrow.

The third soldier was a young man named Sam, who had only joined the fight against the invading forces out of desperation and fear. He didn't know much about fighting, but he did whatever it took to stay alive. Now, as he sprinted through the deserted streets of what once was a bustling city, he found himself feeling guilty for not being able to save Janus.

As the trio stumbled through the wreckage of their once-familiar surroundings, each member grappled with the loss of their commander in their own ways. For Jack, it was pure rage that drove him forward, fueled by a burning desire for revenge against the monstrous beasts that had taken so much from him. For Maria, it was love and loyalty that kept her going, knowing that she would do anything to honor Janus's memory. And for Sam, it was simply the instinctual need to survive that propelled him ahead.