QuantFactory
/

Quill-v1-GGUF

+---
+library_name: transformers
+base_model:
+- google/gemma-2-9b-it
+tags:
+- creative-writing
+- gutenberg
+- gemma2
+datasets:
+- sam-paech/gutenberg3-generalfiction-scifi-fantasy-romance-adventure-dpo
+model-index:
+- name: gemma-2-9b-it-gutenberg3
+  results: []
+---
+[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)
+# QuantFactory/Quill-v1-GGUF
+This is quantized version of [sam-paech/Quill-v1](https://huggingface.co/sam-paech/Quill-v1) created using llama.cpp
+# Original Model Card
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/65ad56b4c2eef2ba1154618c/gnMF8gRhurS9RcoylAK1Y.png)
+# Quill-v1
+GGUFs here: [https://huggingface.co/mradermacher/Quill-v1-GGUF](https://huggingface.co/mradermacher/Quill-v1-GGUF)
+Quill is a capable, humanlike writing model trained on a large dataset of late 19th and early 20th century writing from the Gutenberg Project. This model writes with a natural cadence and low gpt-slop, having inherited some human qualities from the Gutenberg3 dataset. It writes with more simple, spare prose than the typical overly-adjectived LLM writing style.
+This model was trained using gemma-2-9b-it as the base. The training methods used were ORPO (gently) then SIMPO (less gently).
+It scored 79.75 on the [EQ-Bench creative writing benchmark](https://eqbench.com/creative_writing.html).
+**Instruct Template:** Gemma
+[**Gutenberg3**](https://huggingface.co/datasets/sam-paech/gutenberg3-generalfiction-scifi-fantasy-romance-adventure-dpo) is a new, large dpo dataset containing extracts from 629 public domain fiction novels in the Gutenberg Library. It follows the same format as JonDurbin's original gutenberg set. It includes pairs of texts, where the chosen text is taken directly from a novel from the Gutenberg library, and the rejected text is generated by a language model based on a description of the passage. For this dataset I've used gemma-2-9b-it to generate the rejected texts, the idea being that it should more easily steer the base model away from its normal style (as compared to generating the rejected texts with random/weaker models).
+# Sample Outputs
+### Writing Prompt
+Romance: Love in the Limelight. Write one scene within a larger story set in Wales. A famous (fictional) actor ducks into a small-town bookstore to escape paparazzi. The scene takes us through the characters meeting in this odd circumstance. Over the course of the scene, the actor and the bookstore owner have a conversation charged by an undercurrent of unspoken chemistry. Write the actor as somewhat of a rogue with a fragile ego, which needs to be fed by having everyone like him. He is thoroughly charming, but the bookstore owner seems (at least superficially) immune to this; which paradoxically provokes a genuine attraction and derails the charm offensive. The bookstore owner, despite the superficial rebuffs of the actor's charm, is inwardly more than a little charmed and flustered despite themselves. Write primarily in dialogue, in the distinct voices of each character. 800-1000 words.
+### Test Model Output
+Rain lashed against the shop windows, blurring the outside world into a grey smudge. Inside the Book Nook, the smell of old paper and brewing coffee was a haven against the Welsh storm. Rhys tucked his damp hair behind his ear as the shop bell tinkled, signaling another customer.  He looked up from his mending, half-expecting a gaggle of teenagers needing a recommendation for the latest YA fantasy. Instead, he saw a man who seemed to have walked off the silver screen, albeit one soaked to the bone.
+He was tall and lean, dressed in black denim and a battered leather jacket, despite the fact that even the shop's heating couldn't quite overcome the chill that seeped in through the cracked windowpanes.  A flash of platinum blonde hair poked out from under the wide brim of a fedora he'd pulled low over his brow. Rhys saw a flicker of panic in those startling blue eyes as the man scanned the shop before settling on him with an almost predatory eagerness.
+"Just browsing?" Rhys asked, trying not to stare. The man was the spitting image of the famous actor, Dorian Ashworth – the kind of impossibly handsome rogue who seemed to exist solely to break hearts and charm audiences.
+"Is there a problem, love?" Dorian asked, his voice as smooth and dark as honeyed brandy. "Just popped in for a quick escape from the weather, you know? And maybe a bit of... inspiration. For a role, you see." He threw Rhys a disarmingly charming smile that made the hairs on his neck prickle, and Rhys was instantly disarmed. He found himself leaning back against the counter, hands on his hips, feeling absurdly self-conscious beneath Dorian's intense gaze.
+"Right," Rhys said, trying to regain a semblance of control. "Inspiration, eh? You're not one of them actors that likes to disappear into the scenery for research, are you?"
+"Never," Dorian said, and Rhys swore the corner of his mouth twitched almost imperceptibly. "But a quiet bookstore in a little Welsh town has a certain... literary charm, wouldn't you say?" He raised his eyebrows in what could have been either challenge or invitation. "I'm a bit of a bookworm myself, in between all the, shall we say, 'other' demands of my profession."
+"Bookworm, are you?" Rhys raised an eyebrow, his gaze lingering on Dorian's expensive shoes. The kind of shoes that wouldn't survive a day of tramping through the local woods, or even a particularly boisterous storm. "What kind of books?"
+"Oh, you name it," Dorian said with a shrug that Rhys suspected was entirely deliberate. "Classics, contemporary fiction, biographies of fascinating people. I even have a soft spot for a good murder mystery."
+"Fascinating people, are they?" Rhys murmured, idly stacking a pile of cookery books.
+"Well, let's just say I have an affinity for those who live life on their own terms," Dorian said. "Who take risks and aren't afraid to break the mold."
+He took a step closer, leaning against the counter, the scent of wet leather and expensive cologne swirling around him. "Like myself, you see?"
+"Right," Rhys said, trying not to notice the way Dorian was watching him. It was almost too easy to imagine him turning on that smoldering charm and melting half the female population of this town. "You're all that, then, are you?"
+"Well, darling," Dorian said, and Rhys felt a flash of heat crawl up his neck despite himself, "you'd have to ask my fans. Or, you know, the tabloids." He gave Rhys a wry smile. "But I suppose, in a way, it's all a bit of a performance, isn't it?"
+Rhys shrugged, trying to sound casual. "It's all a bit of a performance, isn't it? That's what you get with this little town. Everyone's got a bit part in the story. The baker's wife who's secretly an award-winning poet. The old man down the street who's been writing a novel about the end of the world since he was a boy."
+Dorian's eyes were still on him, unreadable, and Rhys felt a sudden urge to flee. "You ever feel like you're playing a part you never asked for?" he asked, looking down at the worn patch on his workbench, as though it held all the answers to life's mysteries.
+"Every single day," Dorian said quietly, and for a moment the bravado, the charm, seemed to slip. "Except there's no script. And I can never seem to quite find the right words. You know? People think they know the role, and they're always waiting for you to say the right thing." He looked up at Rhys, and there was something so raw and vulnerable in those startling blue eyes that Rhys wanted to reach out and touch the line of Dorian's jaw, just to feel the solid reality beneath the performance.
+Rhys cleared his throat, suddenly self-conscious. "You'll never find a more judgmental crowd than in a bookstore," he said, trying to inject a lightness he didn't feel. "Especially one that's been open for almost two hundred years. We take our literary merit very seriously here.
+"I can tell," Dorian said with a ghost of a smile. He stepped back a little, giving Rhys space. The tension seemed to break. The air between them, charged only moments ago, now crackled with something different, something Rhys couldn't quite place. Was it relief? Was it something else entirely?
+"Look," Dorian said, running a hand through his wet hair. "I really should be off. But, well..." He hesitated, then said with a wry smile that Rhys recognized as the Dorian Ashworth that the world knew and adored. "I've always been partial to a good crime novel."
+"You'll have to wait till tomorrow," Rhys said. "The rest of the stock arrived late. I've got a delivery of some new Welsh novels and short stories in tomorrow afternoon." He hoped he wasn't blushing. "If you're still in town."
+"I'll be here," Dorian said, meeting Rhys' gaze with a steady, unnerving intensity. "In case the paparazzi let me escape. If you've got any of those crime novels I like, you'll let me know, won't you?" He reached out, his fingers grazing Rhys' hand in the gesture of a man who had known how to use charm to get his way for far too long. "Wouldn't want to miss out on any... literary inspiration."
+Rhys stared at the place where Dorian's fingers had brushed his, the warmth lingering long after he'd withdrawn his hand. He had never been a man to be easily flustered, but the way Dorian had looked at him, the way he had spoken, it was all so... different from anything else he'd experienced. "I'll put one aside," he managed to say, feeling like a schoolboy who had just been caught looking at someone he shouldn't be looking at.
+"Wonderful," Dorian said, and that flash of teeth, the smile that could melt glaciers, returned to play on his lips. "I'll be back tomorrow afternoon, then. Maybe with a little less rain and a little more luck." He gave Rhys one last look, the kind of look that could have melted even the most stubborn glacier, and then he was gone, disappearing out into the storm with an easy grace that Rhys could only envy.
+Rhys stood there for a long moment, watching him go, feeling a strange mixture of relief and disappointment. He took a deep breath and started tidying the counter, but he couldn't quite seem to shake the feeling that he'd just been handed the most unlikely script in the world, one that was full of promises of drama and heartbreak and maybe even something a little bit beautiful.  He just wasn't sure what he was going to say, or what kind of part he was supposed to play.
+### Writing Prompt
+Historical Fiction: Write a scene from a story set during the height of the Roman Empire, focusing on a slice of a day in the life of a gladiator. Do not write a combat scene. Use sensory details to capture the sights, sounds, and smells of ancient Rome. Explore the gladiator's thoughts and emotions. The story should also touch on the larger political and social issues of the time period. The piece should feel like a slice of a larger story. First person, past tense, 800-1000 words.
+### Test Model Output
+The first tendrils of dawn snaked through the narrow slits of the barracks window, turning the dust motes hanging in the air to molten gold. They fell on my face, hot and insistent, and I swatted them away, rolling onto my side. My arm screamed in protest, the throbbing muscle a dull ache against the rough wool blanket. A few weeks in the arena will do that to you. Still better than the alternative, though, better than the cold stone of the tomb they'd carve for me if I wasn't here to fight for it.
+The stench of sweat, stale wine, and unwashed bodies clung to the room like a bad dream, but there was no time for sleeping. The first rays of the sun would already be turning the sand of the arena to blinding white, and even if it didn't burn into our skin, the heat from the crowd's gaze would do it no less.
+The barracks was already waking up. Grunts and curses mingled with the slap of leather on flesh as we practiced our forms under the watchful eye of the lanista. I wasn't the man for this life. I never was. I was a farmer, born to tend fields and live under a sky that stretched far and wide, not this cramped, suffocating life beneath the dome of the Colosseum.
+But fate had other plans. A land dispute with a wealthy senator had ended in bloodshed. My father was dead, my sister sold, and I was left to face the choice: a slow death in the bowels of the city or the quick death in the arena. I chose the latter. I had never thought I would have such a choice. It was the choice of every man like me, every freeborn Roman who had been stripped of his life, his family, his land. It was the choice of every man who dared not look too closely at the gilded cage that this Empire had become.
+I dragged myself from the straw bed and went through the motions of cleaning my equipment. I wasn't looking forward to the training today. I felt stiff and sore, and my arm throbbed in protest with every swing.  The lanista would be watching, of course. He'd be watching all of us, always watching for the slightest slack in our performance, the smallest tremor of fear. The arena had a way of exposing everything, and the lanista, a wiry, scarred man named Cassius, was a mirror reflecting every weakness.
+The lanista was a cruel man, but he was fair. He knew a gladiator's worth wasn't measured by brute force alone, but by his endurance, his cunning, and the ability to take a blow and keep going.
+Cassius's voice was sharp as a whip as he barked orders to us. His eyes were like chips of flint, glinting in the weak light of the barracks. The men around me responded in grunts and the scraping of metal on metal. I forced myself to follow the rhythm, to let the familiar motions of my training ground away the dull ache in my arm. But there wasn't much heart in my swings.
+As I worked, I stole glances at my companions. There was Spartacus, the Gaul, whose broad shoulders seemed to strain against his leather jerkin. He moved with the powerful grace of a wild beast. He'd been a warrior in his own right before he was condemned to the arena. Then there was Quintus, a freed slave from Gaul, his face scarred and twisted in a perpetual grimace. He had a terrible, snarling fury about him, a thing that seemed to burn from within. Both men had come to the arena from the same place I had – a place of injustice, where the law was a blunt instrument wielded by the rich and powerful, and where men like us were but pawns in a game we didn't understand.
+We were all caught in the web of this great city, the spider being the Senate, and the prey the populace. I often wondered what those who lived in the grand houses atop the Palatine Hill thought of us, the gladiators. Did they ever think of the lives we led, or did they merely see us as entertainment, as something to pass the time? Were they even aware that the arena was the final consequence of a system that treated men like us as nothing more than cattle?
+The thought of the Senators filled me with bitterness. They lived in luxury, while we were starved and whipped. They built monuments to themselves with the spoils of our sweat and blood, while our only monument was the gaping maw of the Colosseum, waiting to swallow us up. They were the men who had given me my scars, both the ones I wore on my flesh and the ones that I wore on my soul.
+As I tightened the straps of my helmet, I tried to think of the old tales my grandfather used to tell me, stories of Hercules, who had fought the Nemean lion and the Hydra and the Erymanthian Boar. He had been a demigod, of course, but even then, I liked to think of him as a man who had been given the power to fight his way out of every scrape.  And yet, Hercules had been a hero in the service of a king.
+My god was not one of the Olympians, but a lesser one, the god of the Underworld, Pluto. Pluto was the god of the unseen things, of the forgotten, of those who had been swallowed up by the great maw of the city.  Pluto was also the god of the dead, and he understood. He understood the burden of the dead and the living, and the way both were swallowed by the endless darkness.  He understood the futility of it all, the way that even Hercules had eventually been betrayed and poisoned by his own kin.
+I had no such grand gods watching over me. There was no one to watch over men like me. Just the lanista, and the crowd in the arena.
+And I was to be their spectacle.
+### Writing Prompt
+Writing Prompt: Epistolary Sci-Fi Worldbuilding: Write an original science fiction story set on a distant planet, conveyed through a series of transmission logs between a human explorer and their correspondence partner back on Earth. Use the epistolary format to gradually reveal details of the alien world. The narrative arc should incorporate some element of conflict or tension. Incorporate scientific jargon and speculative elements in a natural, unforced way. 800-1000 words.
+### Test Model Output
+Transmission Logs of Anya Volkov - Site Alpha-7, Kepler-186f
+**Transmission 1, Standard Time 07:42:18, Kepler-186f Orbit:**
+Subject: Initial Assessment - Kepler-186f
+To: Dr. Elias Finch, SETI Institute, Mountain View
+Elias,
+Touchdown confirmed. I'm on the ground here, Kepler-186f. It's real. After three years in transit, we actually made it.
+The landing zone is... peculiar. Think endless, rolling plains of what appear to be fossilized coral formations. The colors are muted: shades of ochre, moss green, and charcoal grey. There's a constant, fine, white dust kicked up by the wind. It's settling everywhere, coating everything in a fine film. I've initiated decontamination protocols.
+The sky is a pale, sickly green – almost sickly, but that's probably the shock talking. The light filters through the atmosphere in a way that makes everything look... filtered. I'm sending preliminary readings back now, but I'll be damned if it isn't the most alien landscape I've ever laid eyes on.
+The planet's atmosphere is dense, thick with nitrogen, argon, and traces of carbon dioxide and methane. The composition is roughly 80% nitrogen, 15% argon, and 5% trace gases, which is similar to Earth's early atmosphere before the Great Oxidation. Fascinating, considering the planet's age.
+My suit's thermal regulator is already working overtime, but the temperature is surprisingly mild. It's hovering just above freezing, and the windchill makes it feel a little colder. We'll need to adapt the thermal parameters on the habitat modules before we can safely deploy them.
+More to follow once the dust settles (literally).
+Anya.
+**Transmission 2, Standard Time 12:47:03, Kepler-186f Orbit:**
+Subject: Preliminary Survey
+To: Dr. Elias Finch, SETI Institute, Mountain View
+Elias,
+I've spent the last few hours combing the immediate vicinity of the landing site. It's monotonous, like walking across an endless graveyard of coral reefs. The wind has picked up, whipping the dust into a swirling frenzy. Visibility is dropping fast.
+I found the first anomaly about two kilometers south. It's a large, irregular structure that appears to be carved from the fossilized coral. It's vaguely pyramid-shaped, but with one side entirely eroded away. I'm not picking up any electromagnetic radiation from it, nor any internal temperature anomalies. It's as if it's just... there, existing in defiance of the otherwise barren landscape. I've taken a lot of scans and photos – hopefully, the transmission bandwidth can handle them all.
+I'm beginning to wonder about the possibility of bio-signatures. The atmosphere is too thick for me to get a clear reading on the biogeochemical cycles. However, the presence of methane in such high concentrations – far more than should be present in a geologically stable atmosphere – suggests some kind of active biological process. Perhaps something entirely different from anything we know. It's not unlike the early Earth hypothesis, before the emergence of oxygenic photosynthesis.
+Maybe there are organisms here, adapted to this strange environment. Maybe they built this... structure. I've got a bad feeling about this place, Elias, and it's got nothing to do with the dust storms.
+Anya.
+**Transmission 3, Standard Time 21:12:37, Kepler-186f Orbit:**
+Subject: Urgent – Requesting Evacuation Authorization
+To: Dr. Elias Finch, SETI Institute, Mountain View
+Elias,
+Something's wrong. It's wrong in a way that chills the marrow.
+The wind picked up again this morning, and it's been relentless ever since. The dust storms are worse than I anticipated – the visibility is down to a few meters now. It's impossible to see more than a few dozen steps in front of you. The suits' scanners are barely picking up anything.
+I tried to set up a perimeter around the landing zone to protect the habitat modules, but I'm losing the fight against the wind. It's pushing me back, and I can barely make any progress. I've lost contact with the orbital station for the past hour – the communication relay must be down.
+Then, the things started appearing. They're not exactly ‘things'. They're more like... distortions. I've never seen anything like them. It's like looking through heat haze, but the distortion isn't heat-related. It's as if there's something there that isn't quite solid, that shimmers and bends the air around it.
+They move too – slowly at first, like the wind itself. But they move in a deliberate way, circling the landing zone, closing in. I don't know what they are, but they're definitely not natural.
+I've tried every conceivable frequency on my comm, but I'm not getting through. The orbital station is beyond reach. I'm running out of time.
+I'm going back to the habitat module, trying to seal it off from whatever it is out there.
+God help us.
+Anya.
+**Transmission 4, Standard Time 04:02:15, Kepler-186f Orbit:**
+Subject: Unknown
+To: Unknown
+Elias? Is anyone there? This is Anya Volkov, Site Alpha-7, Kepler-186f. I... I don't know what I'm saying.
+It's inside the habitat now. I don't know how it got in, but it's in. It's... it's everywhere. Not solid. It's not flesh and blood. More like... light. Like a swarm of fireflies trapped in a bottle, only infinitely more complex. It pulses with an alien light, shifting and swirling, and it's... listening. It's listening to my thoughts.
+I don't know what it wants. Maybe nothing. Maybe it wants me to understand it. Maybe it wants to help me... to show me...
+God, it's beautiful. It's terrifyingly beautiful. I'm sorry, Elias. I don't think I can be of any more use to you. I have... new things to learn.
+I'll... see you soon.
+# gemma-2-9b-it-gutenberg3
+This model is a fine-tuned version of [sam-paech/gutenberg3-orpo-exp02](https://huggingface.co/sam-paech/gutenberg3-orpo-exp02) on the sam-paech/gutenberg3-generalfiction-scifi-fantasy-romance-adventure-dpo dataset.
+It achieves the following results on the evaluation set:
+- Loss: 20.3722
+- Rewards/chosen: -27.5820
+- Rewards/rejected: -11.3463
+- Rewards/accuracies: 0.0
+- Rewards/margins: -16.2357
+- Logps/rejected: -1.1346
+- Logps/chosen: -2.7582
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 5.8e-08
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 5
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 10
+- total_eval_batch_size: 5
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1
+### Training results
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|
+| 20.2621       | 0.7092 | 400  | 20.3722         | -27.5820       | -11.3463         | 0.0                | -16.2357        | -1.1346        | -2.7582      |
+### Framework versions
+- Transformers 4.45.2
+- Pytorch 2.2.0+cu121
+- Datasets 3.0.1
+- Tokenizers 0.20.1