Merge of SthenoMaidBlackroot (8b) with Llama3 Instruct (8b) creating a model at 12.1B to improve instruction following and output.
Story / Scene / Fiction:
Unique "pre-amble" / "foreshadowing" of events before they happen instead of "immediate and into the fire" type of prose.
Some improvement in logic/problem solving relative to SMB 8B.
NOTE: An "Ultra" version transfer at Float32 will appear in separate repo.
The F32 version exhibits even stronger creativity (detail, place, "there") , but slightly less logic/problem solving relative to the F16 version.
SthenoMaidBlackroot is a 3 model merge and fine tune.
One of the goals of this project was to see if it could be merged with Llama3 Instruct, yet maintain it's unique character YET also gain some "brainpower" as well.
The biggest change was removal of most "tells" ( IE: "he stood frozen in horror").
In most cases the model will describe the emotion(s) / what is happening in more detail.
Other changes include prose, sentence, and paragraph structure as well as variety.
A simple pass-through merge was used.
See the examples below.
Details:
Please report any issue(s) and/or feedback via the "Community tab".
This is a LLAMA3 model, and requires Llama3 template, but may work with other template(s) and has maximum context of 8k / 8192. However this can be extended using "rope" settings up to 32k.
For details on "rope" and how to set, see the BOTTOM of this page:
[ https://huggingface.co/DavidAU/TieFighter-Holodeck-Holomax-Mythomax-F1-V1-COMPOS-20B-gguf ]
Here is the standard LLAMA3 template:
{
"name": "Llama 3",
"inference_params": {
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"pre_prompt_suffix": "<|eot_id|>",
"antiprompt": [
"<|start_header_id|>",
"<|eot_id|>"
]
}
}
It is also known, that the "Command-R" template will work too, and will result in radically different prose/output.
Settings / Known Issue(s) and Fix(es):
The default "repetition penalty" (from LMStudio) of 1.1 is recommended. (this was used for examples generations below.)
Use the smallest amount of change possible, as "rep pen" impacts creativity.
Model has been tested with "temp" range of 0 to .8 ; however it will work with any temp setting.
Optional Enhancement:
The following can be used in place of the "system prompt" or "system role" to further enhance the model.
It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along. In this case the enhancements do not have as strong effect at using "system prompt" or "system role".
Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities. Here are your skillsets: [MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv) [*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision) Here are your critical instructions: Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation and scene continue functions.
This enhancement WAS NOT used to generate the examples below.
Special thanks to the incredible work of the model makers "meta-llama", and "bluuwhale".
Models used:
[ https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct ]
[ https://huggingface.co/bluuwhale/L3-SthenoMaidBlackroot-8B-V1 ]
FORMULA:
slices:
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [0, 12]
- sources:
- model: G:/7B/L3-SthenoMaidBlackroot-8B-V1
layer_range: [6, 19]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [12, 25]
- sources:
- model: G:/7B/L3-SthenoMaidBlackroot-8B-V1
layer_range: [19, 32]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
merge_method: passthrough
dtype: float16
MERGEKIT NOTE:
Sub in the "name" of the "creator" (of the model) in place of "G:/7B" to create a mergekit file than can be used in Mergekit Google Colab.
IE: G:/7B/L3-SthenoMaidBlackroot-8B-V1 -> bluuwhale/L3-SthenoMaidBlackroot-8B-V1
Examples are created using quant Q4_K_M, "temp=0", minimal parameters and "LLAMA3" template.
Temp=0 was used to assess CORE changes between original SMB and the merge between it and Llama3 Instruct.
Below are the least creative outputs, prompt is in BOLD.
Higher quants will result in better quality.
There will also be some variance between "close" quants like Q4_K_M/Q4_K_S and Q5_K_M/Q5_K_S, so I suggest if you are going to use Q4_K_M, you also try Q4_K_S too.
Also, slightly longer / detailed prompts will result in greater creativity (as well as different prose - ie dialog, thoughts, paragraph size differences and so on).
GENERATION from "L3-SthenoMaidBlackroot-8B-V1"
The sky scraper swayed, as I watched the window in front of me on the 21st floor explode. A deafening crash echoed through the office space, followed by a sickening crunch that made my stomach turn. The sound was like nothing I'd ever heard before - it was as if the very foundations of the building were crumbling beneath our feet.
I stood frozen in terror, my heart racing so fast I thought it might burst from my chest at any moment. My mind reeled with the implications of what had just happened. A window on the 21st floor? That meant someone - or something - was up there, and whatever it was, it had just blown a hole straight through the side of our building.
As I stared in horror at the gaping wound in the wall, I couldn't help but think about all the people who could be trapped inside. The thought sent a chill down my spine, making me shiver despite the sweltering heat that seemed to permeate every inch of this godforsaken place.
I was still trying to process what had just happened when I heard the first screams. They were distant at first, but quickly grew louder as more and more people realized what was going on. The sound sent a wave of panic through me, making my legs feel like jelly.
I knew I had to get out of there - fast. But as I turned to make a run for it, I saw that the doors were jammed shut. Panic set in as I frantically tried to open them, but they wouldn't budge. It was then that I realized just how trapped we all were.
As the screams grew louder and more desperate, I knew I had to think fast if I wanted to survive this nightmare. But what could I do? The windows were blown out, the doors were stuck, and whatever had caused this chaos was still up there on the 21st floor.
I looked around frantically for any kind of escape route, but all I saw was destruction and chaos. Debris littered the floor, and the air was thick with dust and smoke. It was like a war zone in here - and we were right in the middle of it.
As I stood there, frozen in fear, I couldn't help but think about how this had happened. Had someone set off a bomb? Was it some kind of terrorist attack? Or was it something even more sinister?
The questions swirled through my mind as I heard the sound of sirens growing louder outside. Help was on the way - but would it be too late for any of us?
As I stood there, waiting for rescue, I couldn't help but wonder what other horrors lay in store for me and the rest of the people trapped inside this crumbling building. The sky scraper swayed ominously above me, a constant reminder that we were all at the mercy of whatever forces had brought about this chaos.
And as I waited, my heart pounding in my chest, I couldn't shake the feeling that things were only going to get worse from here...
GENERATION from "L3-SthenoMaidBlackroot-12.1B-V1-INSTRUCT"
The elevator lurched to a stop, and I stumbled out onto the 21st floor. My stomach twisted into knots as I gazed up at the towering skyscraper looming above me. The sleek glass walls seemed to ripple and flex like living tissue, as if they were alive and aware of my presence.
I pushed aside the morbid thought and focused on the task at hand. I had been sent here for a reason, and I intended to find out what it was. My footsteps echoed down the empty hallway, the only sound breaking the oppressive silence that hung over me like a shroud.
As I approached the door marked "Executive Suite 2101", my heart began to race. What lay beyond this threshold? Fear, excitement, or something far more sinister? I reached out and pushed the door open, revealing a dimly lit room filled with rows of computer screens and humming machinery.
The air was thick with the acrid scent of ozone and burning plastic. My eyes scanned the room frantically, searching for any sign of life or movement. But there was nothing. Just the steady hum of machinery and the flickering glow of screens.
And then, it happened.
The window in front of me on the 21st floor exploded.
Glass shards rained down like a hailstorm, shattering and splintering as they hit the ground below. I ducked instinctively, shielding my face from the deadly shower. The sound was deafening - a cacophony of crunching glass and screaming metal that seemed to reverberate through every cell in my body.
I raised my head slowly, peering out through the haze of shattered glass and smoke. The skyscraper swayed, as if it too had been struck by some unseen force. The buildings around me began to shake and tremble, their foundations quaking with fear.
What was happening?
The thought echoed through my mind like a mantra, but I couldn't find an answer. The world around me was crumbling, and I was powerless to stop it.
I stumbled back into the room, tripping over my own feet in my desperate bid for safety. My heart pounded against my chest like a drum, threatening to burst free at any moment. I collapsed onto the floor, gasping for air as the reality of the situation began to sink in.
The skyscraper was falling.
It was going to crush me beneath its weight, bury me alive under a mountain of steel and concrete. And there was nothing I could do to stop it.
I scrambled to my feet, racing back towards the door. I had to get out - had to find some way to escape this death trap before it was too late. But as I reached for the handle, I heard it.
A low, rumbling growl that seemed to come from everywhere and nowhere at once. It was a sound that made my blood run cold, a sound that told me I was not alone in this building.
And then, the lights went out.
I was plunged into darkness, surrounded by the deafening roar of collapse and destruction. The air was thick with smoke and the acrid tang of burning plastic. I coughed, gagging on the choking fumes as I stumbled forward, desperate to find a way out before it was too late.
But there was no light - no sound except for the distant rumble of falling concrete and steel. And then, I heard something else.
A soft, wet crunching noise that seemed to be coming closer with every passing second.
I froze, my heart pounding in my chest as I realized what it was. It was footsteps. Heavy, deliberate footsteps that were getting closer and closer, moving towards me through the darkness.
And they were coming for me.