Text Generation
Transformers
Safetensors
English
llama
llama-factory
Not-For-All-Audiences
conversational
text-generation-inference
Inference Endpoints
aaronday3 commited on
Commit
b78d613
1 Parent(s): 6eface1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -5
README.md CHANGED
@@ -187,16 +187,19 @@ We think there is too much secrecy around what data is being used, and different
187
 
188
  ### The Good
189
  We found that increasing the amount of ranks from 64 to 256 has reduced repetition but also led to the language used resembling Claude more than the 64 rank version. No worries, it's still far enough from Claude.
190
- <br>It also led to increased coherency but reduced instruction following, likely because the model started diverging more away from L3 8B Instruct.
191
- <br>**The model is uncensored for RP. For Instruct it needs 2-3 words of prefill for the first message.**
192
  <br>We found that increasing the amount of data from 1K to 6.5K reduced repetition aswell.
193
- <br>**The prose is much better than other synthetic data generations. The model also demonstrates increased style copying abilities likely a result of the long data and varying writing styles found in WritingPrompts.**
194
- <br>**The model is exceptional at being creative in roleplaying**, knows different persona's and even a single character will change persona in different contexts, persona is tied to last few messages rather than system message or character card. **This is great as it often means the model can do impressive things without you needing to explicitly specify.**
 
 
195
 
196
  ### Improvements for Next Run
197
 
198
  Formatting can break sometimes.
199
- Repetition can become an issue with certain types of prompts. Removing system helps.
 
200
 
201
  ### Comments about training
202
 
 
187
 
188
  ### The Good
189
  We found that increasing the amount of ranks from 64 to 256 has reduced repetition but also led to the language used resembling Claude more than the 64 rank version. No worries, it's still far enough from Claude.
190
+ <br>**Model follows "OOC:" prompts religiously. Exceptional!**
191
+ <br>It also led to **increased coherency but reduced system prompt following (when not OOC)**, likely because the model started diverging more away from L3 8B Instruct.
192
  <br>We found that increasing the amount of data from 1K to 6.5K reduced repetition aswell.
193
+
194
+ <br>The model is uncensored for RP. For Instruct it needs 2-3 words of prefill for the first message.
195
+ <br>The **prose is much better** and **the style range is huge** than other synthetic data generations. The model also demonstrates increased **style copying abilities** (from fewshot) likely a result of human longform data and varying writing styles found in WritingPrompts.
196
+ <br>The model is **exceptional at being creative in roleplaying**, knows different persona's and even a single character will change persona in different contexts, persona is tied to last few messages rather than system message or character card. **This is great as it often means the model can do impressive things without you needing to explicitly specify.**
197
 
198
  ### Improvements for Next Run
199
 
200
  Formatting can break sometimes.
201
+ <br>Repetition can become an issue with certain types of prompts. Removing system helps.
202
+ <br>In some contexts the model is "all over the place" and doesn't stick to a coherent narrative. I need to study this further as its a complex trait which manifests in different quantities and can be good or bad depending on what the user wants to get out of the model.
203
 
204
  ### Comments about training
205