Update README.md

README.md

@@ -1,32 +1,79 @@
 ---
+inference: false
 tags:
 - generated_from_trainer
+- Yi
 model-index:
-- name: limarp-lora-out
+- name: limarpv3-yi-llama-34b-lora
   results: []
+license: apache-2.0
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
-# limarp-lora-out
+# limarpv3-yi-llama-34b-lora
+
+This model is an unofficial Yi-34B-Llama training on the LimaRP v3 dataset by [lemonilia](https://huggingface.co/lemonilia). It does not include the pretraining stage using stories.
+
+The [Yi-34B-Llama](https://huggingface.co/chargoddard/Yi-34B-Llama) model is a modified [01-ai/Yi-34B](https://huggingface.co/01-ai/Yi-34B) with keys renamed to match those used in Llama models, eliminating the need for remote code and ensuring compatibility with existing training and inference repositories. Architecturally this is similar to a Llama 2 34B model with an expanded vocab size of 64000.
 
-This model was trained from scratch on the None dataset.
 It achieves the following results on the evaluation set:
 - Loss: 1.9729
 
 ## Model description
 
-More information needed
+For more details about LimaRP, see the model page for the [previously released v2 version for Llama-2](https://huggingface.co/lemonilia/limarp-llama2-v2). Most details written there apply for this version as well. Generally speaking, LimaRP is a longform-oriented, novel-style roleplaying chat model intended to replicate the experience of 1-on-1 roleplay on Internet forums. Short-form, IRC/Discord-style RP (aka "Markdown format") is not supported yet. The model does not include instruction tuning, only manually picked and slightly edited RP conversations with persona and scenario data.
+
+Prompt format is the [extended Alpaca format](https://github.com/tatsu-lab/stanford_alpaca):
+
+```
+### Instruction:
+Character's Persona: {bot character description}
+User's Persona: {user character description}
+Scenario: {what happens in the story}
+Play the role of Character. You must engage in a roleplaying chat with User below this line. Do not write dialogues and narration for User.
+### Input:
+User: {utterance}
+### Response:
+Character: {utterance}
+### Input
+User: {utterance}
+### Response:
+Character: {utterance}
+(etc.)
+```
+
+Inspired by the previously named "Roleplay" preset in SillyTavern, with this version of LimaRP it is possible to append a length modifier to the response instruction sequence, like this:
+
+```
+### Input
+User: {utterance}
+
+### Response: (length = medium)
+Character: {utterance}
+```
+
+This has an immediately noticeable effect on bot responses. The lengths using during training are:
+`micro`, `tiny`, `short`, `medium`, `long`, `massive`, `huge`, `enormous`, `humongous`, `unlimited`.
+**The recommended starting length is medium**. Keep in mind that the AI can ramble or impersonate
+the user with very long messages.
+
+The length control effect is reproducible, but the messages will not necessarily follow
+lengths very precisely, rather follow certain ranges on average, as seen in this table
+with data from tests made with one reply at the beginning of the conversation:
+
+![lengths](https://i.imgur.com/2WXGgaV.png)
+
+Response length control appears to work well also deep into the conversation. **By omitting
+the modifier, the model will choose the most appropriate response length** (although it might
+not necessarily be what the user desires).
 
 ## Intended uses & limitations
 
-More information needed
+The model will show biases similar to those observed in niche roleplaying forums on the Internet, besides those exhibited by the base model.
 
 ## Training and evaluation data
 
-More information needed
+For more details about LimaRP, see the model page for the [previously released v2 version for Llama-2](https://huggingface.co/lemonilia/limarp-llama2-v2).
 
 ## Training procedure
 
@@ -78,4 +125,4 @@ The following hyperparameters were used during training:
 - Transformers 4.34.1
 - Pytorch 2.0.1+cu118
 - Datasets 2.14.6
-- Tokenizers 0.14.1
+- Tokenizers 0.14.1