metadata
license: other
license_name: xt-aurora-license
license_link: LICENSE
language:
- en
- es
tags:
- conversational
- chat
- roleplay
library_name: GGUF
pipeline_tag: text-generation
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T
datasets:
- XeTute/Small-Medium-Conversation-Multilingual
- XeTute/Conversational-Small
We, XeTute, introduce AURORA V1.0 - a humerous, efficient, smart(for its size) and unbiased(because of too low parameter count, consider it a virtual child with a bunch of knowledge =)) Language Model.
Intended usecases:
- Next-Word prediction for mobile devices:
- This Model can be reliably packaged into a keyboard-app to help make Next-Word suggestions more accurate (for performance, INT4 or less might be smart)
- Conversations:
- AURORA can engage in conversations using the Vicuna format, remember to replace "ASSISTANT" with "AURORA" though.
- AURORA can engage in SFW roleplay with simple character definitions. It wasn't trained on NSFW.
- AURORA can engage in simple, short Q&A. It was trained on factual data too, which means it performs well for its size.
Training:
- Trained for two months.
- Dataset created by XeTute, and translated using different free-lancing services.
- Dataset included:
- Mathematic Q&A
- Logic Q&A
- One-Page stories and roleplays with very brief character definitions
- ADAM as an optimizer. Alltogether, the model was trained on additional 20B tokens.
- All previous beta versions of this series of SLMs were deleted, because almost no downloads were made.
- V1.0 is the last model in this series which will be published, because of too little community activity.
Recommended settings:
- Temperature 0.1 - 0,4 is stable.
- Context Length of 2048(base) to 4096(RoPE) will work well for story-telling, role-playing and simple conversations.
- Output Length: 256 will work very stable, but you can extent to 512. Anything beyond that point is risky, text might become repetitous.
- A system prompt which works well can be found at "Files at Versions" => "chat_template". Just copy and paste this into the system prompt or add it before your first message.
- Chat Format:
{name of your roleplay}: {input}
{name of AURORA's character}: {output}
or,
USER: {input}
AURORA: {output}
Chat examples using KoboldCPP and the settings recommended above:
Note, a roleplay where you directly pass character definitions and a starting scenario will work way better, this is just an example.
We wish you a friendly chat with AURORA.