|
--- |
|
language: |
|
- en |
|
- fr |
|
- de |
|
- es |
|
- it |
|
- pt |
|
- ru |
|
- zh |
|
- ja |
|
license: cc-by-nc-4.0 |
|
library_name: transformers |
|
tags: |
|
- nsfw |
|
- Visual novel |
|
- roleplay |
|
- mergekit |
|
- merge |
|
base_model: |
|
- mistralai/Mistral-Nemo-Instruct-2407 |
|
- NeverSleep/Lumimaid-v0.2-12B |
|
- Epiculous/Violet_Twilight-v0.1 |
|
- Sao10K/MN-12B-Lyra-v4 |
|
- anthracite-org/magnum-v2-12b |
|
pipeline_tag: text-generation |
|
model-index: |
|
- name: ChatWaifu_v1.4 |
|
results: |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: IFEval (0-Shot) |
|
type: HuggingFaceH4/ifeval |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: inst_level_strict_acc and prompt_level_strict_acc |
|
value: 56.91 |
|
name: strict accuracy |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: BBH (3-Shot) |
|
type: BBH |
|
args: |
|
num_few_shot: 3 |
|
metrics: |
|
- type: acc_norm |
|
value: 31.63 |
|
name: normalized accuracy |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MATH Lvl 5 (4-Shot) |
|
type: hendrycks/competition_math |
|
args: |
|
num_few_shot: 4 |
|
metrics: |
|
- type: exact_match |
|
value: 7.85 |
|
name: exact match |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: GPQA (0-shot) |
|
type: Idavidrein/gpqa |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: acc_norm |
|
value: 7.61 |
|
name: acc_norm |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MuSR (0-shot) |
|
type: TAUR-Lab/MuSR |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: acc_norm |
|
value: 20.03 |
|
name: acc_norm |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MMLU-PRO (5-shot) |
|
type: TIGER-Lab/MMLU-Pro |
|
config: main |
|
split: test |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 27.5 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4 |
|
name: Open LLM Leaderboard |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
![image](./cover.png) |
|
|
|
Merged model using [mergekit](https://github.com/arcee-ai/mergekit/tree/main/mergekit) |
|
|
|
This model aimed to act like visual novel character. |
|
|
|
## Merge Format |
|
|
|
```yaml |
|
models: |
|
- model: spow12/ChatWaifu_modify_data |
|
- model: anthracite-org/magnum-v2-12b |
|
- model: Sao10K/MN-12B-Lyra-v4 |
|
- model: Gryphe/Pantheon-RP-1.6-12b-Nemo |
|
- model: mistralai/Mistral-Nemo-Instruct-2407 |
|
- model: NeverSleep/Lumimaid-v0.2-12B |
|
- model: Epiculous/Violet_Twilight-v0.1 |
|
merge_method: model_stock |
|
base_model: spow12/ChatWaifu_modify_data |
|
dtype: bfloat16 |
|
|
|
``` |
|
you have to resize chatwaifu and lucimaid's embedding size(131073 to 131072). |
|
|
|
# WaifuModel Collections |
|
|
|
- [TTS](https://huggingface.co/spow12/visual_novel_tts) |
|
- [Chat](https://huggingface.co/spow12/ChatWaifu_v1.3.1) |
|
- [ASR](https://huggingface.co/spow12/Visual-novel-transcriptor) |
|
|
|
# Unified demo |
|
|
|
[WaifuAssistant](https://github.com/yw0nam/WaifuAssistant) |
|
|
|
# Update |
|
- 2024.09.10 Update Ver 1.4 |
|
- Modify data format and applying flitering. |
|
- Merge with model stock |
|
- 2024.08.29 Update Ver 1.3.1 |
|
- Merge Ver1.2, [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) and [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B), [Epiculous/Violet_Twilight-v0.1](https://huggingface.co/Epiculous/Violet_Twilight-v0.1) |
|
- Adjust merge weight. |
|
- 2024.08.16 Update Ver 1.3 |
|
- Merge Ver1.2, [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) and [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B), |
|
- 2024.08.08 Update Ver 1.2.1 |
|
- Merge Ver1.2 and [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) |
|
- 2024.08.07 Update Ver 1.2 |
|
- Add Preference Learning in training pipeline |
|
- 2024.07.29 Update Ver 1.1 |
|
- Add dataset format -> generate novel, fill masked sentences |
|
- Remove system role and integrate at user message. |
|
- Remove 『』 in conversation. |
|
- 2024.06.20 Upload other chara's sample chat history. |
|
- 2024.06.13 Upload Model |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
- **Developed by:** spow12(yw_nam) |
|
- **Shared by :** spow12(yw_nam) |
|
- **Model type:** CausalLM |
|
- **Language(s) (NLP):** japanese |
|
- **Finetuned from model :** [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B) |
|
|
|
Currently, chatbot has below personality. |
|
|
|
character | visual_novel | |
|
--- | --- | |
|
ムラサメ | Senren*Banka | |
|
茉子 | Senren*Banka | |
|
芳乃 | Senren*Banka | |
|
レナ | Senren*Banka | |
|
千咲 | Senren*Banka | |
|
芦花 | Senren*Banka | |
|
愛衣 | Café Stella and the Reaper's Butterflies | |
|
栞那 | Café Stella and the Reaper's Butterflies | |
|
ナツメ | Café Stella and the Reaper's Butterflies | |
|
希 | Café Stella and the Reaper's Butterflies | |
|
涼音 | Café Stella and the Reaper's Butterflies | |
|
あやせ | Riddle Joker | |
|
七海 | Riddle Joker | |
|
羽月 | Riddle Joker | |
|
茉優 | Riddle Joker | |
|
小春 | Riddle Joker | |
|
|
|
### Feature |
|
|
|
- Fluent Chat performance |
|
- Reduce repetition problem when generate with many turn(over 20~30) |
|
- Zero Shot character persona using description of character. |
|
- 128k context window |
|
- Memory ability that does not forget even after long-context generation |
|
|
|
## Demo |
|
|
|
You can use Demo in google colab. |
|
|
|
Check [Here](https://colab.research.google.com/drive/194_FN28reEPTwS51dwpLLBBwEfeoBjP9?usp=sharing) |
|
|
|
## Future Work. |
|
|
|
Now, i'm quite satisfying the model chat performance. |
|
|
|
So, i'm going to focus for integrating the vision modality to model so that our waifu can do more general tasks. |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
This model trained by japanese dataset included visual novel which contain nsfw content. |
|
|
|
So, The model may generate NSFW content. |
|
|
|
## Use & Credit |
|
|
|
This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly. |
|
|
|
By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and anime persons). |
|
|
|
This repository can use Visual novel-based RAG, but i will not distribute it yet because i'm not sure if it is permissible to release the data publicly. |
|
|
|
|
|
## Citation |
|
|
|
```bibtex |
|
@misc {ChatWaifu_v1.4, |
|
author = { YoungWoo Nam }, |
|
title = { ChatWaifu_v1.4 }, |
|
year = 2024, |
|
url = { https://huggingface.co/spow12/ChatWaifu_v1.4 }, |
|
publisher = { Hugging Face } |
|
} |
|
``` |
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) |
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_spow12__ChatWaifu_v1.4) |
|
|
|
| Metric |Value| |
|
|-------------------|----:| |
|
|Avg. |25.25| |
|
|IFEval (0-Shot) |56.91| |
|
|BBH (3-Shot) |31.63| |
|
|MATH Lvl 5 (4-Shot)| 7.85| |
|
|GPQA (0-shot) | 7.61| |
|
|MuSR (0-shot) |20.03| |
|
|MMLU-PRO (5-shot) |27.50| |
|
|
|
|