README.md

---
language:
- en
- fr
- de
- es
- it
- pt
- ru
- zh
- ja
license: cc-by-nc-4.0
library_name: transformers
tags:
- nsfw
- Visual novel
- roleplay
- mergekit
- merge
base_model:
- mistralai/Mistral-Nemo-Instruct-2407
- NeverSleep/Lumimaid-v0.2-12B
- Epiculous/Violet_Twilight-v0.1
- Sao10K/MN-12B-Lyra-v4
- anthracite-org/magnum-v2-12b
pipeline_tag: text-generation
model-index:
- name: ChatWaifu_v1.4
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 56.91
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 31.63
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 7.85
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 7.61
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 20.03
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 27.5
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v1.4
      name: Open LLM Leaderboard
---

# Model Card for Model ID

![image](./cover.png)

Merged model using [mergekit](https://github.com/arcee-ai/mergekit/tree/main/mergekit)

This model aimed to act like visual novel character.

## Merge Format

```yaml
models:
    - model: spow12/ChatWaifu_modify_data
    - model: anthracite-org/magnum-v2-12b
    - model: Sao10K/MN-12B-Lyra-v4
    - model: Gryphe/Pantheon-RP-1.6-12b-Nemo
    - model: mistralai/Mistral-Nemo-Instruct-2407
    - model: NeverSleep/Lumimaid-v0.2-12B
    - model: Epiculous/Violet_Twilight-v0.1
merge_method: model_stock
base_model: spow12/ChatWaifu_modify_data
dtype: bfloat16

```
you have to resize chatwaifu and lucimaid's embedding size(131073 to 131072). 

# WaifuModel Collections 

- [TTS](https://huggingface.co/spow12/visual_novel_tts)
- [Chat](https://huggingface.co/spow12/ChatWaifu_v1.3.1)
- [ASR](https://huggingface.co/spow12/Visual-novel-transcriptor)

# Unified demo

[WaifuAssistant](https://github.com/yw0nam/WaifuAssistant)

# Update
- 2024.09.10 Update Ver 1.4
  - Modify data format and applying flitering.
  - Merge with model stock
- 2024.08.29 Update Ver 1.3.1
  - Merge Ver1.2, [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) and [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B), [Epiculous/Violet_Twilight-v0.1](https://huggingface.co/Epiculous/Violet_Twilight-v0.1)
  - Adjust merge weight.    
- 2024.08.16 Update Ver 1.3
  - Merge Ver1.2, [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) and [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B),
- 2024.08.08 Update Ver 1.2.1
  - Merge Ver1.2 and [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407)
- 2024.08.07 Update Ver 1.2
  - Add Preference Learning in training pipeline
- 2024.07.29 Update Ver 1.1
  - Add dataset format -> generate novel, fill masked sentences
  - Remove system role and integrate at user message.
  - Remove 『』 in conversation.
- 2024.06.20 Upload other chara's sample chat history.
- 2024.06.13 Upload Model

## Model Details

### Model Description

- **Developed by:** spow12(yw_nam)
- **Shared by :** spow12(yw_nam)
- **Model type:** CausalLM
- **Language(s) (NLP):** japanese
- **Finetuned from model :** [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B)

Currently, chatbot has below personality.

character | visual_novel |
--- | --- |
ムラサメ | Senren＊Banka |
茉子  | Senren＊Banka |
芳乃  |  Senren＊Banka |
レナ  | Senren＊Banka |
千咲  | Senren＊Banka |
芦花  | Senren＊Banka |
愛衣  | Café Stella and the Reaper's Butterflies |
栞那  | Café Stella and the Reaper's Butterflies |
ナツメ | Café Stella and the Reaper's Butterflies |
希    | Café Stella and the Reaper's Butterflies |
涼音  | Café Stella and the Reaper's Butterflies |
あやせ    | Riddle Joker |
七海     | Riddle Joker |
羽月     | Riddle Joker |
茉優     | Riddle Joker |
小春     | Riddle Joker |

### Feature

- Fluent Chat performance
- Reduce repetition problem when generate with many turn(over 20~30)
- Zero Shot character persona using description of character.
- 128k context window
- Memory ability that does not forget even after long-context generation

## Demo

You can use Demo in google colab. 

Check [Here](https://colab.research.google.com/drive/194_FN28reEPTwS51dwpLLBBwEfeoBjP9?usp=sharing)

## Future Work.

Now, i'm quite satisfying the model chat performance.

So, i'm going to focus for integrating the vision modality to model so that our waifu can do more general tasks.

## Bias, Risks, and Limitations

This model trained by japanese dataset included visual novel which contain nsfw content.

So, The model may generate NSFW content.

## Use & Credit

This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly. 

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and anime persons).

This repository can use Visual novel-based RAG, but i will not distribute it yet because i'm not sure if it is permissible to release the data publicly.


## Citation

```bibtex
@misc {ChatWaifu_v1.4,
    author       = { YoungWoo Nam },
    title        = { ChatWaifu_v1.4 },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_v1.4 },
    publisher    = { Hugging Face }
}
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_spow12__ChatWaifu_v1.4)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |25.25|
|IFEval (0-Shot)    |56.91|
|BBH (3-Shot)       |31.63|
|MATH Lvl 5 (4-Shot)| 7.85|
|GPQA (0-shot)      | 7.61|
|MuSR (0-shot)      |20.03|
|MMLU-PRO (5-shot)  |27.50|