Update README.md

README.md

@@ -1,9 +1,102 @@
 ---
 library_name: peft
+license: apache-2.0
+datasets:
+- ThatOneShortGuy/SongLyrics
+language:
+- en
+tags:
+- music
 ---
+# Musical Falcon
+
+[OpenAssistant/falcon-7b-sft-mix-2000](https://huggingface.co/OpenAssistant/falcon-7b-sft-mix-2000) model fine tuned using PEFT on
+[Song Lyrics](https://huggingface.co/datasets/ThatOneShortGuy/SongLyrics) to write lyrics to songs.
+
+## Model Details
+- **Finetuned from**: [OpenAssistant/falcon-7b-sft-mix-2000](https://huggingface.co/OpenAssistant/falcon-7b-sft-mix-2000)
+- **Model Type**: Causal decoder-only transformer language model
+- **Language**: English (and limited capabilities in German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish);
+- **License**: Apache 2.0
+- **Contact**: Lol don't. This is just for fun.
+
+## Usage
+The basic basic format getting it in is:
+```python
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM
+
+config = PeftConfig.from_pretrained("ThatOneShortGuy/MusicalFalcon")
+model = AutoModelForCausalLM.from_pretrained("OpenAssistant/falcon-7b-sft-mix-2000")
+model = PeftModel.from_pretrained(model, "ThatOneShortGuy/MusicalFalcon")
+```
+
+## Prompting
+Considering this comes from [OpenAssistant/falcon-7b-sft-mix-2000](https://huggingface.co/OpenAssistant/falcon-7b-sft-mix-2000), it uses the same structure.
+Two special tokens are used to mark the beginning of user and assistant turns: `<|prompter|>` and `<|assistant|>`. Each turn ends with a `<|endoftext|>` token.
+The training prompt used the structure: 
+```
+<|prompter|>Come up with the lyrics for a song from "{artist}" {"from " + year if year else ""} titled "{title}".<|endoftext|>
+<|assistant|>Sure! Here are the lyrics:
+{lyrics}
+<|endoftext|>
+```
+
+However, it still seems to work just fine using:
+```
+<|prompter|>Write me a song titled "{title}"<|endoftext|><|assistant|>
+```
+or any anything of similar nature. Feel free to add a description of the song in there too.
+
+## Examples
+Prompt
+```
+<|prompter|>Write a song with the lyrics for a song from artist "Eminem" from 2018 titled "Suzy's Lemonade Stand"<|endoftext|>
+<|assistant|>Sure! Here are the lyrics:
+```
+Result
+```
+[Intro]
+I'm gonna make you famous, baby
+
+[Verse 1]
+It's been a long time since I had a lemonade stand
+But I'm back in town and I'ma do it again
+You know what? I've been on my own since I was born
+So I'll be here all day long, so come on, let's go
+I got some new tricks up my sleeve that I'd like to show
+And I'mma get your name out of your mouth, so please don't blow
+Let me see if I can get you to say my name
+If not, then I'l just have to take it back to the drawing board
+I gotta get this money, I's a little bit broke
+I need some cash, I need some dough, I gotta get these clothes off
+I wanna be a star, I wanna be the man
+I want to be a superstar, I want to get my hands on them bands
+```
+
+
 ## Training procedure
 
+Trained on 1 epoch of all songs from [Song Lyrics](https://huggingface.co/datasets/ThatOneShortGuy/SongLyrics) with "Rap" or "Hip" in the genre title.
+
+- GPU: RTX Titan
+- load_in_8bit: True
+
+Further trained on ~8 epochs of the top 10% of [Song Lyrics](https://huggingface.co/datasets/ThatOneShortGuy/SongLyrics)
+
+- GPU: RTX Titan
+- load_in_8bit: False
+  - torch_dtype: torch.float16
+
+- load_in_4bit: False
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: True
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: fp4
+- bnb_4bit_use_double_quant: False
+
 ### Framework versions
 
 
-- PEFT 0.5.0.dev0
+- PEFT 0.5.0.dev0