Update README.md

README.md

@@ -37,7 +37,7 @@ For Ollama, it required to be a GGUF file. Once you have this it is pretty strai
 
 Quick Start: 
 - You must already have Ollama running in your setting
-- Download the unsloth.Q4_K_M.gguf model from Files
+- Download the unsloth.Q2_K.gguf model from Files
 - In the same directory create a file call "Modelfile"
 - Inside the "Modelfile" type
 
@@ -140,4 +140,40 @@ dataset = dataset.map(formatting_prompts_func, batched = True,)
 
 ```
 
+# SIDENOTE : 
+
+Because the Fine Tuned data way that way, you could technically try this and make it work..(?) still testing but please feel free to try as well (and do let me know)
+
+```python
+FROM ./GEMMA2_unsloth.Q2_K.gguf
+
+PARAMETER stop ["<|STOP|>"]
+
+TEMPLATE """<|STOP|><|BEGIN_QUERY|>
+{{.Prompt}}
+<|END_QUERY|>
+<|BEGIN_ANALYSIS|>
+
+<|END_ANALYSIS|>
+<|BEGIN_RESPONSE|>
+
+<|END_RESPONSE|>
+<|BEGIN_CLASSIFICATION|>
+
+<|END_CLASSIFICATION|>
+<|BEGIN_SENTIMENT|>
+
+<|END_SENTIMENT|>
+<|STOP|>"""
+
+SYSTEM """You are an AI assistant trained to provide comprehensive and engaging responses. Follow this structure in your replies:
+1. Begin with a brief analysis of the query.
+2. Provide a detailed response, using an enthusiastic and friendly tone. Include specific examples where appropriate.
+3. Add relevant classification keywords.
+4. Conclude with a sentiment analysis of your response.
+Use the BEGIN_ and END_ tokens to clearly delineate each section of your response."""
+
+```
+
+
 Will be updating this periodically.. as I have limited colab resources..