DavidAU
/

Psyonic-Cetacean-Depth-Charge-13B-GGUF

Model card Files Files and versions Community

DavidAU commited on 4 days ago

Commit

f121585

•

1 Parent(s): 52fedca

Update README.md

Files changed (1) hide show

README.md +4 -0

README.md CHANGED Viewed

@@ -79,6 +79,10 @@ Note that temp AND "rep pen" changes will drastically change the output ; adjust
 Also, this model may perform better cold for some prompts: Unload the model, load the model -> prompt it ... rather than
 keeping the model loaded at all times.
 <B>Model Template:</B>
 This is a LLAMA2 model, and requires Alpaca or Llama2 template, but may work with other template(s) and has maximum context of 4k / 4096.

 Also, this model may perform better cold for some prompts: Unload the model, load the model -> prompt it ... rather than
 keeping the model loaded at all times.
+Recommend using the larger quant you can "run" for quality.
+This repo also has the new "arm quants" : Q4_0_4_4, Q4_0_4_8 and Q4_0_8_8
 <B>Model Template:</B>
 This is a LLAMA2 model, and requires Alpaca or Llama2 template, but may work with other template(s) and has maximum context of 4k / 4096.