rnaveensrinivas/Phi-3-mini-4k-instruct-retrained_network_corpus

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 4.4383
+- Loss: 4.3960
 
 ## Model description
 
@@ -47,6 +47,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 1.8696        | 1.0   | 112  | 2.0452          |
+| 1.3684        | 2.0   | 224  | 2.1041          |
+| 0.9071        | 3.0   | 336  | 2.3714          |
+| 0.5282        | 4.0   | 448  | 2.8351          |
+| 0.2728        | 5.0   | 560  | 3.3396          |
+| 0.1308        | 6.0   | 672  | 3.7990          |
+| 0.0594        | 7.0   | 784  | 3.9724          |
+| 0.032         | 8.0   | 896  | 4.0585          |
+| 0.0186        | 9.0   | 1008 | 4.3351          |
+| 0.0133        | 10.0  | 1120 | 4.3960          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,9 +20,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "down_proj",
-    "gate_up_proj",
     "qkv_proj",
+    "gate_up_proj",
+    "down_proj",
     "o_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0153024bf770659634996ab5909d6616f939b3ddafe3a449ad7632117ed98c1b
+oid sha256:5ac6327b6143367dc85c4d4e7aa8efaa60e75614d2db6f13eff0dcf7bc1f8ee5
 size 100697728

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c6dffc4db8fc3183893c6dda6cf997aee96be3e2957b56846529f2be5d50a71
+oid sha256:4f2624aedc746967165bb63db5b0f730fd701ebf8ae1439e242ce19aecfcfad1
 size 5432