neph1 commited on
Commit
ba664c4
1 Parent(s): eed3f5b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -1
README.md CHANGED
@@ -6,11 +6,19 @@ language:
6
  - sv
7
  ---
8
 
 
 
9
  Qlora trained for 5 epochs on 6400 rows of q&a from around 1000 pages from wikipedia + around 100 of python questions and examples from
10
  eph1/Alpaca-Lora-GPT4-Swedish-Refined (because I had spent so much time cleaning them and didn't want to throw them away). Also a couple of hundred rows of manually
11
  gathered examples and some generated using chat-gpt.
12
  Dataset otherwise generated using gpt-3.5-turbo.
13
 
 
 
 
 
 
 
14
  Rank: 16
15
 
16
  Alpha: 16
@@ -19,7 +27,7 @@ Dropout: 0.1
19
 
20
  Context length: 1024
21
 
22
- I may run another 5 epochs on this. But it feels like it's 'aligned' pretty well. (Regular mistral insists Magdalena Andersson is prime minister, still.)
23
 
24
  Example (q8, temp: 0.7):
25
  ```
 
6
  - sv
7
  ---
8
 
9
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/653cd3049107029eb004f968/pLcriXAfp3Y9Z0RGwwVUB.png)
10
+
11
  Qlora trained for 5 epochs on 6400 rows of q&a from around 1000 pages from wikipedia + around 100 of python questions and examples from
12
  eph1/Alpaca-Lora-GPT4-Swedish-Refined (because I had spent so much time cleaning them and didn't want to throw them away). Also a couple of hundred rows of manually
13
  gathered examples and some generated using chat-gpt.
14
  Dataset otherwise generated using gpt-3.5-turbo.
15
 
16
+ The goal is to improve knowledge in Swedish topics, while improving the quality of the language.
17
+ The model seems to have a thing for history and will, if free to speak, probably tell you some tidbit about historical people.
18
+
19
+ The name comes from the Swedish bard and poet Carl Mikael Bellman who lived in the 1700s.
20
+ As with any bard, what this model says should be taken with a grain of salt. Even though it has the best of intentions.
21
+
22
  Rank: 16
23
 
24
  Alpha: 16
 
27
 
28
  Context length: 1024
29
 
30
+ I may run another 5 epochs on this. But I feel like it's 'aligned' pretty well. (Regular mistral insists Magdalena Andersson is prime minister, still.)
31
 
32
  Example (q8, temp: 0.7):
33
  ```