nitky
commited on
Commit
•
6515960
1
Parent(s):
d03e2d0
add know issues
Browse files
README.md
CHANGED
@@ -15,6 +15,14 @@ model_type: llama
|
|
15 |
---
|
16 |
# Superswallow
|
17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
**Important Notice:**
|
19 |
|
20 |
This model partially utilizes the parameters of Tulu V2 DPO finetuned based on Llama 2, so it may inherit the AI2 ImpACT license. Please use the model keeping in mind that there may be changes regarding the license if AI2 contacts me.
|
@@ -33,9 +41,9 @@ As far as I know, Swallow is the full set Llama 2 model(7B, 13B, 70B) that can o
|
|
33 |
|
34 |
This model was tested using [text-generation-webui](https://github.com/oobabooga/text-generation-webui/tree/main). I use preset `simple-1` for Generation.
|
35 |
|
36 |
-
Users reported that setting **repetition_penalty** is important to prevent repeated output. If you run into any issues, be sure to check your settings.
|
37 |
|
38 |
-
- temperature: 0.7
|
39 |
- top_p: 0.9
|
40 |
- **repetition_penalty: 1.15**
|
41 |
- top_k: 20
|
|
|
15 |
---
|
16 |
# Superswallow
|
17 |
|
18 |
+
**Known Performance Issues**
|
19 |
+
|
20 |
+
Two potential bugs have been found in this model:
|
21 |
+
- **NEED `repetition_penalty`**
|
22 |
+
- **NEED high `temperature`**
|
23 |
+
|
24 |
+
These issue may be caused by the self-attention layers and will be fixed in the next version. (However, I don't know if the benchmark score will improve)
|
25 |
+
|
26 |
**Important Notice:**
|
27 |
|
28 |
This model partially utilizes the parameters of Tulu V2 DPO finetuned based on Llama 2, so it may inherit the AI2 ImpACT license. Please use the model keeping in mind that there may be changes regarding the license if AI2 contacts me.
|
|
|
41 |
|
42 |
This model was tested using [text-generation-webui](https://github.com/oobabooga/text-generation-webui/tree/main). I use preset `simple-1` for Generation.
|
43 |
|
44 |
+
Users reported that setting **repetition_penalty** is important to prevent repeated output. If you run into any issues, be sure to check your settings. Additionally, a bug was discovered that caused an error at low temperatures.
|
45 |
|
46 |
+
- **temperature: 0.7**
|
47 |
- top_p: 0.9
|
48 |
- **repetition_penalty: 1.15**
|
49 |
- top_k: 20
|