Undi95 commited on
Commit
bc35358
1 Parent(s): f87864e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +65 -60
README.md CHANGED
@@ -1,79 +1,84 @@
1
  ---
2
- license: apache-2.0
3
- base_model: mistralai/Mistral-7B-v0.1
4
- tags:
5
- - generated_from_trainer
6
- model-index:
7
- - name: Mistral-Noromaid-7B
8
- results: []
9
  ---
10
 
11
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
- should probably proofread and complete it, then remove this comment. -->
13
 
14
- [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
15
- # Mistral-Noromaid-7B
16
 
17
- This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
18
- It achieves the following results on the evaluation set:
19
- - Loss: 1.1514
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
- ## Model description
22
 
23
- More information needed
24
 
25
- ## Intended uses & limitations
26
 
27
- More information needed
 
 
28
 
29
- ## Training and evaluation data
 
30
 
31
- More information needed
 
 
32
 
33
- ## Training procedure
 
34
 
35
- ### Training hyperparameters
36
 
37
- The following hyperparameters were used during training:
38
- - learning_rate: 5e-06
39
- - train_batch_size: 2
40
- - eval_batch_size: 2
41
- - seed: 42
42
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
43
- - lr_scheduler_type: cosine
44
- - lr_scheduler_warmup_steps: 10
45
- - num_epochs: 2
46
 
47
- ### Training results
 
 
 
48
 
49
- | Training Loss | Epoch | Step | Validation Loss |
50
- |:-------------:|:-----:|:----:|:---------------:|
51
- | 1.2103 | 0.0 | 1 | 1.5604 |
52
- | 1.3191 | 0.1 | 192 | 1.2539 |
53
- | 1.1727 | 0.2 | 384 | 1.2346 |
54
- | 1.3466 | 0.3 | 576 | 1.2171 |
55
- | 0.9652 | 0.4 | 768 | 1.2073 |
56
- | 0.996 | 0.5 | 960 | 1.1920 |
57
- | 0.7863 | 0.6 | 1152 | 1.1804 |
58
- | 0.8883 | 0.7 | 1344 | 1.1700 |
59
- | 0.9351 | 0.8 | 1536 | 1.1590 |
60
- | 0.8361 | 0.9 | 1728 | 1.1511 |
61
- | 1.2718 | 1.0 | 1920 | 1.1438 |
62
- | 0.9613 | 1.09 | 2112 | 1.1585 |
63
- | 1.4066 | 1.19 | 2304 | 1.1550 |
64
- | 0.7388 | 1.29 | 2496 | 1.1538 |
65
- | 1.0686 | 1.39 | 2688 | 1.1531 |
66
- | 1.3536 | 1.49 | 2880 | 1.1533 |
67
- | 0.4994 | 1.59 | 3072 | 1.1517 |
68
- | 0.7574 | 1.69 | 3264 | 1.1519 |
69
- | 0.7574 | 1.79 | 3456 | 1.1516 |
70
- | 1.1436 | 1.89 | 3648 | 1.1514 |
71
- | 1.4085 | 1.99 | 3840 | 1.1514 |
72
 
 
73
 
74
- ### Framework versions
75
 
76
- - Transformers 4.37.0.dev0
77
- - Pytorch 2.0.1+cu118
78
- - Datasets 2.15.0
79
- - Tokenizers 0.15.0
 
1
  ---
2
+ license: cc-by-nc-4.0
 
 
 
 
 
 
3
  ---
4
 
5
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/630dfb008df86f1e5becadc3/VKX2Z2yjZX5J8kXzgeCYO.png)
 
6
 
 
 
7
 
8
+ ---
9
+
10
+ # Disclaimer:
11
+ ## This is a ***TEST*** version, don't expect everything to work!!!
12
+
13
+ You may use our custom **prompting format**(scroll down to download them!), or simple alpaca. **(Choose which fits best for you!)**
14
+
15
+ ---
16
+
17
+
18
+ # This model is a collab between [IkariDev](https://huggingface.co/IkariDev) and [Undi](https://huggingface.co/Undi95)!
19
+
20
+ Tired of the same merges everytime? Here it is, the Noromaid-7b-v0.2 model. Suitable for RP, ERP and general stuff.
21
+
22
+ [Recommended generation settings - No settings yet(Please suggest some over in the Community tab!)]
23
+
24
+ <!-- description start -->
25
+ ## Description
26
+
27
+ <!-- [Recommended settings - contributed by localfultonextractor](https://files.catbox.moe/ue0tja.json) -->
28
+
29
+ This repo contains fp16 files of Noromaid-7b-v0.2.
30
+
31
+ [FP16 - by IkariDev and Undi](https://huggingface.co/NeverSleep/Noromaid-7b-v0.2)
32
+
33
+ <!-- [GGUF - By TheBloke](https://huggingface.co/TheBloke/Athena-v4-GGUF)-->
34
+
35
+ <!-- [GPTQ - By TheBloke](https://huggingface.co/TheBloke/Athena-v4-GPTQ)-->
36
+
37
+ <!-- [exl2[8bpw-8h] - by AzureBlack](https://huggingface.co/AzureBlack/Echidna-13b-v0.3-8bpw-8h-exl2)-->
38
+
39
+ <!-- [AWQ - By TheBloke](https://huggingface.co/TheBloke/Athena-v4-AWQ)-->
40
+
41
+ <!-- [fp16 - by IkariDev+Undi95](https://huggingface.co/IkariDev/Athena-v4)-->
42
+
43
+ [GGUF - by IkariDev and Undi](https://huggingface.co/NeverSleep/Noromaid-7b-v0.2-GGUF)
44
+ <!-- [OLD(GGUF - by IkariDev+Undi95)](https://huggingface.co/IkariDev/Athena-v4-GGUF)-->
45
+
46
+ ## Ratings:
47
 
48
+ Note: We have permission of all users to upload their ratings, we DONT screenshot random reviews without asking if we can put them here!
49
 
50
+ No ratings yet!
51
 
52
+ If you want your rating to be here, send us a message over on DC and we'll put up a screenshot of it here. DC name is "ikaridev" and "undi".
53
 
54
+ <!-- description end -->
55
+ <!-- prompt-template start -->
56
+ ## Prompt template: Custom format, or Alpaca
57
 
58
+ ### Custom format:
59
+ UPDATED!! SillyTavern config files: [Context](https://files.catbox.moe/ifmhai.json), [Instruct](https://files.catbox.moe/ttw1l9.json).
60
 
61
+ ### Alpaca:
62
+ ```
63
+ Below is an instruction that describes a task. Write a response that appropriately completes the request.
64
 
65
+ ### Instruction:
66
+ {prompt}
67
 
68
+ ### Response:
69
 
70
+ ```
 
 
 
 
 
 
 
 
71
 
72
+ ## Training data used:
73
+ - [no_robots dataset](https://huggingface.co/Undi95/Llama2-13B-no_robots-alpaca-lora) let the model have more human behavior, enhances the output.
74
+ - [Aesir Private RP dataset] New data from a new and never used before dataset, add fresh data, no LimaRP spam, this is 100% new. Thanks to the [MinvervaAI Team](https://huggingface.co/MinervaAI) and, in particular, [Gryphe](https://huggingface.co/Gryphe) for letting us use it!
75
+ - [Another private Aesir dataset]
76
 
77
+ This is a full finetune.
78
+ Trained until 2 epoch(4000 steps), trained on mistral 0.1 7b base.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
 
80
+ ## Others
81
 
82
+ Undi: If you want to support me, you can [here](https://ko-fi.com/undiai).
83
 
84
+ IkariDev: Visit my [retro/neocities style website](https://ikaridevgit.github.io/) please kek