Update README.md
Browse files
README.md
CHANGED
|
@@ -4,12 +4,17 @@ datasets:
|
|
| 4 |
- ehartford/wizard_vicuna_70k_unfiltered
|
| 5 |
tags:
|
| 6 |
- uncensored
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
# Overview
|
| 10 |
Fine-tuned [Llama-2 70B](https://huggingface.co/TheBloke/Llama-2-70B-fp16) with an uncensored/unfiltered Wizard-Vicuna conversation dataset [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered).
|
| 11 |
[QLoRA](https://arxiv.org/abs/2305.14314) was used for fine-tuning. The model was trained for three epochs on a single NVIDIA A100 80GB GPU instance, taking ~1 week to train.
|
| 12 |
|
|
|
|
|
|
|
| 13 |
Special thanks to [George Sung](https://huggingface.co/georgesung) for creating [llama2_7b_chat_uncensored](https://huggingface.co/georgesung/llama2_7b_chat_uncensored), and to [Eric Hartford](https://huggingface.co/ehartford/) for creating [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered)
|
| 14 |
|
| 15 |
The version here is the fp16 HuggingFace model.
|
|
|
|
| 4 |
- ehartford/wizard_vicuna_70k_unfiltered
|
| 5 |
tags:
|
| 6 |
- uncensored
|
| 7 |
+
- wizard
|
| 8 |
+
- vicuna
|
| 9 |
+
- llama
|
| 10 |
---
|
| 11 |
|
| 12 |
# Overview
|
| 13 |
Fine-tuned [Llama-2 70B](https://huggingface.co/TheBloke/Llama-2-70B-fp16) with an uncensored/unfiltered Wizard-Vicuna conversation dataset [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered).
|
| 14 |
[QLoRA](https://arxiv.org/abs/2305.14314) was used for fine-tuning. The model was trained for three epochs on a single NVIDIA A100 80GB GPU instance, taking ~1 week to train.
|
| 15 |
|
| 16 |
+
Please note that LLama 2 Base model has its inherit biases.
|
| 17 |
+
|
| 18 |
Special thanks to [George Sung](https://huggingface.co/georgesung) for creating [llama2_7b_chat_uncensored](https://huggingface.co/georgesung/llama2_7b_chat_uncensored), and to [Eric Hartford](https://huggingface.co/ehartford/) for creating [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered)
|
| 19 |
|
| 20 |
The version here is the fp16 HuggingFace model.
|