robinsmits commited on
Commit
5fc5555
1 Parent(s): 1a51194

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md CHANGED
@@ -23,6 +23,47 @@ Finetuning was performed on the Dutch [BramVanroy/ultrachat_200k_dutch](https://
23
 
24
  See [Qwen/Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) for all information about the base model.
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## Intended uses & limitations
27
 
28
  As with all LLM's this model can also experience bias and hallucinations. Regardless of how you use this model always perform the necessary testing and validation.
 
23
 
24
  See [Qwen/Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) for all information about the base model.
25
 
26
+ ## Model usage
27
+
28
+ A basic example of how to use the finetuned model.
29
+
30
+ ```
31
+ import torch
32
+ from transformers import AutoTokenizer, AutoModelForCausalLM
33
+
34
+ device = 'cuda'
35
+ model_name = 'robinsmits/Qwen1.5-7B-Dutch-Chat-Sft-Bf16'
36
+
37
+ model = AutoModelForCausalLM.from_pretrained(model_name,
38
+ device_map = "auto",
39
+ torch_dtype = torch.bfloat16)
40
+
41
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
42
+
43
+ messages = [{"role": "user", "content": "Hoi hoe gaat het ermee? Wat kun je me vertellen over appels?"}]
44
+
45
+ encoded_ids = tokenizer.apply_chat_template(messages,
46
+ add_generation_prompt = True,
47
+ return_tensors = "pt")
48
+
49
+ generated_ids = model.generate(input_ids = encoded_ids.to(device),
50
+ max_new_tokens = 256,
51
+ do_sample = True)
52
+ decoded = tokenizer.batch_decode(generated_ids)
53
+ print(decoded[0])
54
+ ```
55
+
56
+ Below the chat template with the generated output.
57
+
58
+ ```
59
+ <|im_start|>system
60
+ Je bent een behulpzame AI assistent<|im_end|>
61
+ <|im_start|>user
62
+ Hoi hoe gaat het ermee? Wat kun je me vertellen over appels?<|im_end|>
63
+ <|im_start|>assistant
64
+ Hallo! Appels zijn zo'n heerlijk fruit. Ze komen in verschillende kleuren en smaken, zoals rode, witte, en goudbruine appels. Appels bevatten veel vezels en vitamines die goed zijn voor je gezondheid. Je kunt ze op verschillende manieren eten, bijvoorbeeld gesneden met wat kaneel of in een smoothie. Wil je meer weten over de voedingswaarde van appels of heb je interesse in andere fruitsoorten?<|im_end|>
65
+ ```
66
+
67
  ## Intended uses & limitations
68
 
69
  As with all LLM's this model can also experience bias and hallucinations. Regardless of how you use this model always perform the necessary testing and validation.