reciprocate commited on
Commit
5558dfa
1 Parent(s): 9005453

fix model paths in the code & add info about the formatting

Browse files
Files changed (1) hide show
  1. README.md +33 -14
README.md CHANGED
@@ -16,7 +16,7 @@ extra_gated_fields:
16
  Organization or Affiliation: text
17
  I ALLOW Stability AI to email me about new model releases: checkbox
18
  ---
19
- # `Stable Zephyr 3B`
20
 
21
  ## Model Description
22
 
@@ -25,33 +25,52 @@ extra_gated_fields:
25
 
26
  ## Usage
27
 
28
- Get started generating text with `Stable Zephyr 3B` by using the following code snippet:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
  ```python
31
  from transformers import AutoModelForCausalLM, AutoTokenizer
32
- tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablelm-zephyr-3b-dpo")
 
33
  model = AutoModelForCausalLM.from_pretrained(
34
- "stable-zephyr-3b",
35
  trust_remote_code=True,
36
- torch_dtype="auto",
37
  )
38
- model.cuda()
39
- prompt = "<|user|>\nIn the field of quantum physics, what is superposition, and how does it relate to the phenomenon of quantum entanglement?<|endoftext|>\n<|assistant|>\n"
40
- inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
 
41
  tokens = model.generate(
42
- **inputs,
43
  max_new_tokens=1024,
44
- temperature=0.7,
45
- top_p=0.95,
46
- do_sample=True,
47
  )
48
- print(tokenizer.decode(tokens[0], skip_special_tokens=True))
 
49
  ```
50
 
51
  ## Model Details
52
 
53
  * **Developed by**: [Stability AI](https://stability.ai/)
54
- * **Model type**: `StableLM Zephyr 3B` models are auto-regressive language models based on the transformer decoder architecture.
55
  * **Language(s)**: English
56
  * **Library**: [Alignment Handbook](https://github.com/huggingface/alignment-handbook.git)
57
  * **Finetuned from model**: [stabilityai/stablelm-3b-4e1t](https://huggingface.co/stabilityai/stablelm-3b-4e1t)
 
16
  Organization or Affiliation: text
17
  I ALLOW Stability AI to email me about new model releases: checkbox
18
  ---
19
+ # `StableLM Zephyr 3B`
20
 
21
  ## Model Description
22
 
 
25
 
26
  ## Usage
27
 
28
+ `StableLM Zephyr 3B` uses the following instruction format:
29
+ ```
30
+ <|user|>
31
+ List 10 synonyms for the word "tiny"<|endoftext|>
32
+ <|assistant|>
33
+ 1. Dwarf
34
+ 2. Little
35
+ 3. Petite
36
+ 4. Miniature
37
+ 5. Small
38
+ 6. Compact
39
+ 7. Cramped
40
+ 8. Wee
41
+ 9. Nibble
42
+ 10. Crumble<|endoftext|>
43
+ ```
44
+
45
+ This format is also available through the tokenizer's `apply_chat_template` method:
46
 
47
  ```python
48
  from transformers import AutoModelForCausalLM, AutoTokenizer
49
+
50
+ tokenizer = AutoTokenizer.from_pretrained('stabilityai/stablelm-zephyr-3b')
51
  model = AutoModelForCausalLM.from_pretrained(
52
+ 'stabilityai/stablelm-zephyr-3b',
53
  trust_remote_code=True,
54
+ device_map="auto"
55
  )
56
+
57
+ prompt = [{'role': 'user', 'content': 'List 10 synonyms for the word "tiny"'}]
58
+ inputs = tokenizer.apply_chat_template(prompt, add_generation_prompt=True, return_tensors='pt')
59
+
60
  tokens = model.generate(
61
+ inputs.to(model.device),
62
  max_new_tokens=1024,
63
+ temperature=0.8,
64
+ do_sample=True
 
65
  )
66
+
67
+ print(tokenizer.decode(tokens[0], skip_special_tokens=False))
68
  ```
69
 
70
  ## Model Details
71
 
72
  * **Developed by**: [Stability AI](https://stability.ai/)
73
+ * **Model type**: `StableLM Zephyr 3B` model is an auto-regressive language model based on the transformer decoder architecture.
74
  * **Language(s)**: English
75
  * **Library**: [Alignment Handbook](https://github.com/huggingface/alignment-handbook.git)
76
  * **Finetuned from model**: [stabilityai/stablelm-3b-4e1t](https://huggingface.co/stabilityai/stablelm-3b-4e1t)