Text Generation
Transformers
Safetensors
llama
code
conversational
Eval Results
text-generation-inference
Inference Endpoints
wyt2000 commited on
Commit
b0add30
1 Parent(s): 5675004

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +127 -139
README.md CHANGED
@@ -3,146 +3,135 @@ license: llama2
3
  license_name: deepseek
4
  license_link: LICENSE
5
  datasets:
 
6
  - ise-uiuc/Magicoder-Evol-Instruct-110K
7
  library_name: transformers
8
  pipeline_tag: text-generation
9
  tags:
10
  - code
11
  model-index:
12
- - name: InverseCoder-CL-7B
13
- results:
14
- - task:
15
- type: text-generation
16
- dataset:
17
- type: openai_humaneval
18
- name: HumanEval
19
- metrics:
20
- - name: pass@1
21
- type: pass@1
22
- value: 0.762
23
- verified: false
24
-
25
- - task:
26
- type: text-generation
27
- dataset:
28
- type: openai_humaneval
29
- name: HumanEval(+)
30
- metrics:
31
- - name: pass@1
32
- type: pass@1
33
- value: 0.720
34
- verified: false
35
-
36
- - task:
37
- type: text-generation
38
- dataset:
39
- type: mbpp
40
- name: MBPP
41
- metrics:
42
- - name: pass@1
43
- type: pass@1
44
- value: 0.706
45
- verified: false
46
-
47
- - task:
48
- type: text-generation
49
- dataset:
50
- type: mbpp
51
- name: MBPP(+)
52
- metrics:
53
- - name: pass@1
54
- type: pass@1
55
- value: 0.601
56
- verified: false
57
-
58
- - task:
59
- type: text-generation
60
- dataset:
61
- type: ds1000
62
- name: DS-1000 (Overall Completion)
63
- metrics:
64
- - name: pass@1
65
- type: pass@1
66
- value: 0.399
67
- verified: false
68
-
69
- - task:
70
- type: text-generation
71
- dataset:
72
- type: nuprl/MultiPL-E
73
- name: MultiPL-HumanEval (Java)
74
- metrics:
75
- - name: pass@1
76
- type: pass@1
77
- value: 0.487
78
- verified: false
79
-
80
- - task:
81
- type: text-generation
82
- dataset:
83
- type: nuprl/MultiPL-E
84
- name: MultiPL-HumanEval (JavaScript)
85
- metrics:
86
- - name: pass@1
87
- type: pass@1
88
- value: 0.619
89
- verified: false
90
-
91
- - task:
92
- type: text-generation
93
- dataset:
94
- type: nuprl/MultiPL-E
95
- name: MultiPL-HumanEval (C++)
96
- metrics:
97
- - name: pass@1
98
- type: pass@1
99
- value: 0.526
100
- verified: false
101
-
102
- - task:
103
- type: text-generation
104
- dataset:
105
- type: nuprl/MultiPL-E
106
- name: MultiPL-HumanEval (PHP)
107
- metrics:
108
- - name: pass@1
109
- type: pass@1
110
- value: 0.552
111
- verified: false
112
-
113
- - task:
114
- type: text-generation
115
- dataset:
116
- type: nuprl/MultiPL-E
117
- name: MultiPL-HumanEval (Swift)
118
- metrics:
119
- - name: pass@1
120
- type: pass@1
121
- value: 0.530
122
- verified: false
123
-
124
- - task:
125
- type: text-generation
126
- dataset:
127
- type: nuprl/MultiPL-E
128
- name: MultiPL-HumanEval (Rust)
129
- metrics:
130
- - name: pass@1
131
- type: pass@1
132
- value: 0.461
133
- verified: false
134
-
135
- - task:
136
- type: text-generation
137
- dataset:
138
- type: nuprl/MultiPL-E
139
- name: MultiPL-HumanEval (Average for non-python languages)
140
- metrics:
141
- - name: pass@1
142
- type: pass@1
143
- value: 0.529
144
- verified: false
145
-
146
  ---
147
  <div align="center">
148
  <img src="https://huggingface.co/wyt2000/InverseCoder-CL-7B/resolve/main/assets/logo.png" style="zoom:25%;" />
@@ -155,8 +144,7 @@ model-index:
155
  InverseCoder is a series of code LLMs instruction-tuned by generating data from itself through Inverse-Instruct.
156
 
157
  ## Models
158
- | | Base Model | InverseCoder |
159
- | --- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
160
- | 7B | [codellama/CodeLlama-7b-Python-hf](https://huggingface.co/codellama/CodeLlama-7b-Python-hf) | [wyt2000/InverseCoder-CL-7B](https://huggingface.co/wyt2000/InverseCoder-CL-7B) |
161
- | 6.7B | [deepseek-ai/deepseek-coder-6.7b-base](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base) | [wyt2000/InverseCoder-DS-6.7B](https://huggingface.co/wyt2000/InverseCoder-DS-6.7B) |
162
-
 
3
  license_name: deepseek
4
  license_link: LICENSE
5
  datasets:
6
+ - wyt2000/InverseCoder-CL-7B-Evol-Instruct-90K
7
  - ise-uiuc/Magicoder-Evol-Instruct-110K
8
  library_name: transformers
9
  pipeline_tag: text-generation
10
  tags:
11
  - code
12
  model-index:
13
+ - name: InverseCoder-CL-7B
14
+ results:
15
+ - task:
16
+ type: text-generation
17
+ dataset:
18
+ type: openai_humaneval
19
+ name: HumanEval
20
+ metrics:
21
+ - name: pass@1
22
+ type: pass@1
23
+ value: 0.762
24
+ verified: false
25
+ - task:
26
+ type: text-generation
27
+ dataset:
28
+ type: openai_humaneval
29
+ name: HumanEval(+)
30
+ metrics:
31
+ - name: pass@1
32
+ type: pass@1
33
+ value: 0.72
34
+ verified: false
35
+ - task:
36
+ type: text-generation
37
+ dataset:
38
+ type: mbpp
39
+ name: MBPP
40
+ metrics:
41
+ - name: pass@1
42
+ type: pass@1
43
+ value: 0.706
44
+ verified: false
45
+ - task:
46
+ type: text-generation
47
+ dataset:
48
+ type: mbpp
49
+ name: MBPP(+)
50
+ metrics:
51
+ - name: pass@1
52
+ type: pass@1
53
+ value: 0.601
54
+ verified: false
55
+ - task:
56
+ type: text-generation
57
+ dataset:
58
+ type: ds1000
59
+ name: DS-1000 (Overall Completion)
60
+ metrics:
61
+ - name: pass@1
62
+ type: pass@1
63
+ value: 0.399
64
+ verified: false
65
+ - task:
66
+ type: text-generation
67
+ dataset:
68
+ type: nuprl/MultiPL-E
69
+ name: MultiPL-HumanEval (Java)
70
+ metrics:
71
+ - name: pass@1
72
+ type: pass@1
73
+ value: 0.487
74
+ verified: false
75
+ - task:
76
+ type: text-generation
77
+ dataset:
78
+ type: nuprl/MultiPL-E
79
+ name: MultiPL-HumanEval (JavaScript)
80
+ metrics:
81
+ - name: pass@1
82
+ type: pass@1
83
+ value: 0.619
84
+ verified: false
85
+ - task:
86
+ type: text-generation
87
+ dataset:
88
+ type: nuprl/MultiPL-E
89
+ name: MultiPL-HumanEval (C++)
90
+ metrics:
91
+ - name: pass@1
92
+ type: pass@1
93
+ value: 0.526
94
+ verified: false
95
+ - task:
96
+ type: text-generation
97
+ dataset:
98
+ type: nuprl/MultiPL-E
99
+ name: MultiPL-HumanEval (PHP)
100
+ metrics:
101
+ - name: pass@1
102
+ type: pass@1
103
+ value: 0.552
104
+ verified: false
105
+ - task:
106
+ type: text-generation
107
+ dataset:
108
+ type: nuprl/MultiPL-E
109
+ name: MultiPL-HumanEval (Swift)
110
+ metrics:
111
+ - name: pass@1
112
+ type: pass@1
113
+ value: 0.53
114
+ verified: false
115
+ - task:
116
+ type: text-generation
117
+ dataset:
118
+ type: nuprl/MultiPL-E
119
+ name: MultiPL-HumanEval (Rust)
120
+ metrics:
121
+ - name: pass@1
122
+ type: pass@1
123
+ value: 0.461
124
+ verified: false
125
+ - task:
126
+ type: text-generation
127
+ dataset:
128
+ type: nuprl/MultiPL-E
129
+ name: MultiPL-HumanEval (Average for non-python languages)
130
+ metrics:
131
+ - name: pass@1
132
+ type: pass@1
133
+ value: 0.529
134
+ verified: false
 
 
 
 
 
 
 
 
 
 
 
 
135
  ---
136
  <div align="center">
137
  <img src="https://huggingface.co/wyt2000/InverseCoder-CL-7B/resolve/main/assets/logo.png" style="zoom:25%;" />
 
144
  InverseCoder is a series of code LLMs instruction-tuned by generating data from itself through Inverse-Instruct.
145
 
146
  ## Models
147
+ | | Base Model | InverseCoder | Dataset |
148
+ | --- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
149
+ | 7B | [codellama/CodeLlama-7b-Python-hf](https://huggingface.co/codellama/CodeLlama-7b-Python-hf) | [wyt2000/InverseCoder-CL-7B](https://huggingface.co/wyt2000/InverseCoder-CL-7B) | [wyt2000/InverseCoder-CL-7B-Evol-Instruct-90K](https://huggingface.co/datasets/wyt2000/InverseCoder-DS-6.7B-Evol-Instruct-90K) |
150
+ | 6.7B | [deepseek-ai/deepseek-coder-6.7b-base](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base) | [wyt2000/InverseCoder-DS-6.7B](https://huggingface.co/wyt2000/InverseCoder-DS-6.7B) | [wyt2000/InverseCoder-DS-6.7B-Evol-Instruct-90K](https://huggingface.co/datasets/wyt2000/InverseCoder-DS-6.7B-Evol-Instruct-90K) |