Mozilla
/

granite-34b-code-instruct-llamafile

Text Generation

Inference Endpoints

Model card Files Files and versions Community

jartine commited on May 26

Commit

deada0d

•

1 Parent(s): e2d5598

Update README.md

Files changed (1) hide show

README.md +8 -6

README.md CHANGED Viewed

@@ -270,12 +270,14 @@ It can be changed, e.g. `--temp 0.8`.
 ## Benchmarks
-|                                   cpu\_info |                           model\_filename |       size |          test |             t/s |
-| -----------------------------------------: | ---------------------------------------: | ---------: | ------------: | --------------: |
-| AMD Ryzen Threadripper PRO 7995WX (znver4) |           granite-34b-code-instruct.Q8\_0 |  33.82 GiB |         pp512 |           94.34 |
-| AMD Ryzen Threadripper PRO 7995WX (znver4) |           granite-34b-code-instruct.Q8\_0 |  33.82 GiB |          tg16 |            5.61 |
-| AMD Ryzen Threadripper PRO 7995WX (znver4) |           granite-34b-code-instruct.Q5\_0 |  22.03 GiB |         pp512 |           95.08 |
-| AMD Ryzen Threadripper PRO 7995WX (znver4) |           granite-34b-code-instruct.Q5\_0 |  22.03 GiB |          tg16 |            7.78 |
 ## About Quantization

 ## Benchmarks
+| hardware                                   | model\_filename                          | size       | test          | t/s             |
+| :----------------------------------------- | :--------------------------------------- | ---------: | ------------: | --------------: |
+| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | pp512         | 186.14          |
+| Apple M2 Ultra (Metal GPU)                 | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | tg16          | 14.13           |
+| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | pp512         | 94.34           |
+| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0          | 33.82 GiB  | tg16          | 5.61            |
+| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | pp512         | 95.08           |
+| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0          | 22.03 GiB  | tg16          | 7.78            |
 ## About Quantization