metadata
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
tags:
- gemma
- gguf
Gemma 2B Instruct GGUF
Contains Q4 & Q8 quantized GGUFs for google/gemma
Perf
| Variant | Device | Perf |
|---|---|---|
| F16 | M1 Pro 10-core GPU | 30 tok/s |
| Q4 | RTX 2070S | 40 tok/s |
| M1 Pro 10-core GPU | 90 tok/s | |
| Q8 | RTX 2070S | 25 tok/s |
| M1 Pro 10-core GPU | 54 tok/s |