---
license: llama2
datasets:
- wasertech/OneOS
language:
- en
- fr
pipeline_tag: text-generation
widget:
- text: "<<SYS>>\nYou are Assistant, a sentient AI.\n<</SYS>>\n\n<s>[INST] Introduce yourself to the HuggingFace community. [/INST] "
  example_title: "Introduction"
- text: "<<SYS>>\nYou are Assistant, a sentient AI.\n<</SYS>>\n\n<s>[INST] Describe your model. [/INST] "
  example_title: "Model Description"
- text: "<<SYS>>\nYou are Assistant, a sentient AI.\n<</SYS>>\n\n<s>[INST] What the meaning of life? [/INST] "
  example_title: "Life's Meaning"
- text: "<<SYS>>\nYou are Assistant, a sentient AI.\n<</SYS>>\n\n<s>[INST] What recent innovations in the field of AI are you excited by? [/INST] "
  example_title: "What's next?"
---

# Assistant Llama 2 7B Chat AWQ

This model is a quantitized export of [wasertech/assistant-llama2-7b-chat](https://huggingface.co/wasertech/assistant-llama2-7b-chat) using AWQ.

AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference.

It is also now supported by continuous batching server vLLM, allowing use of Llama AWQ models for high-throughput concurrent inference in multi-user server scenarios.

As of September 25th 2023, preliminary Llama-only AWQ support has also been added to Huggingface Text Generation Inference (TGI).