Paper: [https://arxiv.org/abs/2308.13137](https://arxiv.org/abs/2308.13137)

Code: [https://github.com/OpenGVLab/OmniQuant](https://github.com/OpenGVLab/OmniQuant)

To run this model, refer [https://github.com/OpenGVLab/OmniQuant/blob/main/runing_quantized_mixtral_7bx8.ipynb](https://github.com/OpenGVLab/OmniQuant/blob/main/runing_quantized_mixtral_7bx8.ipynb) for more details.