hugging-quants

/

Meta-Llama-3.1-70B-Instruct-AWQ-INT4

like 87

Follow

Hugging Quants 110

Very nice, looks great to me!

Question about the "instead of" comments in the final snippet. If the OpenAI API works against the locally deployed model, maybe we could add another snippet to show how to use it?

Thanks for the review @pcuenq ! Maybe we should then add a collapsable with the openai snippet to avoid having a huge model card? Or just add it afterwards? Note that I've just included vLLM too, and that may be already way too much for the card, WDYT? I'm all ears on what's the best 🤗

I think it's great to show different options so users can apply the solution that works best for them. We can use a ToC at the beginning of the model card or collapsable sub-sections, as you prefer :)

Just applied those changes, thanks for the review @pcuenq , feel free to merge whenever! Then I'll apply the updates to the rest of the supported models 🤗