Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
chansung 
posted an update Mar 20
Post
2536
Realize LLM powered idea on Hugging Face Space.

I made Space for you to duplicate, then it comes with Gradio and LLM served by Hugging Face's efficient Text Generation Inference(TGI) framework packed into a single machine.

It provides a sample app code snippet with gr.ChatInterface. However, it is not limited to chat usage, but you can leverage the efficiency of TGI for any sort of apps built in Gradio.

Have you ever enjoyed playing with Hugging Chat? Then, you will enjoy writing your own idea with this. Because both are built on top of TGI!

Focus on your app code, and go beyond chat!

chansung/gradio_together_tgi
radames
•
edited Mar 21

thanks @chansung this is so helpful! btw you could launch a quantize version here https://huggingface.co/spaces/chansung/gradio_together_tgi/blob/main/entrypoint.sh.template#L11 and even try running this on CPU.

·
chansung

awesome! going to add one more env var to switch mode then :)

In this post