@winglian on Hugging Face: "Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in…"

Hugging Face

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Back to feed

winglian

posted an update Jan 8

Post

Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to @abacaj and @vikhyatk for their help with gradient checkpointing and flash attention fixes.

alpaca finetune: openaccess-ai-collective/phi2-alpaca
wandb: https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
merged PR: https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058

sbrandeis

Jan 10

alpaca finetune: https://huggingface.co/openaccess-ai-collective/phi2-alpaca

That link gives a 404 error to me, is the model public?

winglian

Jan 10

Sorry about that! Public now.

mrfakename

Jan 15

Nice! @winglian do you know what the largest model you can fit on a single 24GB GPU (w/o LoRA/QLoRA) is?

samusenps

Jan 21

Hey question for @brucethemoose & @LoneStriker I recall you guys mentioning model merging with only 24GB VRam & 32GB RAM as well as swap being a must in your workflows for model merging. Do you mind sharing some of your workflow to enlighten us gpu poor🫠

SicariusSicariiStuff

Jun 10

@winglian , I want to start by expressing my immense gratitude for all your hard work! Axolotl has become my go-to repository for training. I have a quick question: Is there any possibility we might see FlashAttention for Qwen2 in the near future? 😊

In this post