guoyww/AnimateDiff · Apply for community grant: Academic project (gpu)

We proposed AnimateDiff, a framework to animate the existing DreamBooth/LoRA model in the community without specific tuning. At the core of the proposed framework is to append a newly-initialized motion modeling module to the frozen based text-to-image model, and train it on video clips thereafter to distill a reasonable motion prior. Once trained, by simply injecting this motion modeling module, all personalized versions derived from the same base one readily become text-driven models that produce diverse and personalized animated images.

Project page: https://animatediff.github.io/
Github: https://github.com/guoyww/AnimateDiff