Spaces:

ali-vilab

/

modelscope-text-to-video-synthesis

like 1.45k

Starting on L4

Hi, @zedwork
I think the following would work:

sudo apt install ffmpeg
git clone https://huggingface.co/spaces/damo-vilab/modelscope-text-to-video-synthesis
cd modelscope-text-to-video-synthesis
pip install -r requirements.txt
python app.py

pardon my ignorance (n00b), but how would we actually go about getting this into the 1111 stable diffusion web UI?

Hi, @nerkderk What I wrote is just a way to run this Space locally, and not the way to integrate this to the AUTOMATIC1111 web UI. Sorry for the confusion.

@hysts thanks. if i follow your steps to run it locally how do I then use it?
also does that work on windows or mac?

@nerkderk As for how to use it, you'll see a message like Running on local URL: http://127.0.0.1:7860 in your terminal after running python app.py, so you can just open the URL with your browser.
As for the required environment, this demo uses about 16GB of RAM (main memory) and 16GB of VRAM (GPU memory), so, you first need to make sure that your local environment satisfies at least these. I only tested this on Ubuntu, so I'm not sure if it'll work on Windows. Also, I'm not 100% sure, but I don't think it will run on Mac.

do u think that it can run on a 3090?

Hi, @Manni1000
3090 seems to have 24GB VRAM, so regarding the VRAM, I think it's enough.

Hi. If you install it locally, is there a way to increase the video length?

Hi, @johnblues Yes. The maximum video length is limited to 32 using the environment variable MAX_NUM_FRAMES in this Space so as not to slow down the demo, but you can change it if you duplicate this Space or install it locally. Note that increasing the number of frames also increases memory usage.

@hysts Thanks. I thought it would only increase the time to process it, I hadn't though that it would also eat the memory as well.

@hysts Where/in which file are you able to change the environment variable MAX_NUM_FRAMES when you are running it locally?

Hi, @Hypsy
You can do something like this if you are using Ubuntu:

export MAX_NUM_FRAMES=200
python app.py

or

MAX_NUM_FRAMES=200 python app.py

As I don't have Windows environment, I'm not sure if it's correct, but it seems you can do this in Windows:

set MAX_NUM_FRAMES=200
python app.py

BTW, the default value of MAX_NUM_FRAMES is actually 200 (see here), so I don't think you have to manually set the environment variable in most cases.

All good had a script running from a week old so the number of frames slider wasn't there yet. Played around with it, but sometimes i'm getting this error. It appears when getting above 25 number of frames. It keeps creating videos but they are really abstract like lines etc. Running on a 3090. Any idea how to solve this

Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users...\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users...\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 162, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host