Error loading gpt4-x-alpaca-13b-native-4bit-128g on Alienware M15 Ryzen Edition R5 laptop
I am having trouble loading the gpt4-x-alpaca-13b-native-4bit-128g model on my Alienware M15 Ryzen Edition R5 laptop. I am getting the following error message:
RuntimeError: [enforce fail at C:\cb\pytorch_1000000000000\work\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 327690240 bytes.
I have checked the installation instructions and the documentation for the oobabooga web UI, and I believe that the oobabooga web UI is installed correctly and is compatible with my hardware configuration. I have also tried restarting the oobabooga web UI, but the error persists. Same issue occurred when trying to run Vicuna even though, theoretically my laptop should be able to run it.
My laptop specs are as follows:
CPU: AMD Ryzen 7 5800H
GPU: NVIDIA GeForce RTX 3050 Ti Laptop GPU
RAM: 16GB
Storage: 512GB SSD
I would appreciate any help you can provide in resolving this issue.
Thank you.
Read this post and see if it helps: https://huggingface.co/anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g/discussions/15