meta-llama

/

Llama-3.1-405B-FP8

like 97

Follow

Meta Llama 7,626

The modules_to_not_convert need to be the linear layers to work with vLLM, or they are ignored. Setting them to parent modules does not work.

Also, updated the _name_or_path field to the correct HF model id.