mlabonne

/

BigLlama-3.1-1T-Instruct

like 74

+1
Will my GTX1650 laptop be able to run?

int4 quantized should be able to run on NASA cluster pretty decently

To run this model, the recommended hardware is 4 nodes of 8xH100s in FP8. But you can just run some layers of the 405B model multiple times during inference to get the same result.

can't wait for gguf quant BigLlama-3.1-1T-Instruct.IQ-negative-0000000.9_K_M-imat.gguf
that was amazing model and any office pc can run it :)

nah dawg i'm waiting for BigLlama-3.1-1T-Instruct.IQ0.googol_XS-imat.gguf

that's gonna run even on the crappiest hardware

To run this model, the recommended hardware is 4 nodes of 8xH100s in FP8. But you can just run some layers of the 405B model multiple times during inference to get the same result.

well so basically not a model launchable by the average ChatGPT user

What is 1T in the name mean?

What is 1T in the name mean?

1 trillion parameters.

What is 1T in the name mean?

1 trillion parameters.

this also means the model weighs a staggering 2 TB and cannot be loaded on ANY home PC.

we are not in the year 3000 and have not engineered PB (petabytes) of SSD storage, or GPUs with TB (terabytes) of VRAM.

we are not in the year 3000 and have not engineered PB (petabytes) of SSD storage, or GPUs with TB (terabytes) of VRAM.

but it can run on your brain :)
just upload model directly to you brain use repo thinkingmeat on github and neural interface cockonbrain v3 .
Like in old classic movie Johnny Mnemonic

i use 450B and

What is 1T in the name mean?

1 trillion parameters.

i use 450B version and it seem work well, except that the context length 128K is quite shot, i think it is better if any one can make context length longer

i use 450B and

What is 1T in the name mean?

1 trillion parameters.

i use 450B version and it seem work well, except that the context length 128K is quite shot, i think it is better if any one can make context length longer

For a 405B 128K may feel short, but it is indeed among the biggest. Any 1Ms coming yet? We need to battle Gemini 1.5 Pro

i use 450B and

What is 1T in the name mean?

1 trillion parameters.

i use 450B version and it seem work well, except that the context length 128K is quite shot, i think it is better if any one can make context length longer

For a 405B 128K may feel short, but it is indeed among the biggest. Any 1Ms coming yet? We need to battle Gemini 1.5 Pro

agree with you, we need 1M context lenght or more