neuralmagic
/

Nous-Hermes-2-Yi-34B-pruned2.4

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mwitiderrick commited on Mar 5

Commit

06b1d52

•

1 Parent(s): 92d9539

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -73,6 +73,7 @@ test_stage:
     SparseGPTModifier:
       sparsity: 0.5
       sequential_update: true
       targets: ['re:model.layers.\d*$']
 """

     SparseGPTModifier:
       sparsity: 0.5
       sequential_update: true
+      mask_structure: '2:4'
       targets: ['re:model.layers.\d*$']
 """