Hugging Face's logo

lucadiliello
/
opt-30b-deepspeed-inference-fp16-shard-2

Text Generation
Transformers
opt
Inference Endpoints
Model card Files Community
lucadiliello commited on
Commit
7891f1a
•
1 Parent(s): 8c8b767

Update ds_inference_config.json

Browse files
Files changed (1) hide show
  1. ds_inference_config.json +1 -1
ds_inference_config.json CHANGED
@@ -1 +1 @@
1
- {"type": "bloom", "base_dir": "/science/lucadiliello/models/facebook/sharded/opt-30b-deepspeed-inference-fp16-shard-2", "checkpoints": {"non_tp": ["non-tp.pt"], "tp": ["tp_00_00.pt", "tp_01_00.pt", "tp_00_01.pt", "tp_01_01.pt", "tp_00_02.pt", "tp_01_02.pt", "tp_00_03.pt", "tp_01_03.pt", "tp_00_04.pt", "tp_01_04.pt", "tp_00_05.pt", "tp_01_05.pt", "tp_00_06.pt", "tp_01_06.pt", "tp_00_07.pt", "tp_01_07.pt"]}, "version": 1.0, "parallelization": "tp", "tp_size": 2, "dtype": "float16"}
 
1
+ {"type": "ds_model", "base_dir": "/science/lucadiliello/models/facebook/sharded/opt-30b-deepspeed-inference-fp16-shard-2", "checkpoints": {"non_tp": ["non-tp.pt"], "tp": ["tp_00_00.pt", "tp_01_00.pt", "tp_00_01.pt", "tp_01_01.pt", "tp_00_02.pt", "tp_01_02.pt", "tp_00_03.pt", "tp_01_03.pt", "tp_00_04.pt", "tp_01_04.pt", "tp_00_05.pt", "tp_01_05.pt", "tp_00_06.pt", "tp_01_06.pt", "tp_00_07.pt", "tp_01_07.pt"]}, "version": 1.0, "parallelization": "tp", "tp_size": 2, "dtype": "float16"}