TheBloke commited on
Commit
a4d7e9c
1 Parent(s): 8099567

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -2
README.md CHANGED
@@ -33,7 +33,7 @@ Huggingface Hub has a 50GB per-file limit. I have therefore been forced to spli
33
 
34
  I did this using the simple *nix command `split`.
35
 
36
- To join the files on any *nix system, run:
37
  ```
38
  cat gptq_model-4bit--1g.JOINBEFOREUSE.split-*.safetensors > gptq_model-4bit--1g.safetensors
39
  ```
@@ -43,6 +43,8 @@ To join the files on Windows, open a Command Prompt and run:
43
  COPY /B gptq_model-4bit--1g.JOINBEFOREUSE.split-a.safetensors + gptq_model-4bit--1g.JOINBEFOREUSE.split-b.safetensors + gptq_model-4bit--1g.JOINBEFOREUSE.split-c.safetensors gptq_model-4bit--1g.safetensors
44
  ```
45
 
 
 
46
  The SHA256SUM of the joined file will be:
47
  ```
48
  9cc359fa266d2523566e818ca58e8782718b25cc2e714cb5449b7841e1c59830 gptq_model-4bit--1g.safetensors
@@ -118,7 +120,19 @@ snapshot_download(repo_id="TheBloke/BLOOMChat-176B-v1-GPTQ",
118
 
119
  If you want to download the group_size 128g file instead, add `revision="group_size_128g"` to the above command.
120
 
121
- Now join the three `split` files up as described above, to get a single `safetensors` file.
 
 
 
 
 
 
 
 
 
 
 
 
122
 
123
  Then try the following example code:
124
 
 
33
 
34
  I did this using the simple *nix command `split`.
35
 
36
+ To join the files on any *nix system, you can run:
37
  ```
38
  cat gptq_model-4bit--1g.JOINBEFOREUSE.split-*.safetensors > gptq_model-4bit--1g.safetensors
39
  ```
 
43
  COPY /B gptq_model-4bit--1g.JOINBEFOREUSE.split-a.safetensors + gptq_model-4bit--1g.JOINBEFOREUSE.split-b.safetensors + gptq_model-4bit--1g.JOINBEFOREUSE.split-c.safetensors gptq_model-4bit--1g.safetensors
44
  ```
45
 
46
+ Or for Python code for joining the files, see the Python section below.
47
+
48
  The SHA256SUM of the joined file will be:
49
  ```
50
  9cc359fa266d2523566e818ca58e8782718b25cc2e714cb5449b7841e1c59830 gptq_model-4bit--1g.safetensors
 
120
 
121
  If you want to download the group_size 128g file instead, add `revision="group_size_128g"` to the above command.
122
 
123
+ Now join the three `split` files, which can be done with the following Python code:
124
+ ```python
125
+ import glob
126
+
127
+ # Get the list of all files matching the pattern
128
+ files = glob.glob('gptq_model-4bit--1g.JOINBEFOREUSE.split-*.safetensors')
129
+
130
+ # Open the output file in binary write mode
131
+ with open('gptq_model-4bit--1g.safetensors', 'wb') as outfile:
132
+ for filename in files:
133
+ with open(filename, 'rb') as infile:
134
+ outfile.write(infile.read())
135
+ ```
136
 
137
  Then try the following example code:
138