kiddothe2b commited on
Commit
3f2aa1f
1 Parent(s): e917aad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -6
README.md CHANGED
@@ -3,8 +3,7 @@ license: cc-by-nc-sa-4.0
3
  pipeline_tag: fill-mask
4
  language: en
5
  tags:
6
- - long_documents
7
- - hierarchical_transformers
8
  datasets:
9
  - wikipedia
10
  model-index:
@@ -34,9 +33,10 @@ Note that this model is primarily aimed at being fine-tuned on tasks that use th
34
  You can use this model directly with a pipeline for masked language modeling:
35
 
36
  ```python
37
- from transformers import pipeline
38
- mlm_model = pipeline('fill-mask', model='kiddothe2b/hierarchical-transformer-I3-mini-1024', trust_remote_code=True)
39
- mlm_model("Hello I'm a <mask> model.")
 
40
  ```
41
 
42
  You can also fine-tun it for SequenceClassification, SequentialSentenceClassification, and MultipleChoice down-stream tasks:
@@ -97,7 +97,8 @@ The following hyperparameters were used during training:
97
  - Tokenizers 0.11.6
98
 
99
 
100
- ##Citing
 
101
  If you use HAT in your research, please cite [An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification](https://arxiv.org/abs/xxx)
102
 
103
  ```
 
3
  pipeline_tag: fill-mask
4
  language: en
5
  tags:
6
+ - long-documents
 
7
  datasets:
8
  - wikipedia
9
  model-index:
 
33
  You can use this model directly with a pipeline for masked language modeling:
34
 
35
  ```python
36
+ from transformers import AutoTokenizer, AutoModelforForMaskedLM
37
+ tokenizer = AutoTokenizer.from_pretrained("kiddothe2b/hierarchical-transformer-I3-mini-1024", trust_remote_code=True)
38
+ mlm_model = AutoModelforForMaskedLM(model='kiddothe2b/hierarchical-transformer-I3-mini-1024', trust_remote_code=True)
39
+
40
  ```
41
 
42
  You can also fine-tun it for SequenceClassification, SequentialSentenceClassification, and MultipleChoice down-stream tasks:
 
97
  - Tokenizers 0.11.6
98
 
99
 
100
+ ## Citing
101
+
102
  If you use HAT in your research, please cite [An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification](https://arxiv.org/abs/xxx)
103
 
104
  ```