Tuana commited on
Commit
3158608
1 Parent(s): 5e59482

updated model card

Browse files
Files changed (1) hide show
  1. README.md +46 -52
README.md CHANGED
@@ -7,6 +7,8 @@ license: cc-by-4.0
7
 
8
  # tinyroberta-squad2
9
 
 
 
10
  ## Overview
11
  **Language model:** tinyroberta-squad2
12
  **Language:** English
@@ -38,24 +40,17 @@ This model was distilled using the TinyBERT approach described in [this paper](h
38
  Firstly, we have performed intermediate layer distillation with roberta-base as the teacher which resulted in [deepset/tinyroberta-6l-768d](https://huggingface.co/deepset/tinyroberta-6l-768d).
39
  Secondly, we have performed task-specific distillation with [deepset/roberta-base-squad2](https://huggingface.co/deepset/roberta-base-squad2) as the teacher for further intermediate layer distillation on an augmented version of SQuADv2 and then with [deepset/roberta-large-squad2](https://huggingface.co/deepset/roberta-large-squad2) as the teacher for prediction layer distillation.
40
 
41
- ## Performance
42
- Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/).
43
 
44
- ```
45
- "exact": 78.69114798281817,
46
- "f1": 81.9198998536977,
47
 
48
- "total": 11873,
49
- "HasAns_exact": 76.19770580296895,
50
- "HasAns_f1": 82.66446878592329,
51
- "HasAns_total": 5928,
52
- "NoAns_exact": 81.17746005046257,
53
- "NoAns_f1": 81.17746005046257,
54
- "NoAns_total": 5945
55
  ```
56
 
57
- ## Usage
58
-
59
  ### In Transformers
60
  ```python
61
  from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
@@ -75,54 +70,53 @@ model = AutoModelForQuestionAnswering.from_pretrained(model_name)
75
  tokenizer = AutoTokenizer.from_pretrained(model_name)
76
  ```
77
 
78
- ### In FARM
79
-
80
- ```python
81
- from farm.modeling.adaptive_model import AdaptiveModel
82
- from farm.modeling.tokenization import Tokenizer
83
- from farm.infer import Inferencer
84
-
85
- model_name = "deepset/tinyroberta-squad2"
86
-
87
- # a) Get predictions
88
- nlp = Inferencer.load(model_name, task_type="question_answering")
89
- QA_input = [{"questions": ["Why is model conversion important?"],
90
- "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}]
91
- res = nlp.inference_from_dicts(dicts=QA_input, rest_api_schema=True)
92
 
93
- # b) Load model & tokenizer
94
- model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering")
95
- tokenizer = Tokenizer.load(model_name)
96
  ```
 
 
97
 
98
- ### In haystack
99
- For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/):
100
- ```python
101
- reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2")
102
- # or
103
- reader = TransformersReader(model_name_or_path="deepset/roberta-base-squad2",tokenizer="deepset/roberta-base-squad2")
 
104
  ```
105
 
106
-
107
  ## Authors
108
- Branden Chan: `branden.chan [at] deepset.ai`
109
- Timo Möller: `timo.moeller [at] deepset.ai`
110
- Malte Pietsch: `malte.pietsch [at] deepset.ai`
111
- Tanay Soni: `tanay.soni [at] deepset.ai`
112
- Michel Bartels: `michel.bartels [at] deepset.ai`
113
 
114
  ## About us
115
- ![deepset logo](https://workablehr.s3.amazonaws.com/uploads/account/logo/476306/logo)
116
- We bring NLP to the industry via open source!
117
- Our focus: Industry specific language models & large scale QA systems.
118
-
119
- Some of our work:
 
 
 
 
 
 
 
 
 
120
  - [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert)
121
  - [GermanQuAD and GermanDPR datasets and models (aka "gelectra-base-germanquad", "gbert-base-germandpr")](https://deepset.ai/germanquad)
122
- - [FARM](https://github.com/deepset-ai/FARM)
123
- - [Haystack](https://github.com/deepset-ai/haystack/)
124
 
125
- Get in touch:
 
 
 
 
 
126
  [Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Slack](https://haystack.deepset.ai/community/join) | [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) | [Website](https://deepset.ai)
127
 
128
- By the way: [we're hiring!](http://www.deepset.ai/jobs)
 
7
 
8
  # tinyroberta-squad2
9
 
10
+ This is the *distilled* version of the [deepset/roberta-base-squad2](https://huggingface.co/deepset/roberta-base-squad2) model. This model has a comparable prediction quality and runs at twice the speed of the base model.
11
+
12
  ## Overview
13
  **Language model:** tinyroberta-squad2
14
  **Language:** English
 
40
  Firstly, we have performed intermediate layer distillation with roberta-base as the teacher which resulted in [deepset/tinyroberta-6l-768d](https://huggingface.co/deepset/tinyroberta-6l-768d).
41
  Secondly, we have performed task-specific distillation with [deepset/roberta-base-squad2](https://huggingface.co/deepset/roberta-base-squad2) as the teacher for further intermediate layer distillation on an augmented version of SQuADv2 and then with [deepset/roberta-large-squad2](https://huggingface.co/deepset/roberta-large-squad2) as the teacher for prediction layer distillation.
42
 
43
+ ## Usage
 
44
 
45
+ ### In Haystack
46
+ Haystack is an NLP framework by deepset. You can use this model in a Hasytack pipeline to do question answering at scale (over many documents). To load the model in [Haystack](https://github.com/deepset-ai/haystack/):
 
47
 
48
+ ```python
49
+ reader = FARMReader(model_name_or_path="deepset/tinyroberta-squad2")
50
+ # or
51
+ reader = TransformersReader(model_name_or_path="deepset/tinyroberta-squad2")
 
 
 
52
  ```
53
 
 
 
54
  ### In Transformers
55
  ```python
56
  from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
 
70
  tokenizer = AutoTokenizer.from_pretrained(model_name)
71
  ```
72
 
73
+ ## Performance
74
+ Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/).
 
 
 
 
 
 
 
 
 
 
 
 
75
 
 
 
 
76
  ```
77
+ "exact": 78.69114798281817,
78
+ "f1": 81.9198998536977,
79
 
80
+ "total": 11873,
81
+ "HasAns_exact": 76.19770580296895,
82
+ "HasAns_f1": 82.66446878592329,
83
+ "HasAns_total": 5928,
84
+ "NoAns_exact": 81.17746005046257,
85
+ "NoAns_f1": 81.17746005046257,
86
+ "NoAns_total": 5945
87
  ```
88
 
 
89
  ## Authors
90
+ **Branden Chan:** branden.chan@deepset.ai
91
+ **Timo Möller:** timo.moeller@deepset.ai
92
+ **Malte Pietsch:** malte.pietsch@deepset.ai
93
+ **Tanay Soni:** tanay.soni@deepset.ai
94
+ **Michel Bartels:** michel.bartels@deepset.ai
95
 
96
  ## About us
97
+ <div class="grid lg:grid-cols-2 gap-x-4 gap-y-3">
98
+ <div class="w-full h-40 object-cover mb-2 rounded-lg flex items-center justify-center">
99
+ <img alt="" src="https://huggingface.co/spaces/deepset/README/resolve/main/haystack-logo-colored.svg" class="w-40"/>
100
+ </div>
101
+ <div class="w-full h-40 object-cover mb-2 rounded-lg flex items-center justify-center">
102
+ <img alt="" src="https://huggingface.co/spaces/deepset/README/resolve/main/deepset-logo-colored.svg" class="w-40"/>
103
+ </div>
104
+ </div>
105
+
106
+ [deepset](http://deepset.ai/) is the company behind the open-source NLP framework [Haystack](https://haystack.deepset.ai/) which is designed to help you build production ready NLP systems that use: Question answering, summarization, ranking etc.
107
+
108
+
109
+ Some of our other work:
110
+ - [Distilled roberta-base-squad2 (aka "tinyroberta-squad2")]([https://huggingface.co/deepset/tinyroberta-squad2)
111
  - [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert)
112
  - [GermanQuAD and GermanDPR datasets and models (aka "gelectra-base-germanquad", "gbert-base-germandpr")](https://deepset.ai/germanquad)
 
 
113
 
114
+ ## Get in touch and join the Haystack community
115
+
116
+ <p>For more info on Haystack, visit our <strong><a href="https://github.com/deepset-ai/haystack">GitHub</a></strong> repo and <strong><a href="https://haystack.deepset.ai">Documentation</a></strong>.
117
+
118
+ We also have a <strong><a class="h-7" href="https://haystack.deepset.ai/community/join"><img alt="slack" class="h-7 inline-block m-0" style="margin: 0" src="https://huggingface.co/spaces/deepset/README/resolve/main/Slack_RGB.png"/>community open to everyone!</a></strong></p>
119
+
120
  [Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Slack](https://haystack.deepset.ai/community/join) | [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) | [Website](https://deepset.ai)
121
 
122
+ By the way: [we're hiring!](http://www.deepset.ai/jobs)