Update README.md
Browse filesupdate evaluation section
README.md
CHANGED
@@ -39,6 +39,20 @@ Jargon is available in several versions with different context sizes and types o
|
|
39 |
|
40 |
The Jargon models were evaluated on an range of specialized downstream tasks.
|
41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
For more info please check out the [paper](https://hal.science/hal-04535557/file/FB2_domaines_specialises_LREC_COLING24.pdf), accepted for publication at [LREC-COLING 2024](https://lrec-coling-2024.org/list-of-accepted-papers/).
|
43 |
|
44 |
|
|
|
39 |
|
40 |
The Jargon models were evaluated on an range of specialized downstream tasks.
|
41 |
|
42 |
+
### Legal Domain Benchmark
|
43 |
+
|
44 |
+
Results averaged across five funs with varying random seeds.
|
45 |
+
|
46 |
+
| **Task Name** | ECtHR-FR | OACS | SJP |
|
47 |
+
|-------------------------|:-----------------------:|:-----------------------:|:-----------------------:|
|
48 |
+
| **Task Type** | Document Classification | Document Classification | Document Classification |
|
49 |
+
| **Metric** | Macro-F1 | Macro-F1 | Macro-F1 |
|
50 |
+
| jargon-general-base | 42.9 | 50.8 | 55.1 |
|
51 |
+
| jargon-multidomain-base | 44.5 | 55.6 | 58.1 |
|
52 |
+
| jargon-general-legal | 43.1 | 49.9 | 54.5 |
|
53 |
+
| jargon-legal | 44.6 | 51.6 | 56.7 |
|
54 |
+
| jargon-legal-4096 | 45.9 | 54.1 | 68.2 |
|
55 |
+
|
56 |
For more info please check out the [paper](https://hal.science/hal-04535557/file/FB2_domaines_specialises_LREC_COLING24.pdf), accepted for publication at [LREC-COLING 2024](https://lrec-coling-2024.org/list-of-accepted-papers/).
|
57 |
|
58 |
|