a-mannion commited on
Commit
0659b9f
1 Parent(s): 60d1188

Update README.md

Browse files

update evaluation section

Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -39,6 +39,20 @@ Jargon is available in several versions with different context sizes and types o
39
 
40
  The Jargon models were evaluated on an range of specialized downstream tasks.
41
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  For more info please check out the [paper](https://hal.science/hal-04535557/file/FB2_domaines_specialises_LREC_COLING24.pdf), accepted for publication at [LREC-COLING 2024](https://lrec-coling-2024.org/list-of-accepted-papers/).
43
 
44
 
 
39
 
40
  The Jargon models were evaluated on an range of specialized downstream tasks.
41
 
42
+ ### Legal Domain Benchmark
43
+
44
+ Results averaged across five funs with varying random seeds.
45
+
46
+ | **Task Name** | ECtHR-FR | OACS | SJP |
47
+ |-------------------------|:-----------------------:|:-----------------------:|:-----------------------:|
48
+ | **Task Type** | Document Classification | Document Classification | Document Classification |
49
+ | **Metric** | Macro-F1 | Macro-F1 | Macro-F1 |
50
+ | jargon-general-base | 42.9 | 50.8 | 55.1 |
51
+ | jargon-multidomain-base | 44.5 | 55.6 | 58.1 |
52
+ | jargon-general-legal | 43.1 | 49.9 | 54.5 |
53
+ | jargon-legal | 44.6 | 51.6 | 56.7 |
54
+ | jargon-legal-4096 | 45.9 | 54.1 | 68.2 |
55
+
56
  For more info please check out the [paper](https://hal.science/hal-04535557/file/FB2_domaines_specialises_LREC_COLING24.pdf), accepted for publication at [LREC-COLING 2024](https://lrec-coling-2024.org/list-of-accepted-papers/).
57
 
58