yinsong1986 commited on
Commit
2348608
1 Parent(s): f4c8132

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -85,6 +85,20 @@ there were some limitations on its performance on longer context. Motivated by i
85
  - **Contact:** [GitHub issues](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/issues)
86
  - **Inference Code** [Github Repo](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/)
87
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
  ## How to Use MistralLite from Python Code (HuggingFace transformers) ##
89
 
90
  **Important** - For an end-to-end example Jupyter notebook, please refer to [this link](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/huggingface-transformers/example_usage.ipynb).
 
85
  - **Contact:** [GitHub issues](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/issues)
86
  - **Inference Code** [Github Repo](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/)
87
 
88
+ ## MistralLite LM-Eval Results
89
+
90
+ ### Methodology
91
+
92
+ - Please see https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
93
+ - revision=4ececff
94
+ - Note: we used --model hf-causal-experimental instead of --model hf-causal
95
+
96
+ ### Results
97
+
98
+ |Average|hellaswag| arc_challenge|truthful_qa (mc2)| MMLU (acc)|
99
+ |----------|-------------:|------------:|-----------:|-----------:|
100
+ | 0.57221 | 0.81617 | 0.58874 | 0.38275 | 0.5012 |
101
+
102
  ## How to Use MistralLite from Python Code (HuggingFace transformers) ##
103
 
104
  **Important** - For an end-to-end example Jupyter notebook, please refer to [this link](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/huggingface-transformers/example_usage.ipynb).