Add evaluations

README.md

@@ -16,18 +16,18 @@ interpretability research. It contains two sets of eight models of sizes
 models: one trained on the Pile, and one trained on the Pile after the dataset 
 has been globally deduplicated. All 8 model sizes are trained on the exact 
 same data, in the exact same order. All Pythia models are available 
-[on Hugging Face](https://huggingface.co/EleutherAI).
+[on Hugging Face](https://huggingface.co/models?other=pythia).
 
 The Pythia model suite was deliberately designed to promote scientific 
 research on large language models, especially interpretability research. 
 Despite not centering downstream performance as a design goal, we find the 
-models match or exceed the performance of similar and same-sized models, 
-such as those in the OPT and GPT-Neo suites.
+models <a href="#evaluations">match or exceed</a> the performance of 
+similar and same-sized models, such as those in the OPT and GPT-Neo suites.
 
-Please note that all models in the *Pythia* suite were re-named in January 
+Please note that all models in the *Pythia* suite were renamed in January 
 2023. For clarity, a <a href="#naming-convention-and-parameter-count">table 
 comparing the old and new names</a> is provided in this model card, together 
-with exact model parameter counts.
+with exact parameter counts.
 
 ## Pythia-410M
 
@@ -79,7 +79,7 @@ exactly to the model checkpoint on the `main` branch of each model.
 You may also further fine-tune and adapt Pythia-410M for deployment, 
 as long as your use is in accordance with the Apache 2.0 license. Pythia 
 models work with the Hugging Face [Transformers 
-Library](https://huggingface.co/docs/transformers/index).If you decide to use 
+Library](https://huggingface.co/docs/transformers/index). If you decide to use 
 pre-trained Pythia-410M as a basis for your fine-tuned model, please 
 conduct your own risk and bias assessment. 
 
@@ -143,8 +143,7 @@ tokenizer.decode(tokens[0])
 ```
 
 Revision/branch `step143000` corresponds exactly to the model checkpoint on 
-the `main` branch of each model.
-
+the `main` branch of each model.<br>
 For more information on how to use all Pythia models, see [documentation on 
 GitHub](https://github.com/EleutherAI/pythia).
 
@@ -163,15 +162,11 @@ methodology, and a discussion of ethical implications. Consult [the
 datasheet](https://arxiv.org/abs/2201.07311) for more detailed documentation 
 about the Pile and its component datasets. The Pile can be downloaded from 
 the [official website](https://pile.eleuther.ai/), or from a [community 
-mirror](https://the-eye.eu/public/AI/pile/).
-
+mirror](https://the-eye.eu/public/AI/pile/).<br>
 The Pile was **not** deduplicated before being used to train Pythia-410M.
 
 #### Training procedure
 
-Pythia uses the same tokenizer as [GPT-NeoX-
-20B](https://huggingface.co/EleutherAI/gpt-neox-20b). 
-
 All models were trained on the exact same data, in the exact same order. Each 
 model saw 299,892,736,000 tokens during training, and 143 checkpoints for each 
 model are saved every 2,097,152,000 tokens, spaced evenly throughout training. 
@@ -185,25 +180,50 @@ checkpoints every 500 steps. The checkpoints on Hugging Face are renamed for
 consistency with all 2M batch models, so `step1000` is the first checkpoint 
 for `pythia-1.4b` that was saved (corresponding to step 500 in training), and 
 `step1000` is likewise the first `pythia-6.9b` checkpoint that was saved 
-(corresponding to 1000 “actual” steps).
-
+(corresponding to 1000 “actual” steps).<br>
 See [GitHub](https://github.com/EleutherAI/pythia) for more details on training
  procedure, including [how to reproduce 
- it](https://github.com/EleutherAI/pythia/blob/main/README.md#reproducing-training).
+ it](https://github.com/EleutherAI/pythia/blob/main/README.md#reproducing-training).<br>
+Pythia uses the same tokenizer as [GPT-NeoX-
+20B](https://huggingface.co/EleutherAI/gpt-neox-20b).
 
 ### Evaluations
 
 All 16 *Pythia* models were evaluated using the [LM Evaluation 
 Harness](https://github.com/EleutherAI/lm-evaluation-harness). You can access 
 the results by model and step at `results/json/*` in the [GitHub 
-repository](https://github.com/EleutherAI/pythia/tree/main/results/json).
-
-February 2023 note: select evaluations and comparison with OPT and BLOOM 
-models will be added here at a later date.
+repository](https://github.com/EleutherAI/pythia/tree/main/results/json).<br>
+Expand the sections below to see plots of evaluation results for all 
+Pythia and Pythia-deduped models compared with OPT and BLOOM.
+
+<details>
+  <summary>LAMBADA – OpenAI</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/lambada_openai.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>Physical Interaction: Question Answering (PIQA)</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/piqa.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>WinoGrande</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/winogrande.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>AI2 Reasoning Challenge—Challenge Set</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/arc_challenge.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>SciQ</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/sciq.png" style="width:auto"/>
+</details>
 
 ### Naming convention and parameter count
 
-*Pythia* models were re-named in January 2023. It is possible that the old 
+*Pythia* models were renamed in January 2023. It is possible that the old 
 naming convention still persists in some documentation by accident. The 
 current naming convention (70M, 160M, etc.) is based on total parameter count.