metadata

language:
  - en
license: apache-2.0
library_name: transformers
model-index:
  - name: Rhea-72b-v0.5
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AI2 Reasoning Challenge (25-Shot)
          type: ai2_arc
          config: ARC-Challenge
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: acc_norm
            value: 79.78
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=davidkim205/Rhea-72b-v0.5
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: HellaSwag (10-Shot)
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: acc_norm
            value: 91.15
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=davidkim205/Rhea-72b-v0.5
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU (5-Shot)
          type: cais/mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 77.95
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=davidkim205/Rhea-72b-v0.5
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: TruthfulQA (0-shot)
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: mc2
            value: 74.5
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=davidkim205/Rhea-72b-v0.5
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Winogrande (5-shot)
          type: winogrande
          config: winogrande_xl
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 87.85
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=davidkim205/Rhea-72b-v0.5
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GSM8k (5-shot)
          type: gsm8k
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 76.12
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=davidkim205/Rhea-72b-v0.5
          name: Open LLM Leaderboard

Rhea-72b-v0.5

The Rhea project is a project that conducts research on various learning methods to improve llm model performance. We fine-tuned the existing model using the nox framework. We built a dataset for SFT learning based on the currently open dataset, and created a dataset using SGD (Self-Generated Dataset Creation Method for DPO Learning) for DPO learning.

Our model ranked first on HuggingFace's Open LLM leaderboard.

SGD : A Study on Self-Generated Dataset creation method for DPO Learning

This method proposes a novel method for generating datasets for DPO (Self-supervised Learning) models. We suggest a technique where sentences generated by the model are compared with the actual correct answers from an existing dataset, and sentences where the model's generated results do not match the correct answers are added. This enables the model to autonomously create training data, thereby enhancing the performance of DPO models.

Model Details

Model Developers : davidkim(changyeon kim)
Repository : https://github.com/davidkim205/nox
base mode : abacusai/Smaug-72B-v0.1
sft dataset : datasets_enconv_4m
dpo dataset : datasets_encomp_151k

sft dataset info : datasets_enconv_4m

100k random shuffle datasets

stack-exchange-preferences
SlimOrca
alpaca-gpt4
SHP
HC3
databricks-dolly-15k
orca-dpo-pairs
us-stockname
OpenHermes2.5-dpo-binarized-alpha
distilabel-math-preference-dpo
Neural-DPO
truthy-dpo-v0.1
distilabel-capybara-dpo-7k-binarized
us-sentiment
contextual-dpo-v0.1

1k random shuffle datasets

bigbench
glue_mnli
glue_qqp
xnli
codexglue_code2text_go
trivia_qa
medmcqa
hendrycks_ethics
super_glue_record
glue_qnli
anli_r3
swag
squad_v2
nq_open
drop
glue_sst2
blimp
paws-x
unscramble
anli_r2
babi
math_qa
social_i_qa
piqa
arithmetic
anli_r1
prost
sciq
mc_taco
medqa
super_glue_boolq
hendrycks_math
lambada
toxigen-data
glue_cola
pubmed_qa
logiqa
mutual
headqa
bbh
super_glue_wic
openbookqa
glue_mrpc
web_questions
qasper
super_glue_multirc
story_cloze
super_glue_rte
glue_rte
race
xwinograd
asdiv
xstory_cloze
crows_pairs_multilingual
belebele
glue_wnli
super_glue_wsc
coqa
super_glue_copa
super_glue_cb
winograd_wsc
mgsm
scrolls_contract_nli

If the data set cannot be found, it is internal company data and cannot be made public.

dpo dataset info : datasets_encomp_151k

Randomly selecting data from each category within the training dataset, we constructed a DPO (Direct Preference Optimization) dataset using sentences with logits lower than the mean within the model-generated sentences.

I'm sorry I can't reveal it.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	81.22
AI2 Reasoning Challenge (25-Shot)	79.78
HellaSwag (10-Shot)	91.15
MMLU (5-Shot)	77.95
TruthfulQA (0-shot)	74.50
Winogrande (5-shot)	87.85
GSM8k (5-shot)	76.12