This is Xander Boyce's OmegLLaMA LoRA merged with OpenLLama 3B.
Prompt format:
Interests: {interests}
Conversation:
You: {prompt}
Stranger:
For multiple interests, seperate them with space. Repeat You and Stranger for multi-turn conversations, which means Interests and Conversation are technically part of the system prompt.
GGUF quantizations available here.
This model is very good at NSFW ERP and sexting(For a 3B model). I recommend using this with Faraday.dev if you want ERP or sexting.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 38.28 |
| ARC (25-shot) | 40.36 |
| HellaSwag (10-shot) | 66.13 |
| MMLU (5-shot) | 28.0 |
| TruthfulQA (0-shot) | 33.31 |
| Winogrande (5-shot) | 61.64 |
| GSM8K (5-shot) | 0.23 |
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 38.28 |
| AI2 Reasoning Challenge (25-Shot) | 40.36 |
| HellaSwag (10-Shot) | 66.13 |
| MMLU (5-Shot) | 28.00 |
| TruthfulQA (0-shot) | 33.31 |
| Winogrande (5-shot) | 61.64 |
| GSM8k (5-shot) | 0.23 |
- Downloads last month
- 719
Inference API (serverless) is not available, repository is disabled.
Dataset used to train acrastt/OmegLLaMA-3B
Spaces using acrastt/OmegLLaMA-3B 20
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard40.360
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard66.130
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard28.000
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard33.310
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard61.640
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard0.230