LLmRA-3B-v0.1
A conversational Large Language Model fine-tune.
LLmRA-3B is a proof-of-concept fine-tuned model, derived from GeneZC/MiniMA-3B, with a primary focus on optimizing performance for dialogue, particularly in roleplaying scenarios.
Disclaimer: This model has undergone fine-tuning that involves NSFW data. Although it typically generates SFW outputs for SFW inputs, users are strongly advised to exercise caution during interactions. The model is not intended for use by minors.
Warning: LLmRA-3B is explicitly not suitable for minors. It may produce X-rated content under specific circumstances. Additionally, it's important to note that the fine-tuning process predominantly emphasizes roleplaying data, and as such, the model's proficiency in fields like chemistry and math may not be as extensive. Use with awareness of its strengths and limitations.
As stated in the GeneZC/MiniMA-3B repository, users must comply with the LICENSE of LLaMA2 since MiniMA-3B is derived from LLaMA2.
Prompting
The model has been trained on prompts using three different roles, which are denoted by the following tokens: <|system|>
, <|user|>
, <|model|>
.
It is recommended to use 'SillyTavern' for interference.
<|system|>I'm here to provide information and assistance on a wide range of topics.
<|model|>Greetings! Welcome to our AI-powered assistant. How can I assist you today?
<|user|>Salutations! Could you tell me, what is one plus one?
<|model|>
You may also use:
<|system|>What is one plus one?<|model|>
Loading The Model
To use the model and interact with it, use the Python code below:
from transformers import (AutoModelForCausalLM,
AutoTokenizer,
pipeline,
)
model = AutoModelForCausalLM.from_pretrained('L-R/LLmRA-3B-v0.1')
tokenizer = AutoTokenizer.from_pretrained('L-R/LLmRA-3B-v0.1')
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=100)
input_question = 'Your question here.'
question_formatted = f'<|system|>{input_question}<|model|>'
result = pipe(question_formatted)
print(f"[model]: {result[0]['generated_text'][len(question_formatted):]}")
Limitations and biases
This model is designed for the purpose of creating fictional content for entertainment. Any other form of utilization is not within its intended scope.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 39.25 |
AI2 Reasoning Challenge (25-Shot) | 39.42 |
HellaSwag (10-Shot) | 59.79 |
MMLU (5-Shot) | 25.16 |
TruthfulQA (0-shot) | 50.62 |
Winogrande (5-shot) | 59.43 |
GSM8k (5-shot) | 1.06 |
- Downloads last month
- 669
Model tree for L-R/LLmRA-3B-v0.1
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard39.420
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard59.790
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard25.160
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard50.620
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard59.430
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard1.060