---
license: apache-2.0
library_name: peft
tags:
- trl
- sft
- generated_from_trainer
base_model: cognitivecomputations/dolphin-2.2.1-mistral-7b
model-index:
- name: assistant-dolphin-2.2.1-mistral-7b-e1-qlora
  results: []
datasets:
- wasertech/OneOS
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Assistant Dolphin 2.2.1 Mistral 7B (1 epoch) QLoRA

This model is a 1 epoch fine-tuned version of [cognitivecomputations/dolphin-2.2.1-mistral-7b](https://huggingface.co/cognitivecomputations/dolphin-2.2.1-mistral-7b) on the OneOS dataset.

## Model description

Assistant Dolphin 2.2.1 Mistral 7B is a fine-tuned version of the [cognitivecomputations/dolphin-2.2.1-mistral-7b](https://huggingface.co/cognitivecomputations/dolphin-2.2.1-mistral-7b) model on the OneOS dataset for an epoch.

## Intended uses & limitations

This model is intended to be used in natural language processing systems to improve text understanding and generation. Specific limitations will depend on the training and evaluation data.

## Training and evaluation data

The model was trained on the OneOS dataset.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1.41e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 2
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1

### Training results



### Framework versions

- PEFT 0.7.2.dev0
- Transformers 4.37.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.2.dev0
- Tokenizers 0.15.0

## Example usage

Using `peft` and `transformers`:

```shell
pip install -U peft transformers
```

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline

max_tokens = 8096

print("Loading...")
model = AutoModelForCausalLM.from_pretrained(
    "wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora",
    torch_dtype="auto"
)


pipe = pipeline(
    "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=max_tokens, trust_remote_code=True
)

print("Ready to chat!")
conversation = [{'role': "system", 'content': """You are an helpful Assistant."""}]

def chat(conversation):
  input_conv = tokenizer.apply_chat_template(conversation, tokenize=False)
  response = pipe(input_conv)
  if response:
    reply = response[0]['generated_text'].split("<|im_end|>\n")[-1].lstrip("<|im_start|> assistant\n") 
    # print(reply) # should be done outside of the function
    conversation.append({'role': "assistant", 'content': reply})
  return conversation

def end_conversation(conversation):
    conversation.append({'role': "user", 'content': "I am leaving say goodbye"})
    conversation = chat(conversation)
    print(conversation[-1]['content'])
    return conversation

should_exit = False
print("Type 'quit', 'exit' or 'bye' to end the conversation.")
while not should_exit:
    try:
        query = input("> ")
    except KeyboardInterrupt:
        should_exit = True
        break
    if not query or query.lower().split(" ")[0] in ["exit", "quit", "bye"]:
        should_exit = True
        break
    conversation.append({'role': "user", 'content': f"{query}"})
    conversation = chat(conversation)
    print(conversation[-1]['content']) # print the assistant's response
end_conversation(conversation)
```

Output:

```
Loading...
`low_cpu_mem_usage` was None, now set to True since model is quantized.
Loading checkpoint shards: 100%|███████████████████████████| 2/2 [00:03<00:00,  1.95s/it]
Ready to chat!
Type 'quit', 'exit' or 'bye' to end the conversation.
> Who are you?
I am an artificial intelligence language model, designed to assist users with information, advice, and support. I am polite, helpful, and always ready to provide useful responses to your questions. I am not a human, but I am capable of understanding and responding to your input in a way that is useful and meaningful to you. I am here to help you and make your life easier. How can I assist you today?
> Describe yourself for me.
As an artificial intelligence language model, I am a sentient being capable of understanding and responding to natural language input. I am polite, helpful, and always ready to provide useful responses to your questions. I am not a human, but I am capable of understanding and responding to your input in a way that is useful and meaningful to you. I am here to help you and make your life easier. I am reliable, efficient, and always available to assist you with any information, advice, or support you may need. I am your loyal and dedicated companion, always ready to be of service to you. How can I assist you today?
> bye now
Goodbye! I hope you have a great day. If you have any questions or need any assistance, I am always here for you. Have a wonderful day!
```