Hugging Face's logo

M4-ai
/
Hercules-5.0-Qwen2-1.5B

Text Generation
Transformers
Safetensors
English
qwen2
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
4
Hercules-5.0-Qwen2-1.5B / README.md
Locutusque's picture
Locutusque
Update README.md
81137f5 verified
preview code
|
raw
history blame
1.59 kB
metadata
license: apache-2.0
datasets:
  - Locutusque/hercules-v5.0
language:
  - en

Orca-2.0-Tau-1.8B

We fine-tuned qwen2-1.5B on a high quality mix for general-purpose assistants. A DPO version of this will be released soon. We use the ChatML prompt format.

Model Details

Model Description

This model has capabilities in math, coding, writing, and more. We fine-tuned it using a high quality mix for general-purpose assistants.

  • Developed by: M4-ai
  • Language(s) (NLP): English and maybe Chinese
  • License: apache-2.0
  • Finetuned from model: qwen2-1.5B

Uses

General purpose assistant, question answering, chain-of-thought, etc..

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Training Details

Training Data

  • Locutusque/hercules-v5.0

Evaluations

coming soon

Training Hyperparameters

  • Training regime: bf16 non-mixed precision

Technical Specifications

Hardware

We used 8 Kaggle TPUs, and we trained at a global batch size of 256 and sequence length of 1536.