gpt2_custom_c_q_and_a

This model is a fine-tuned version of lighteternal/gpt2-finetuned-greek on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0098

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss
2.5542	4.76	100	0.7706
0.4813	9.52	200	0.1839
0.1838	14.29	300	0.0892
0.109	19.05	400	0.0532
0.0767	23.81	500	0.0385
0.0579	28.57	600	0.0290
0.0447	33.33	700	0.0213
0.0365	38.1	800	0.0178
0.0319	42.86	900	0.0157
0.0277	47.62	1000	0.0140
0.0252	52.38	1100	0.0125
0.0229	57.14	1200	0.0113
0.0213	61.9	1300	0.0107
0.0199	66.67	1400	0.0104
0.0181	71.43	1500	0.0099
0.0184	76.19	1600	0.0098

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu118
• Tokenizers 0.13.3