deit-base-distilled-patch16-224-65-fold3

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4993
• Accuracy: 0.8732

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9231	3	0.7403	0.5493
No log	1.8462	6	0.7199	0.5211
No log	2.7692	9	0.7111	0.5634
0.7693	4.0	13	0.7015	0.5352
0.7693	4.9231	16	0.6471	0.6197
0.7693	5.8462	19	0.6691	0.6056
0.6542	6.7692	22	0.6188	0.6197
0.6542	8.0	26	0.6967	0.5775
0.6542	8.9231	29	0.5732	0.7324
0.5935	9.8462	32	0.5184	0.7042
0.5935	10.7692	35	0.6031	0.7183
0.5935	12.0	39	0.6671	0.6479
0.549	12.9231	42	0.5281	0.7183
0.549	13.8462	45	0.5792	0.7183
0.549	14.7692	48	0.5389	0.7465
0.4778	16.0	52	0.6010	0.7042
0.4778	16.9231	55	0.5245	0.7606
0.4778	17.8462	58	0.5491	0.7183
0.4039	18.7692	61	0.5590	0.7465
0.4039	20.0	65	0.4886	0.7324
0.4039	20.9231	68	0.5050	0.7324
0.3409	21.8462	71	0.4912	0.7465
0.3409	22.7692	74	0.4929	0.7746
0.3409	24.0	78	0.5365	0.7746
0.3202	24.9231	81	0.4685	0.8028
0.3202	25.8462	84	0.4404	0.8169
0.3202	26.7692	87	0.4639	0.8028
0.2466	28.0	91	0.5491	0.7606
0.2466	28.9231	94	0.5170	0.7606
0.2466	29.8462	97	0.4444	0.8028
0.2433	30.7692	100	0.4517	0.8310
0.2433	32.0	104	0.7797	0.7606
0.2433	32.9231	107	0.4321	0.8169
0.2535	33.8462	110	0.5956	0.7746
0.2535	34.7692	113	0.4695	0.7887
0.2535	36.0	117	0.8106	0.6901
0.2215	36.9231	120	0.7119	0.7465
0.2215	37.8462	123	0.4752	0.8028
0.2215	38.7692	126	0.4784	0.8169
0.2143	40.0	130	0.4773	0.8028
0.2143	40.9231	133	0.5581	0.8169
0.2143	41.8462	136	0.6098	0.8028
0.2143	42.7692	139	0.5193	0.8169
0.1726	44.0	143	0.4306	0.8451
0.1726	44.9231	146	0.4234	0.8592
0.1726	45.8462	149	0.5264	0.8169
0.1684	46.7692	152	0.7303	0.8028
0.1684	48.0	156	0.5079	0.8169
0.1684	48.9231	159	0.5392	0.8169
0.1604	49.8462	162	0.3951	0.8169
0.1604	50.7692	165	0.4311	0.8028
0.1604	52.0	169	0.4082	0.8028
0.1457	52.9231	172	0.4173	0.7887
0.1457	53.8462	175	0.4311	0.8310
0.1457	54.7692	178	0.4213	0.8028
0.1549	56.0	182	0.4713	0.8451
0.1549	56.9231	185	0.7493	0.8028
0.1549	57.8462	188	0.5161	0.8451
0.1391	58.7692	191	0.4685	0.8169
0.1391	60.0	195	0.6968	0.8028
0.1391	60.9231	198	0.5837	0.8310
0.1272	61.8462	201	0.5863	0.8169
0.1272	62.7692	204	0.5460	0.8310
0.1272	64.0	208	0.6198	0.8310
0.1341	64.9231	211	0.5584	0.8592
0.1341	65.8462	214	0.6429	0.8451
0.1341	66.7692	217	0.8592	0.8028
0.1144	68.0	221	0.8472	0.8028
0.1144	68.9231	224	0.8360	0.8169
0.1144	69.8462	227	0.6697	0.8169
0.1321	70.7692	230	0.6625	0.8028
0.1321	72.0	234	0.7228	0.8310
0.1321	72.9231	237	0.6793	0.8310
0.1206	73.8462	240	0.5571	0.8592
0.1206	74.7692	243	0.5106	0.8451
0.1206	76.0	247	0.6686	0.8310
0.131	76.9231	250	0.7132	0.8310
0.131	77.8462	253	0.5945	0.8451
0.131	78.7692	256	0.5516	0.7746
0.1009	80.0	260	0.5474	0.7606
0.1009	80.9231	263	0.5219	0.7887
0.1009	81.8462	266	0.5375	0.8451
0.1009	82.7692	269	0.5133	0.8451
0.1084	84.0	273	0.4911	0.8451
0.1084	84.9231	276	0.4993	0.8732
0.1084	85.8462	279	0.5418	0.8592
0.0851	86.7692	282	0.6010	0.8451
0.0851	88.0	286	0.6305	0.8451
0.0851	88.9231	289	0.6016	0.8451
0.1071	89.8462	292	0.5773	0.8592
0.1071	90.7692	295	0.5610	0.8732
0.1071	92.0	299	0.5522	0.8732
0.1139	92.3077	300	0.5514	0.8732

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1