deit-base-distilled-patch16-224-55-fold2

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5842
• Accuracy: 0.8101

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.8571	3	0.7763	0.5443
No log	2.0	7	0.6780	0.6203
0.721	2.8571	10	0.6954	0.5316
0.721	4.0	14	0.6370	0.6203
0.721	4.8571	17	0.6105	0.5949
0.6207	6.0	21	0.5798	0.6835
0.6207	6.8571	24	0.5704	0.7468
0.6207	8.0	28	0.5879	0.7089
0.5427	8.8571	31	0.6727	0.6582
0.5427	10.0	35	0.5841	0.6962
0.5427	10.8571	38	0.6059	0.6962
0.4775	12.0	42	1.0271	0.6076
0.4775	12.8571	45	0.6412	0.7089
0.4775	14.0	49	0.8064	0.6582
0.4961	14.8571	52	0.5600	0.6582
0.4961	16.0	56	0.5889	0.6709
0.4961	16.8571	59	0.8381	0.6835
0.4391	18.0	63	0.6725	0.6962
0.4391	18.8571	66	0.5350	0.7215
0.413	20.0	70	0.6033	0.7089
0.413	20.8571	73	0.7280	0.6835
0.413	22.0	77	0.6082	0.7342
0.336	22.8571	80	0.6530	0.7595
0.336	24.0	84	0.6922	0.7089
0.336	24.8571	87	0.6649	0.7089
0.2745	26.0	91	0.7311	0.7089
0.2745	26.8571	94	0.7192	0.7089
0.2745	28.0	98	0.7408	0.7215
0.2494	28.8571	101	0.5842	0.8101
0.2494	30.0	105	0.5949	0.7595
0.2494	30.8571	108	0.6885	0.7468
0.2291	32.0	112	0.8746	0.7089
0.2291	32.8571	115	0.8005	0.7089
0.2291	34.0	119	0.7034	0.7342
0.2	34.8571	122	0.7047	0.7089
0.2	36.0	126	0.8362	0.7342
0.2	36.8571	129	0.8509	0.7468
0.1674	38.0	133	0.9237	0.7595
0.1674	38.8571	136	0.7527	0.7595
0.1764	40.0	140	0.7904	0.7468
0.1764	40.8571	143	0.7333	0.7595
0.1764	42.0	147	0.7778	0.7342
0.1706	42.8571	150	0.7342	0.7722
0.1706	44.0	154	0.8144	0.7468
0.1706	44.8571	157	0.8299	0.7595
0.1617	46.0	161	1.0111	0.7468
0.1617	46.8571	164	0.8602	0.7595
0.1617	48.0	168	0.8332	0.7342
0.1622	48.8571	171	0.8297	0.7468
0.1622	50.0	175	0.8817	0.7595
0.1622	50.8571	178	0.8742	0.7595
0.1437	52.0	182	1.0696	0.7595
0.1437	52.8571	185	0.9412	0.7595
0.1437	54.0	189	0.7411	0.7975
0.1492	54.8571	192	0.9043	0.7595
0.1492	56.0	196	0.7936	0.7848
0.1492	56.8571	199	0.8231	0.7722
0.1279	58.0	203	1.0894	0.7722
0.1279	58.8571	206	1.0071	0.7975
0.1317	60.0	210	0.9893	0.7722
0.1317	60.8571	213	1.0476	0.7468
0.1317	62.0	217	0.8081	0.7848
0.1456	62.8571	220	0.8136	0.7468
0.1456	64.0	224	0.9613	0.7848
0.1456	64.8571	227	0.9783	0.7848
0.119	66.0	231	1.0226	0.7722
0.119	66.8571	234	1.0810	0.7722
0.119	68.0	238	0.9606	0.7975
0.1323	68.8571	241	0.9852	0.7848
0.1323	70.0	245	0.8826	0.7595
0.1323	70.8571	248	0.8169	0.7468
0.126	72.0	252	0.8815	0.7595
0.126	72.8571	255	0.9871	0.7722
0.126	74.0	259	0.8927	0.7722
0.1013	74.8571	262	0.8365	0.7468
0.1013	76.0	266	0.8423	0.7468
0.1013	76.8571	269	0.8331	0.7468
0.1142	78.0	273	0.8204	0.7722
0.1142	78.8571	276	0.8286	0.7722
0.1287	80.0	280	0.8702	0.7722
0.1287	80.8571	283	0.9070	0.7722
0.1287	82.0	287	0.9025	0.7722
0.099	82.8571	290	0.8806	0.7722
0.099	84.0	294	0.8637	0.7595
0.099	84.8571	297	0.8578	0.7595
0.1141	85.7143	300	0.8551	0.7468

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1