deit-base-distilled-patch16-224-55-fold1

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4563
• Accuracy: 0.8354

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.8571	3	0.7880	0.4304
No log	2.0	7	0.8337	0.5443
0.776	2.8571	10	0.7047	0.5443
0.776	4.0	14	0.6729	0.6582
0.776	4.8571	17	0.6784	0.5696
0.6852	6.0	21	0.6251	0.6329
0.6852	6.8571	24	0.6246	0.6329
0.6852	8.0	28	0.5865	0.6835
0.6223	8.8571	31	0.5573	0.6582
0.6223	10.0	35	0.5841	0.6582
0.6223	10.8571	38	0.5791	0.7089
0.5573	12.0	42	0.5283	0.7215
0.5573	12.8571	45	0.5126	0.7595
0.5573	14.0	49	0.5643	0.7089
0.4772	14.8571	52	0.6736	0.6582
0.4772	16.0	56	0.5707	0.7595
0.4772	16.8571	59	0.5199	0.7215
0.4656	18.0	63	0.5285	0.7595
0.4656	18.8571	66	0.4535	0.7848
0.4147	20.0	70	0.4557	0.7975
0.4147	20.8571	73	0.4483	0.7975
0.4147	22.0	77	0.5025	0.7342
0.342	22.8571	80	0.4716	0.7215
0.342	24.0	84	0.5311	0.7342
0.342	24.8571	87	0.4560	0.7848
0.2993	26.0	91	0.5119	0.7848
0.2993	26.8571	94	0.5321	0.7722
0.2993	28.0	98	0.4937	0.7975
0.2506	28.8571	101	0.4563	0.8354
0.2506	30.0	105	0.5234	0.7975
0.2506	30.8571	108	0.5359	0.7848
0.2201	32.0	112	0.5145	0.7975
0.2201	32.8571	115	0.5343	0.8101
0.2201	34.0	119	0.4689	0.7975
0.2098	34.8571	122	0.6465	0.8101
0.2098	36.0	126	0.5003	0.7848
0.2098	36.8571	129	0.6113	0.7468
0.1808	38.0	133	0.8216	0.7595
0.1808	38.8571	136	0.5603	0.7975
0.1892	40.0	140	0.6136	0.7848
0.1892	40.8571	143	0.6074	0.7722
0.1892	42.0	147	0.6503	0.7848
0.154	42.8571	150	0.7923	0.7595
0.154	44.0	154	0.7791	0.7722
0.154	44.8571	157	0.7948	0.7722
0.1613	46.0	161	0.7270	0.7722
0.1613	46.8571	164	0.7283	0.7848
0.1613	48.0	168	0.7057	0.7975
0.141	48.8571	171	0.6692	0.7848
0.141	50.0	175	0.6390	0.7975
0.141	50.8571	178	0.6543	0.7975
0.1434	52.0	182	0.7736	0.7468
0.1434	52.8571	185	0.6426	0.7722
0.1434	54.0	189	0.6891	0.7848
0.1583	54.8571	192	0.7521	0.7848
0.1583	56.0	196	0.6495	0.8101
0.1583	56.8571	199	0.7049	0.7975
0.1418	58.0	203	0.7534	0.7848
0.1418	58.8571	206	0.6892	0.7975
0.1488	60.0	210	0.7528	0.7722
0.1488	60.8571	213	0.6920	0.7975
0.1488	62.0	217	0.6767	0.7722
0.1481	62.8571	220	0.7510	0.7848
0.1481	64.0	224	0.6075	0.7848
0.1481	64.8571	227	0.5858	0.7975
0.1014	66.0	231	0.6668	0.7848
0.1014	66.8571	234	0.6127	0.7975
0.1014	68.0	238	0.6295	0.7975
0.1147	68.8571	241	0.6723	0.8101
0.1147	70.0	245	0.7167	0.7848
0.1147	70.8571	248	0.6914	0.7975
0.1289	72.0	252	0.6676	0.7975
0.1289	72.8571	255	0.6874	0.8101
0.1289	74.0	259	0.7486	0.8101
0.1084	74.8571	262	0.7193	0.8101
0.1084	76.0	266	0.7054	0.8354
0.1084	76.8571	269	0.7052	0.8228
0.11	78.0	273	0.6885	0.7975
0.11	78.8571	276	0.7163	0.8101
0.1144	80.0	280	0.6902	0.7975
0.1144	80.8571	283	0.6886	0.7975
0.1144	82.0	287	0.7062	0.8228
0.1026	82.8571	290	0.7196	0.8101
0.1026	84.0	294	0.7332	0.7975
0.1026	84.8571	297	0.7225	0.7975
0.1143	85.7143	300	0.7162	0.7975

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1