deit-base-distilled-patch16-224-75-fold3

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.2177
• Accuracy: 0.9535

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.5697	0.6744
No log	2.0	4	0.6348	0.6977
No log	3.0	6	0.8030	0.6977
No log	4.0	8	0.7092	0.6977
0.5313	5.0	10	0.4644	0.8140
0.5313	6.0	12	0.4211	0.7907
0.5313	7.0	14	0.4913	0.8140
0.5313	8.0	16	0.4788	0.8372
0.5313	9.0	18	0.3717	0.7907
0.3672	10.0	20	0.3763	0.8140
0.3672	11.0	22	0.5082	0.8372
0.3672	12.0	24	0.3661	0.8140
0.3672	13.0	26	0.3693	0.8140
0.3672	14.0	28	0.3808	0.8605
0.2837	15.0	30	0.3492	0.8372
0.2837	16.0	32	0.3446	0.7907
0.2837	17.0	34	0.3909	0.8605
0.2837	18.0	36	0.4379	0.8372
0.2837	19.0	38	0.3905	0.8140
0.2268	20.0	40	0.3453	0.8140
0.2268	21.0	42	0.4145	0.8372
0.2268	22.0	44	0.3370	0.8372
0.2268	23.0	46	0.3502	0.8372
0.2268	24.0	48	0.3295	0.8372
0.1735	25.0	50	0.3118	0.8837
0.1735	26.0	52	0.3050	0.8837
0.1735	27.0	54	0.6940	0.8140
0.1735	28.0	56	0.5913	0.8372
0.1735	29.0	58	0.3190	0.8837
0.1221	30.0	60	0.4141	0.8372
0.1221	31.0	62	0.4572	0.8372
0.1221	32.0	64	0.3048	0.8837
0.1221	33.0	66	0.3139	0.9070
0.1221	34.0	68	0.3090	0.8837
0.1158	35.0	70	0.3393	0.8837
0.1158	36.0	72	0.3035	0.8605
0.1158	37.0	74	0.4730	0.8140
0.1158	38.0	76	0.3788	0.8605
0.1158	39.0	78	0.2904	0.8837
0.1075	40.0	80	0.2750	0.8837
0.1075	41.0	82	0.3328	0.8837
0.1075	42.0	84	0.2648	0.9070
0.1075	43.0	86	0.2517	0.8837
0.1075	44.0	88	0.4402	0.8837
0.0925	45.0	90	0.4076	0.8837
0.0925	46.0	92	0.2390	0.9070
0.0925	47.0	94	0.2176	0.9070
0.0925	48.0	96	0.2580	0.9302
0.0925	49.0	98	0.2049	0.9070
0.1085	50.0	100	0.2244	0.8837
0.1085	51.0	102	0.2377	0.9070
0.1085	52.0	104	0.4591	0.8372
0.1085	53.0	106	0.5054	0.8372
0.1085	54.0	108	0.2994	0.9302
0.0876	55.0	110	0.2387	0.9070
0.0876	56.0	112	0.3078	0.9070
0.0876	57.0	114	0.4470	0.8372
0.0876	58.0	116	0.3457	0.9070
0.0876	59.0	118	0.2655	0.9070
0.0823	60.0	120	0.2150	0.9070
0.0823	61.0	122	0.2116	0.9070
0.0823	62.0	124	0.2305	0.9302
0.0823	63.0	126	0.2070	0.9302
0.0823	64.0	128	0.1808	0.9070
0.0791	65.0	130	0.1669	0.9070
0.0791	66.0	132	0.1721	0.9070
0.0791	67.0	134	0.2194	0.9302
0.0791	68.0	136	0.3454	0.8837
0.0791	69.0	138	0.5415	0.8372
0.0607	70.0	140	0.4457	0.8605
0.0607	71.0	142	0.2411	0.8837
0.0607	72.0	144	0.2057	0.9070
0.0607	73.0	146	0.2200	0.9070
0.0607	74.0	148	0.2677	0.8837
0.0715	75.0	150	0.2950	0.8837
0.0715	76.0	152	0.2874	0.8837
0.0715	77.0	154	0.2236	0.9070
0.0715	78.0	156	0.2052	0.9302
0.0715	79.0	158	0.2177	0.9535
0.0644	80.0	160	0.2178	0.9535
0.0644	81.0	162	0.2126	0.9302
0.0644	82.0	164	0.2127	0.9302
0.0644	83.0	166	0.2216	0.9070
0.0644	84.0	168	0.2420	0.9070
0.0622	85.0	170	0.2305	0.9070
0.0622	86.0	172	0.2247	0.9070
0.0622	87.0	174	0.2492	0.9070
0.0622	88.0	176	0.3292	0.8837
0.0622	89.0	178	0.3876	0.8837
0.0564	90.0	180	0.3886	0.8837
0.0564	91.0	182	0.3707	0.8837
0.0564	92.0	184	0.3377	0.8837
0.0564	93.0	186	0.3186	0.8837
0.0564	94.0	188	0.3038	0.8837
0.0578	95.0	190	0.2818	0.8605
0.0578	96.0	192	0.2756	0.8837
0.0578	97.0	194	0.2694	0.8837
0.0578	98.0	196	0.2698	0.8837
0.0578	99.0	198	0.2732	0.8837
0.0424	100.0	200	0.2739	0.8837

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1