deit-base-distilled-patch16-224-85-fold3

This model is a fine-tuned version of facebook/deit-base-distilled-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3477
• Accuracy: 0.9091

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.7892	0.3409
No log	2.0	4	0.5546	0.7727
No log	3.0	6	0.6493	0.7727
No log	4.0	8	0.6648	0.7727
0.6939	5.0	10	0.5187	0.7727
0.6939	6.0	12	0.4903	0.8182
0.6939	7.0	14	0.5087	0.7955
0.6939	8.0	16	0.5789	0.7727
0.6939	9.0	18	0.4919	0.8409
0.4553	10.0	20	0.4707	0.75
0.4553	11.0	22	0.5120	0.8182
0.4553	12.0	24	0.4734	0.75
0.4553	13.0	26	0.4255	0.7727
0.4553	14.0	28	0.3695	0.8636
0.3658	15.0	30	0.3848	0.8182
0.3658	16.0	32	0.3586	0.8409
0.3658	17.0	34	0.4962	0.8409
0.3658	18.0	36	0.3645	0.8636
0.3658	19.0	38	0.3455	0.8864
0.2667	20.0	40	0.3477	0.9091
0.2667	21.0	42	0.3275	0.8864
0.2667	22.0	44	0.3400	0.8864
0.2667	23.0	46	0.3780	0.8864
0.2667	24.0	48	0.4243	0.8409
0.1794	25.0	50	0.4429	0.8409
0.1794	26.0	52	0.5026	0.8409
0.1794	27.0	54	0.4811	0.8409
0.1794	28.0	56	0.4733	0.8182
0.1794	29.0	58	0.4384	0.8636
0.1861	30.0	60	0.4354	0.9091
0.1861	31.0	62	0.4511	0.8864
0.1861	32.0	64	0.3315	0.8636
0.1861	33.0	66	0.3100	0.8864
0.1861	34.0	68	0.3594	0.9091
0.1521	35.0	70	0.4052	0.9091
0.1521	36.0	72	0.3878	0.8864
0.1521	37.0	74	0.3905	0.9091
0.1521	38.0	76	0.4173	0.9091
0.1521	39.0	78	0.4774	0.9091
0.1333	40.0	80	0.5656	0.8864
0.1333	41.0	82	0.5146	0.9091
0.1333	42.0	84	0.4158	0.8636
0.1333	43.0	86	0.4067	0.8636
0.1333	44.0	88	0.4412	0.9091
0.1297	45.0	90	0.4733	0.9091
0.1297	46.0	92	0.4243	0.9091
0.1297	47.0	94	0.4279	0.9091
0.1297	48.0	96	0.4020	0.9091
0.1297	49.0	98	0.3842	0.8636
0.1038	50.0	100	0.3811	0.8409
0.1038	51.0	102	0.3947	0.8636
0.1038	52.0	104	0.4587	0.9091
0.1038	53.0	106	0.4300	0.9091
0.1038	54.0	108	0.3804	0.8636
0.1101	55.0	110	0.4216	0.8636
0.1101	56.0	112	0.3966	0.8636
0.1101	57.0	114	0.4216	0.9091
0.1101	58.0	116	0.4569	0.9091
0.1101	59.0	118	0.4392	0.9091
0.1085	60.0	120	0.4479	0.9091
0.1085	61.0	122	0.4657	0.9091
0.1085	62.0	124	0.5242	0.9091
0.1085	63.0	126	0.5626	0.9091
0.1085	64.0	128	0.5570	0.9091
0.105	65.0	130	0.5035	0.9091
0.105	66.0	132	0.4490	0.9091
0.105	67.0	134	0.4366	0.9091
0.105	68.0	136	0.4416	0.8636
0.105	69.0	138	0.4597	0.9091
0.0918	70.0	140	0.4795	0.8636
0.0918	71.0	142	0.4922	0.8636
0.0918	72.0	144	0.5078	0.8409
0.0918	73.0	146	0.5089	0.8636
0.0918	74.0	148	0.5109	0.8636
0.1072	75.0	150	0.5125	0.8864
0.1072	76.0	152	0.5267	0.8864
0.1072	77.0	154	0.5346	0.9091
0.1072	78.0	156	0.5291	0.8864
0.1072	79.0	158	0.5188	0.8636
0.0895	80.0	160	0.5222	0.8636
0.0895	81.0	162	0.5319	0.8636
0.0895	82.0	164	0.5475	0.8864
0.0895	83.0	166	0.5576	0.9091
0.0895	84.0	168	0.5441	0.9091
0.0836	85.0	170	0.5266	0.8864
0.0836	86.0	172	0.5047	0.8864
0.0836	87.0	174	0.4888	0.8864
0.0836	88.0	176	0.4824	0.8864
0.0836	89.0	178	0.4814	0.8864
0.0996	90.0	180	0.4823	0.9091
0.0996	91.0	182	0.4826	0.9091
0.0996	92.0	184	0.4841	0.8864
0.0996	93.0	186	0.4880	0.9091
0.0996	94.0	188	0.4879	0.9091
0.086	95.0	190	0.4829	0.9091
0.086	96.0	192	0.4798	0.8864
0.086	97.0	194	0.4811	0.8864
0.086	98.0	196	0.4819	0.8864
0.086	99.0	198	0.4816	0.8864
0.0745	100.0	200	0.4816	0.8864

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1