napsternxg/nyt-ingredient-tagger-paraphrase-MiniLM-L3-v2

model

This model is a fine-tuned version of sentence-transformers/paraphrase-MiniLM-L3-v2 on the nyt_ingredients dataset. It achieves the following results on the evaluation set:

• Loss: 0.4745
• Comment: {'precision': 0.6381763059701493, 'recall': 0.7527162701141521, 'f1': 0.6907301066447908, 'number': 7271}
• Name: {'precision': 0.7925138150349286, 'recall': 0.8159081150708458, 'f1': 0.8040408314380917, 'number': 9316}
• Qty: {'precision': 0.9870301746956062, 'recall': 0.9904382470119522, 'f1': 0.988731274028901, 'number': 7530}
• Range End: {'precision': 0.6532258064516129, 'recall': 0.9310344827586207, 'f1': 0.7677725118483412, 'number': 87}
• Unit: {'precision': 0.9281956050758279, 'recall': 0.9844083374364024, 'f1': 0.9554759060135404, 'number': 6093}
• Overall Precision: 0.8236
• Overall Recall: 0.8783
• Overall F1: 0.8501
• Overall Accuracy: 0.8310

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Comment	Name	Qty	Range End	Unit	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.5473	0.2	1000	0.5439	{'precision': 0.53239608801956, 'recall': 0.6330862043901729, 'f1': 0.5783916594727406, 'number': 6879}	{'precision': 0.7656748140276302, 'recall': 0.816245610060043, 'f1': 0.7901518890168339, 'number': 8827}	{'precision': 0.9752864835013116, 'recall': 0.9824756606397774, 'f1': 0.9788678722372341, 'number': 7190}	{'precision': 0.6060606060606061, 'recall': 0.7317073170731707, 'f1': 0.6629834254143646, 'number': 82}	{'precision': 0.923214867949136, 'recall': 0.9828184658104825, 'f1': 0.9520847343644923, 'number': 5762}	0.7837	0.8471	0.8142	0.8057
0.5634	0.4	2000	0.5237	{'precision': 0.5564878997932629, 'recall': 0.6652129670010176, 'f1': 0.6060124486822938, 'number': 6879}	{'precision': 0.7951952610794208, 'recall': 0.8212303160756769, 'f1': 0.8080031209942595, 'number': 8827}	{'precision': 0.9757675891504888, 'recall': 0.9856745479833101, 'f1': 0.9806960492631287, 'number': 7190}	{'precision': 0.5725806451612904, 'recall': 0.8658536585365854, 'f1': 0.6893203883495146, 'number': 82}	{'precision': 0.9235782955841616, 'recall': 0.9836862200624783, 'f1': 0.9526850995882007, 'number': 5762}	0.7987	0.8577	0.8272	0.8120
0.5535	0.59	3000	0.5022	{'precision': 0.5893937596393404, 'recall': 0.7221979938944614, 'f1': 0.6490723804546643, 'number': 6879}	{'precision': 0.7913148371531966, 'recall': 0.8174917865639515, 'f1': 0.8041903488242506, 'number': 8827}	{'precision': 0.9812708102108768, 'recall': 0.9837273991655077, 'f1': 0.9824975691068204, 'number': 7190}	{'precision': 0.562962962962963, 'recall': 0.926829268292683, 'f1': 0.7004608294930875, 'number': 82}	{'precision': 0.931615460852329, 'recall': 0.9788267962513016, 'f1': 0.9546377792823292, 'number': 5762}	0.8070	0.8689	0.8368	0.8213
0.5366	0.79	4000	0.4892	{'precision': 0.6037854098771622, 'recall': 0.7002471289431603, 'f1': 0.6484485427744499, 'number': 6879}	{'precision': 0.7957470010905126, 'recall': 0.826668177183641, 'f1': 0.8109129299327665, 'number': 8827}	{'precision': 0.9751884852638794, 'recall': 0.9894297635605007, 'f1': 0.9822575077666552, 'number': 7190}	{'precision': 0.5652173913043478, 'recall': 0.9512195121951219, 'f1': 0.7090909090909091, 'number': 82}	{'precision': 0.9284076015727392, 'recall': 0.9835126692120791, 'f1': 0.955166020562953, 'number': 5762}	0.8139	0.8689	0.8405	0.8251
0.5256	0.99	5000	0.4813	{'precision': 0.6161294276259346, 'recall': 0.730774821921791, 'f1': 0.6685729485303898, 'number': 6879}	{'precision': 0.7992788461538461, 'recall': 0.8287073750991277, 'f1': 0.8137271260915513, 'number': 8827}	{'precision': 0.9784340659340659, 'recall': 0.9906815020862308, 'f1': 0.9845196959225985, 'number': 7190}	{'precision': 0.6330275229357798, 'recall': 0.8414634146341463, 'f1': 0.7225130890052357, 'number': 82}	{'precision': 0.9291687161829808, 'recall': 0.9835126692120791, 'f1': 0.9555686704325098, 'number': 5762}	0.8182	0.8769	0.8465	0.8299
0.5079	1.19	6000	0.4766	{'precision': 0.6228698444060262, 'recall': 0.7332461113533943, 'f1': 0.6735661347399347, 'number': 6879}	{'precision': 0.8044889426779623, 'recall': 0.82836750877988, 'f1': 0.8162536280419737, 'number': 8827}	{'precision': 0.9840742279462679, 'recall': 0.988317107093185, 'f1': 0.9861911040177642, 'number': 7190}	{'precision': 0.6306306306306306, 'recall': 0.8536585365853658, 'f1': 0.7253886010362693, 'number': 82}	{'precision': 0.928082191780822, 'recall': 0.9876778896216591, 'f1': 0.9569530855893728, 'number': 5762}	0.8229	0.8776	0.8494	0.8313
0.5047	1.39	7000	0.4780	{'precision': 0.6244848484848485, 'recall': 0.7489460677424045, 'f1': 0.6810760790534734, 'number': 6879}	{'precision': 0.8084753263996459, 'recall': 0.8278010649144669, 'f1': 0.8180240694094598, 'number': 8827}	{'precision': 0.9799036476256022, 'recall': 0.990125173852573, 'f1': 0.9849878934624697, 'number': 7190}	{'precision': 0.5923076923076923, 'recall': 0.9390243902439024, 'f1': 0.7264150943396225, 'number': 82}	{'precision': 0.9348113831899404, 'recall': 0.9805623047552933, 'f1': 0.9571404370658986, 'number': 5762}	0.8235	0.8805	0.8511	0.8305
0.4912	1.58	8000	0.4725	{'precision': 0.6316635745207174, 'recall': 0.7424044192469835, 'f1': 0.6825715049452018, 'number': 6879}	{'precision': 0.8068570168669386, 'recall': 0.8291605301914581, 'f1': 0.8178567437702537, 'number': 8827}	{'precision': 0.9846047156726768, 'recall': 0.9873435326842838, 'f1': 0.9859722222222222, 'number': 7190}	{'precision': 0.6428571428571429, 'recall': 0.8780487804878049, 'f1': 0.7422680412371134, 'number': 82}	{'precision': 0.9298820445609436, 'recall': 0.9850746268656716, 'f1': 0.9566829597168379, 'number': 5762}	0.8264	0.8794	0.8521	0.8342
0.4955	1.78	9000	0.4725	{'precision': 0.6421661012690036, 'recall': 0.7429858991132432, 'f1': 0.688906860762906, 'number': 6879}	{'precision': 0.8048323036187114, 'recall': 0.8264415996374759, 'f1': 0.8154938237102454, 'number': 8827}	{'precision': 0.9815401570464252, 'recall': 0.9909596662030598, 'f1': 0.9862274205827393, 'number': 7190}	{'precision': 0.582089552238806, 'recall': 0.9512195121951219, 'f1': 0.7222222222222221, 'number': 82}	{'precision': 0.9313403416557161, 'recall': 0.9840333217632766, 'f1': 0.9569620253164556, 'number': 5762}	0.8287	0.8796	0.8534	0.8332
0.4917	1.98	10000	0.4697	{'precision': 0.6389365351629502, 'recall': 0.7581043756359936, 'f1': 0.6934379363074265, 'number': 6879}	{'precision': 0.8106822956983302, 'recall': 0.8305199954684491, 'f1': 0.8204812534974818, 'number': 8827}	{'precision': 0.9851553829078802, 'recall': 0.9876216968011127, 'f1': 0.9863869981941935, 'number': 7190}	{'precision': 0.6347826086956522, 'recall': 0.8902439024390244, 'f1': 0.7411167512690355, 'number': 82}	{'precision': 0.9327744904667982, 'recall': 0.9849010760152724, 'f1': 0.9581293263548878, 'number': 5762}	0.8296	0.8836	0.8557	0.8341
0.4913	2.18	11000	0.4685	{'precision': 0.6405220633934121, 'recall': 0.7490914377089694, 'f1': 0.6905655320289467, 'number': 6879}	{'precision': 0.8053573388955978, 'recall': 0.8310864393338621, 'f1': 0.8180196253345228, 'number': 8827}	{'precision': 0.9836745987825124, 'recall': 0.9888734353268428, 'f1': 0.9862671660424469, 'number': 7190}	{'precision': 0.6454545454545455, 'recall': 0.8658536585365854, 'f1': 0.7395833333333335, 'number': 82}	{'precision': 0.9313854235062377, 'recall': 0.9847275251648733, 'f1': 0.9573139868398851, 'number': 5762}	0.8287	0.8818	0.8544	0.8355
0.4769	2.38	12000	0.4659	{'precision': 0.6392910634048926, 'recall': 0.7445849687454572, 'f1': 0.6879323081055672, 'number': 6879}	{'precision': 0.8030103274005713, 'recall': 0.8280276424606321, 'f1': 0.8153271236544146, 'number': 8827}	{'precision': 0.9858431644691187, 'recall': 0.9878998609179416, 'f1': 0.9868704411253908, 'number': 7190}	{'precision': 0.6607142857142857, 'recall': 0.9024390243902439, 'f1': 0.7628865979381443, 'number': 82}	{'precision': 0.9313339888561127, 'recall': 0.9862894828184658, 'f1': 0.958024275118004, 'number': 5762}	0.8283	0.8800	0.8534	0.8353
0.4752	2.57	13000	0.4651	{'precision': 0.641625, 'recall': 0.7461840383776712, 'f1': 0.6899657235029236, 'number': 6879}	{'precision': 0.8089998899768952, 'recall': 0.833012348476266, 'f1': 0.8208305425318152, 'number': 8827}	{'precision': 0.9854389127721537, 'recall': 0.988317107093185, 'f1': 0.9868759113950422, 'number': 7190}	{'precision': 0.6634615384615384, 'recall': 0.8414634146341463, 'f1': 0.7419354838709676, 'number': 82}	{'precision': 0.932905772076961, 'recall': 0.9845539743144741, 'f1': 0.95803428185426, 'number': 5762}	0.8310	0.8815	0.8555	0.8359
0.4834	2.77	14000	0.4628	{'precision': 0.6457421533074903, 'recall': 0.7506905073411834, 'f1': 0.694272653939231, 'number': 6879}	{'precision': 0.8060932688077431, 'recall': 0.830293417922284, 'f1': 0.8180143981248955, 'number': 8827}	{'precision': 0.9835589941972921, 'recall': 0.990125173852573, 'f1': 0.9868311616301636, 'number': 7190}	{'precision': 0.6324786324786325, 'recall': 0.9024390243902439, 'f1': 0.7437185929648242, 'number': 82}	{'precision': 0.9318890530116527, 'recall': 0.9854217285664699, 'f1': 0.9579080556727119, 'number': 5762}	0.8306	0.8825	0.8558	0.8365
0.4784	2.97	15000	0.4626	{'precision': 0.6482109227871939, 'recall': 0.7505451373746184, 'f1': 0.6956345998383185, 'number': 6879}	{'precision': 0.8074424749532093, 'recall': 0.8308598617876969, 'f1': 0.8189838079285315, 'number': 8827}	{'precision': 0.9836881393419962, 'recall': 0.9897079276773296, 'f1': 0.9866888519134775, 'number': 7190}	{'precision': 0.6460176991150443, 'recall': 0.8902439024390244, 'f1': 0.7487179487179487, 'number': 82}	{'precision': 0.9323925172300623, 'recall': 0.9861159319680667, 'f1': 0.958502024291498, 'number': 5762}	0.8320	0.8827	0.8566	0.8370

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3