Spaces:

competitions

/

aiornot

like 137

Runtime error

Thanks a lot for this competition! It was really fun to participate outside of kaggle. At first, I only joined to test out the new competition setup on huggingface, but once seeing the movement on the leaderboard, I couldn't resist of fitting a few more models and subbing a few larger blends.

Validation

Simple random 5 KFold split had very good agreement with LB score, so I just created the folds once and always sticked to them.

Models

At start, I ran a few basic backbones from the timm library and quickly settled to the efficientnetv2 family as being strong with this data and easy to tune. Later, I tried to add more models to the mix, but had a hard time to find good hyperparameters that would match the efficientnetv2 performance. In all experiments I used AdamW optimizer, cosine scheduler, ImageNet normalization and very moderate augmentation including hflip and mixup with 0.3 to 0.4 chance. I applied horizontal flip as test time augmentation.
The final blend consists of:

Post-processing

With logloss metric, the absolute values of the predictions are important. Especially for blends, I assumed that one can tune the predictions to be more certain than an average would suggest. So, I decided to apply a power transform and was surprised how much it helped the CV score. I also decided to clip the predictions at 1e-4 and (1 - 1e-4), so the logloss can not explode on few errors.

val["pred_label"] = np.mean([x["pred_label"] for x in vals], axis=0) ** 1.40
val.loc[val["pred_label"] > 0.5, "pred_label"] **= 0.5
val.loc[val["pred_label"] < 0.5, "pred_label"] **= 1.5
val.loc[val["pred_label"] > 0.999, "pred_label"] = 1 - 1e-4
val.loc[val["pred_label"] < 0.001, "pred_label"] = 1e-4

which gives a pred distribution (5fold train oofs) like this for the final blend:

I applied the same post-processing to test, and it seemed to have the same impact judging by the public LB scores.

What I missed

• Cutmix probably ...
• Upscaling (I tried stride 1, but that didn't help so I didn't look into it further, it was probably a mistake)
• Other backbones

Bugs

I made a late submission (shows march 1st) and it seemed to have removed me from the public leaderboard.
One submission did not score, though submitting it again did the trick.