Why was this taken off the Leaderboard?
Not sure why, but I can't find this model's evals on the leaderboard..
I agree, I find this model performs better than Stable Beluga 7B
Exactly I am trying to get this answer from HuggingFace team, since last week, so far no real explanation was given, just the message that it failed, obviously anyone can with repo link can do the same evals and they are working fine,so not sure why they are holding up this model and 7 other of mine, when other model submission are easily showing up on LB, here is thread
Wow.. @huggingface
What's up? If you want to build our trust and keep it, maybe explain what's going on here?
Wtf, I can't tag HuggingFace here..
Can I clarify something here Pankaj -- did this appear on the leaderboard and then got taken down? Because I'm pretty sure I saw the evals for this model on the leaderboard (but maybe that was on Twitter).. Can you clarify please? I'm going to ask this on Twitter, but want to get the facts right first.
No, I don't think it was taken down or something, I think it just FAILED after submission, and the HF team is trying to redo the evals but I guess it failing again for them. so really not sure what's going on, unless the team reply.
here are the logs
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/psmathur/orca_mini_v3_7b_eval_request_False_float16_Original.json
Gotcha.. Okay, thanks man.
What's up? If you want to build our trust and keep it, maybe explain what's going on here?
lets not attribute malice where it could just be software problems :D
I get it! :)
Congratulations on hitting the leaderboard Pankaj!
Thanks 🙏