Finally decent results.
I've been perplexity testing some of these on different datasets. For any chat type log, OG mixtral-instruct does best with more experts, even up to 8. Other tunes it's 3 or bust, no matter what you feed them, it's the exact same pattern. This one however, is all over the place.
Chatlogs - textgen:
2 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.8989474773406982
3 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.8434770107269287
4 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.8221964836120605
5 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.814671277999878
6 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.814753293991089
7 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.819715738296509
8 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.8253870010375977
ptb_new (199) - textgen:
2 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 19.62730598449707
3 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 18.78565788269043
4 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 18.902372360229492
5 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 18.98589515686035
6 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 20.042049407958984
7 experts -
8 experts -
GU_small (134) - textgen:
2 experts -
3 experts -
4 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 2.887450695037842
5 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 2.8757941722869873
6 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 2.8695790767669678
roleplay_half (402) - textgen:
2 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.2560477256774902
3 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.2203352451324463
4 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.217078685760498
5 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.217573642730713
6 experts - Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-6.0bpw-h6-exl2-rpcal is: 3.2202346324920654
Didn't finish some yet, once I saw where the trend was going.
4 experts is what I'm going with until I run more d/s. Its definitely keeping up with original instruct in terms of how it plays characters. In all the other community tunes I tried, none of them ever picked up accents from the card on their own. Thus far, it seems to break less and repeat loop a bit less too. Even at the nuclear option of 4 temp and .1 minP.