Any chance of getting this in Q2_K?
I assume it would land here- the other quants you already released haven't changed, have they?
Thanks again for all that you do!
Correct they've not changed. Well, to be exact, old format quants created with latest llama.cpp would not be compatible with older versions. Which is a bit annoying. But fortunately the latest llama.cpp code is still compatible with files made with the previous versions.
Therefore from now on I will be making q4_0, q4_1, q5_0, q5_1 and q8_0 using llama.cpp from a couple of days ago, so they will be compatible with both the latest and older llama.cpp code. Plus I will be adding the newer quants that only work with latest llama.cpp.
And yes I will be adding the new quant types to most or all of my already released models. That process will be starting shortly.