Code Tiny Experimental
Hello again!
I had a few days to compare your imatrix quant vs the one from Bartowski, and I can say a few things:
When I lock the seed, both versions will almost always output the same code.
There were slight differences with a few seed/prompt combos, and those were generally in your favor!
I'd say your experiment is generally as good and sometimes a bit better than the Bartowski version, but it's hard to tell without doing some proper testing haha.
I tested these in Roocode and OpenWebUI with my usual test prompts, but no big projects or anything like that. So far, so good though!
Thank you very much for this test. This means that there is actually a difference, even if tiny ;).
This is encouraging to try to run it with a bigger imatrix dataset to get even better results.
I will try to compute a larger one. Its gonna take a day or two.
Just finished the imatrix of the code_medium dataset, so uploaded a new one