r/LocalLLaMA • u/po_stulate • 15h ago

Discussion Deceptive model quantization from AtomicChat?

I kept seeing guys in this sub saying how AtomicChat's Qwen3.8-Flash-Next quant is so good, fits in their machine when unsloth's can't, runs faster than other quants etc, so I went check out what's happening there.

First thing I noticed was that AtomicChat's Q4_K_M quant is suspiciously small when the ngram table is removed (only ~56GB), it seems like most of the tensors in this quant are IQ2_S instead of the usual Q4_K, Q5_K and Q6_K that you usually find in Q4_K_M quants, the GGUF filetype metadata also says IQ2_S instead of Q4_K_M. In their model card, their Q4_K_M also has suspiciously high KLD (0.084).

It seems pretty obvious to me that they're pretending a IQ2_S quant as a Q4_K_M, but at the same time I'm genuinely not sure because it can't be only me who found this right? How can nobody be pointing this out? Am I missing something or what may they be doing?

Their HF repo ID: AtomicChat/Qwen3.8-Flash-Next-GGUF

56 Upvotes

83% Upvoted

38 comments sorted by

View all comments

-3

u/ParaboloidalCrest 14h ago

Why so emotional? This is not Taylor Swift sub! What you have "uncovered" is already there in the model card. It's not like you need to reverse engineer the quants or anything.

Besides, there are TENS and very soon HUNDREDS of qwen3.8next quants and all are free. Use whatever you want and ignore the rest.

9

u/po_stulate 14h ago

I made my own quant for qwen3.8-flash-next (because most either don't have unquantized ngram table or don't have mtp) and I'm happy with it. I'm just curious since I didn't see any discussion about their quants regarding this.