We don’t know what quantization level was used for the weights or the kv cache f...

cellularmitosis 26 days ago | parent | context | favorite | on: Why your local LLM feels dumber than it is

We don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison.