Breaking the 1.58-Bit Barrier: How BITCOS Squeezes More Out of Ternary LLMs
A new paper from Intel researchers introduces BITCOS, a distribution-aware storage layout for ternary LLMs that exploits the real-world prevalence of zero weights to push effective bit-widths well below the conventional 1.585-bit floor - delivering up to 1.28× throughput gains in production serving.