Chinese AI lab Moonshot AI released Kimi K3, an open-weight model with 2.8 trillion parameters, the largest open-weight model released to date, and the first to enter what the industry calls the “3T class.” That headline number has dominated the coverage. But the more revealing story isn’t how big K3 is; it’s how Moonshot built something that big while working under US chip export restrictions. The answer: by engineering around a compute shortage using memory-saving tricks, in a domain where China’s chip industry happens to be weakest.
Compute vs. Memory: Two Different Bottlenecks
Running a large language model costs resources in two separate ways. Compute is the raw calculation needed to produce each word. Memory is how much of the model has to sit loaded and instantly accessible the whole time it’s running. Export controls have primarily targeted compute-heavy chips, so Chinese labs have increasingly designed models that spend less of it even if that shifts the burden onto memory instead.
Kimi K3 is a case study in that shift, trading compute for memory at nearly every layer of its design.
How K3 Cuts Compute Without Cutting Memory
K3’s core trick is mixture-of-experts (MoE): instead of activating the entire 2.8-trillion-parameter model for every word it generates, it splits itself into 896 specialized “expert” sections and calls on just 16 at a time, roughly 1.8% of the total. That slashes the calculation needed per word.
The catch: memory usage doesn’t budge. All 2.8 trillion parameters still have to be loaded and ready, since any of them could be the ones called on next. MoE solves the compute problem but leaves the memory problem fully intact, which is where the rest of K3’s design comes in.
Shrinking the Memory Bill
Moonshot attacked the memory cost directly, in two ways:
- Aggressive quantization. K3 was trained to run at 4 bits of precision per parameter instead of the usual 16, a technique Moonshot applied from the fine-tuning stage onward, describing the goal as broad hardware compatibility (widely read as a hedge against having to run on non-Nvidia chips). The payoff is large: independent analysis puts the model at roughly 1.4TB in this format, versus 5.6TB at full precision.
- Kimi Delta Attention. As a model reads through a long document, it builds up a running memory of everything it has already processed. At K3’s advertised limit of one million tokens (a few thousand pages), that accumulated store, not the model’s own weights, becomes the largest thing occupying memory. Moonshot’s new attention mechanism is designed to shrink that cost, and the company has contributed related caching code to the open-source vLLM serving project, saying this combination is what makes K3’s pricing viable at its size.
Moonshot recommends running K3 across 64 or more accelerators pooled tightly enough to function as one unit, the same broad strategy behind Huawei’s CloudMatrix systems. That points to the real workaround at play: memory can be pooled across many ordinary chips, but high-end training compute can’t be assembled the same way.
The Hardware Question Nobody’s Answering
Moonshot’s technical disclosures are notably vague about where K3 actually runs. Its chip-level tests used Nvidia H200s and an unnamed “GPGPU from an alternative vendor,” while other benchmarks ran on the Nvidia L20, the reduced-capability card permitted for sale into China. Where the H200 testing physically took place isn’t stated, a detail that matters given that the US House passed a bill in January 2026 to close the offshore-cloud-rental loophole that let Chinese firms access restricted accelerators remotely.
This also lines up with what China itself has flagged as the binding constraint. In August 2025 trade talks, Beijing reportedly pushed for relief on high-bandwidth memory restrictions specifically, not lithography tools or TSMC access, suggesting officials see memory, not raw processing power, as the tighter bottleneck. Domestic memory output is projected at around 2 million stacks this year, enough for roughly 250,000–300,000 Huawei Ascend 910C-class chips, even though SMIC has wafer capacity for over a million.
Can Enterprises Actually Deploy It?
For businesses, especially across Asia, the real question isn’t leaderboard rank; it’s deployability. Companies gravitate to open-weight models for cost, data sovereignty, and regional-language support, and sectors like banking and insurance have been piloting self-hosted open models specifically to keep records on their own infrastructure.
K3’s weights become available on July 27, but running them is a data-center-scale commitment, not a server-room one:
- ~1.4TB just for the weights (before accounting for long-context memory)
- Recommended deployment across 64+ pooled accelerators
- Tooling isn’t fully ready; K3’s architectural changes are new enough that standard open-source inference tools don’t yet support them; Moonshot says it’s working with partners to close that gap before general availability.
For most organizations, that likely means renting dedicated capacity rather than owning hardware outright, which still satisfies in-country data requirements but doesn’t deliver the infrastructure independence that draws many buyers to open weights in the first place.
Pricing Has Shifted Up-Market
K3 costs $3 per million input tokens (dropping to $0.30 for cached input) and $15 per million output tokens. For context: that’s well below Claude Fable 5’s $50 output pricing, but noticeably above budget rivals like z.ai’s GLM-5.2 ($4.40) and DeepSeek V4 ($0.87). K3 has effectively moved out of the budget tier its predecessors occupied. It also currently launches with only a maximum-reasoning-effort setting (lower-effort modes are planned later), meaning long reasoning chains and their costs add up quickly.
Benchmark Claims vs. What’s Verified
K3 placed first in Arena’s Frontend Code evaluation (1,679 points), ahead of Claude Fable 5, according to Tom’s Hardware’s coverage of blind developer testing a real result, but from a single domain.
Moonshot itself is fairly candid about limitations, stating that K3’s overall performance still trails Claude Fable 5 and GPT 5.6 Sol, and flagging three specific weak points: instability when a serving harness fails to pass back prior reasoning context, occasional unrequested decisions when user intent is ambiguous, and a noticeable user-experience gap versus those two rivals. The company’s own footnotes also note that Fable 5 hit fallback behavior on 35% of tasks in Moonshot’s internal “SWE Marathon” benchmark, which may have affected Fable 5’s measured score there.
Everything beyond independently reproduced results remains a first-party claim; nothing can be fully verified until the weights are actually public on July 27.
Bank of America analysts, in a note led by Alex Liu, offered a more measured read: that large-scale pre-training paired with targeted architectural work can still produce meaningful gains for flagship Chinese models despite compute constraints, a claim closer to what Moonshot’s own technical disclosures actually support.
The Bigger Trend
Whatever K3’s exact standing turns out to be, the shift it represents is already visible in usage data: open-weight models accounted for 29% of tokens routed through Vercel’s production AI gateway in June 2026, up from roughly one-ninth in April, while representing under 4% of total spending. July 27, when K3’s weights actually ship, will be the first real test of how much of that growth K3 can claim for itself.
