Moonshot AI Kimi K3.1 Pro: 2.8T Mixture-of-Experts, Dual-Attention, and 1M Multimodal Reasoning

Prior Generation: ← Under the Hood of Moonshot AI’s Kimi K3: The Architecture of 3-Trillion Parameter Thinking Models | Hosting Moonshot AI’s Kimi K3 Open Weights with vLLM: High-Throughput Serving at Scale

Summary

The frontier of artificial intelligence has moved rapidly from static next-token prediction to test-time compute and agentic reasoning. While proprietary laboratories have guarded their reasoning chains behind closed APIs, Moonshot AI has pushed the open-weights frontier forward with the release of Kimi K3.1 Pro—a massive 2.8-trillion parameter Mixture-of-Experts (MoE) model featuring a native 1-million-token context window, joint multimodal perception, and top-tier agentic execution.

Post-trained from Kimi K3 using large-scale joint reinforcement learning across hundreds of knowledge domains, K3.1 Pro delivers a massive leap in long-horizon coding, financial analysis, terminal execution, and factual grounding. To conquer the quadratic memory scaling of million-token context windows, Moonshot AI replaced traditional full-attention layers with a pioneering Hybrid Attention Backbone: pairing 69 linear-time Kimi Delta Attention (KDA) layers with 24 Gated DeepSeek Sparse Attention (Gated DSA) layers.

In this architectural deep dive, we explore the engineering innovations behind Kimi K3.1 Pro, dissect its 896-expert LatentMoE routing topology (activating 104B parameters per token), analyze how its MoonViT-V2 vision encoder handles high-resolution video frames, benchmark its empirical performance across Terminal-Bench 4.0 and Agent Arena, and demonstrate a complete production quickstart using the official Python SDK.

Prior Reading Material

To build a thorough conceptual foundation on Mixture-of-Experts routing, reasoning gateways, and memory architectures, explore these related guides:

• Under the Hood of Moonshot AI’s Kimi K3: The Architecture of 3-Trillion Parameter Thinking Models — Foundational analysis of Preserved Thinking, reasoning effort, and the initial K3 MoE router.
• Hosting Moonshot AI’s Kimi K3 Open Weights with vLLM: High-Throughput Serving at Scale — Multi-node tensor parallelism, chunked prefill, and FP8 serving strategies.
• Understanding Mixture-of-Experts (MoE): From Specialist Clinics to Kimi K3’s 896-Expert Router — Intuitive conceptual guide to sparse gating, router load balancing, and expert capacity factors.
• Basics of AI Inference: Compute vs Memory Bound — Operational intensity, memory bandwidth ceilings, and KV cache scaling economics.

Official Model Card Summary

The official weights, model checkpoints, and developer documentation are openly distributed across the Hugging Face Repository and the Kimi API Platform.

1. Architectural Anatomy: From MLA to Dual-Attention

When scaling context windows beyond 128,000 tokens toward 1 million tokens, traditional Multi-Head Attention (MHA) and even Multi-Head Latent Attention (MLA) hit severe memory bandwidth and compute barriers. Computing pairwise quadratic attention scores across 1M tokens demands massive GPU High Bandwidth Memory (HBM) capacity simply to store Key-Value (KV) cache tensors.

To solve this dilemma without sacrificing pinpoint associative recall, Kimi K3.1 Pro introduces a Hybrid Attention Backbone:

Figure 1: Full-stack architectural breakdown of Kimi K3.1 Pro. The system pairs MoonViT-V2 multimodal perception with a 93-layer hybrid backbone (69 linear KDA layers + 24 Gated DSA sparse layers) driving an 896-expert LatentMoE router activating 104B parameters per token. Source: Moonshot AI Technical Release.

69× Kimi Delta Attention (KDA) Layers

The majority of the network’s layers (69 out of 93) utilize Kimi Delta Attention (KDA). Unlike standard softmax attention that must compare every new token against all previous tokens, KDA maintains a compact, fixed-size recurrent memory state. As new tokens stream into the context window:

1. The model applies a selective delta update rule: new information replaces or adjusts the internal memory state in constant time per step.
2. The memory footprint of the KV representation remains flat regardless of sequence length, dropping operational memory consumption during multi-turn sessions by orders of magnitude.

24× Gated DeepSeek Sparse Attention (Gated DSA) Layers

While linear attention mechanisms excel at bulk information flow, long-range needle-in-a-haystack retrieval requires direct token-to-token indexing. Interleaved throughout the network are 24 Gated DeepSeek Sparse Attention (Gated DSA) layers:

1. Each DSA layer features 32 dedicated index heads (head dimension 128) that scan the context window.
2. Rather than calculating full dense attention matrices, the router dynamically selects only the Top-2048 most relevant key vectors across the 1-million-token horizon.
3. A non-linear gating mechanism dynamically suppresses noisy or irrelevant background tokens, delivering 99.8%+ accuracy on complex multi-document retrieval benchmarks.

2. Fine-Grained LatentMoE: 896 Experts at Scale

In traditional Mixture-of-Experts architectures (such as Mixtral 8x7B), models feature a small pool of 8 to 16 large experts. If a token triggers an expert, that entire large sub-network must be activated, creating high communication overhead and limited specialization.

Kimi K3.1 Pro adopts a Fine-Grained LatentMoE topology:

flowchart TD
    classDef inputStyle fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#ffffff;
    classDef routerStyle fill:#312e81,stroke:#818cf8,stroke-width:2px,color:#ffffff;
    classDef expertStyle fill:#4a044e,stroke:#e879f9,stroke-width:2px,color:#ffffff;
    classDef sharedStyle fill:#064e3b,stroke:#10b981,stroke-width:2px,color:#ffffff;
    classDef outStyle fill:#1e293b,stroke:#f59e0b,stroke-width:2px,color:#ffffff;

    T1["Hidden State Vector (Dimension: 7,168)"]:::inputStyle
    T2["Latent Projection Compression (Dimension: 3,584)"]:::routerStyle
    T3["Routing Gate: Affinity Scoring across 896 Routed Experts"]:::routerStyle
    T4["Select Top-16 Specialized Experts + 2 Dedicated Shared Experts"]:::expertStyle
    T5["Expert Computations with SiTU-GLU Gated Activations"]:::expertStyle
    T6["Weighted Linear Fusion into 104B Activated Parameters per Token"]:::outStyle

    T1 --> T2 --> T3 --> T4 --> T5 --> T6

Why Fine-Grained Experts Matter

• 896 Routed Experts: Each individual expert has a hidden dimension of 3,072. Because each expert is specialized, individual experts master precise conceptual domains (e.g. Linux kernel sys-calls, financial SEC filings, Japanese grammar, or differential equations).
• Top-16 Expert Activation: For each token, the router selects the 16 most relevant experts. This yields 104 billion active parameters per forward pass—delivering frontier-class reasoning capacity while keeping inference execution latency comparable to a much smaller model.
• 2 Always-On Shared Experts: Two dedicated shared experts process every single token unconditionally. These experts capture universal grammar, sentence syntax, and coordination semantics, freeing the 896 routed experts to concentrate exclusively on specialized domain knowledge.

3. Empirical Benchmarks & Factual Grounding

Moonshot AI specifically engineered K3.1 Pro to excel at high-stakes technical and financial knowledge work, where hallucinations are unacceptable and complex multi-step tool execution is mandatory.

Figure 2: Empirical benchmark evaluations of Kimi K3.1 Pro across agentic workflows (Terminal-Bench 4.0), open-weight leaderboard rankings (Agent Arena), hallucination mitigation (Kimi Hallu Bench Hard), and bio-safety competencies (SecureBio BCI). Source: Moonshot AI Technical Report / Agent Arena.

1. Terminal-Bench 4.0: 65.7%

Terminal-Bench evaluates an AI agent’s ability to operate real command-line environments, debug multi-file software repositories, inspect Docker containers, and fix system configuration errors. Kimi K3.1 Pro achieves 65.7%, surpassing Claude 3.7 Sonnet (64.9%) and outperforming vanilla GPT-4o (43.1%) by over 22 percentage points.

2. Agent Arena #1 Open-Weight Model

On the competitive Agent Arena evaluation leaderboard, Kimi K3.1 Pro captured the #1 overall position among open-weight models (ranking #13 globally against all proprietary frontier systems), demonstrating a +5.1% net improvement across programming, complex chat, and data science workflows.

3. Kimi Hallu Bench Hard: 4.2× Factual Accuracy

In financial modeling and investment research, small factual errors can invalidate multi-million dollar portfolios. On Moonshot AI’s rigorous Kimi Hallu Bench Hard, the claim-level hallucination rate collapsed from 23.4% in K3 down to 5.6% in K3.1 Pro—delivering an effective 4.2× performance multiplier in financial and regulatory statement accuracy.

4. Preserved Thinking & Reasoning Effort

Like its predecessor, Kimi K3.1 Pro is a native thinking model. Rather than jumping straight into response generation, the model produces internal “reasoning tokens” where it deconstructs problems, tests edge cases, and self-corrects before outputting a final answer.

K3.1 Pro refines this paradigm in two critical dimensions:

flowchart TD
    classDef turnStyle fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#ffffff;
    classDef thinkStyle fill:#312e81,stroke:#a855f7,stroke-width:2px,color:#ffffff;
    classDef respStyle fill:#064e3b,stroke:#10b981,stroke-width:2px,color:#ffffff;

    U1["Turn 1 User Prompt: 'Analyze this 400-page 10-K filing'"]:::turnStyle
    R1["Kimi Internal Deliberation: Generates 3,200 Reasoning Tokens"]:::thinkStyle
    A1["Assistant Response: Synthesizes Capital Expenditure Risks"]:::respStyle
    U2["Turn 2 User Follow-Up: 'Why did depreciation increase in Q3?'"]:::turnStyle
    P2["Preserved Thinking Context: Re-reads Previous Reasoning Chain"]:::thinkStyle
    A2["Immediate Grounded Answer without Re-Computing from Scratch"]:::respStyle

    U1 --> R1 --> A1 --> U2 --> P2 --> A2

1. Configurable Reasoning Effort: Developers can control the depth of deliberation via the API parameter reasoning_effort:
   • low: Fast, direct responses for lightweight chat and basic text summarization.
   • high: Balanced deliberation for standard programming and architectural planning.
   • max: Deep mathematical proofs, multi-file code refactoring, and exhaustive financial auditing.
2. Preserved Thinking Across Multi-Turn Sessions: In conventional reasoning APIs, reasoning tokens are discarded after each turn, forcing the model to re-analyze entire contexts when a user asks a follow-up question. Kimi K3.1 Pro allows developers to preserve and pass back reasoning_content in conversation histories, maintaining active cognitive state across multi-turn sessions without redundant deliberation delays.

5. Hands-On Quickstart: Production API Integration

The Kimi API Platform provides complete OpenAI SDK compatibility. The script below demonstrates initializing the client, setting maximum reasoning effort, submitting a long-horizon reasoning query, and parsing both the reasoning deliberation chain and the final answer.

"""
Kimi K3.1 Pro Production Quickstart.
Demonstrates client initialization, reasoning effort configuration,
and multi-turn preserved thinking using the official OpenAI Python SDK.
"""

import os
from openai import OpenAI

def run_kimi_k3_1_pro_query():
    # 1. Retrieve API key from environment
    api_key = os.environ.get("MOONSHOT_API_KEY", "your-api-key-here")
    
    # 2. Initialize client with Moonshot AI's production base URL
    client = OpenAI(
        api_key=api_key,
        base_url="https://api.moonshot.ai/v1",
    )
    
    # 3. Formulate prompt requiring deep reasoning and tool planning
    messages = [
        {
            "role": "system",
            "content": (
                "You are Kimi K3.1 Pro, an advanced reasoning and agentic model developed by Moonshot AI. "
                "Deliberate thoroughly before answering."
            )
        },
        {
            "role": "user",
            "content": (
                "Design a distributed event-driven microservices architecture for handling 500,000 "
                "cryptocurrency trades per second with sub-5ms p99 latency. Analyze memory bandwidth, "
                "network serialization bottlenecks, and Raft consensus failure modes."
            )
        }
    ]
    
    print("Submitting query to Kimi K3.1 Pro...")
    
    # 4. Execute completion with reasoning effort dial set to 'max'
    response = client.chat.completions.create(
        model="kimi-k3.1-pro",  # Alternatively use 'kimi-k3.1-pro-highspeed' for lower latency
        messages=messages,
        temperature=1.0,
        top_p=0.95,
        extra_body={
            "reasoning_effort": "max"  # Options: 'low', 'high', 'max'
        }
    )
    
    choice = response.choices[0].message
    
    # 5. Extract internal thinking tokens if returned by the endpoint
    reasoning_text = getattr(choice, "reasoning_content", None)
    if reasoning_text:
        print("\n" + "=" * 60)
        print("KIMI K3.1 PRO INTERNAL REASONING TRACE:")
        print("=" * 60)
        print(reasoning_text[:1200] + "\n... [Remaining reasoning trace collapsed] ...")
        
    print("\n" + "=" * 60)
    print("FINAL SYNTHESIZED RESPONSE:")
    print("=" * 60)
    print(choice.content)
    
    # 6. Usage & Token Economics
    usage = response.usage
    if usage:
        print("\n" + "-" * 60)
        print(f"Token Accounting: Prompt Tokens = {usage.prompt_tokens:,} | "
              f"Completion Tokens = {usage.completion_tokens:,} | "
              f"Total = {usage.total_tokens:,}")
        print("-" * 60)

if __name__ == "__main__":
    run_kimi_k3_1_pro_query()

6. Token Economics & Pricing Breakdown

Despite its 2.8-trillion parameter architecture, Kimi K3.1 Pro’s fine-grained LatentMoE design (activating only 104B parameters) and hybrid attention mechanics allow Moonshot AI to offer aggressive pricing for a million-token context window:

Direct Platform Pricing (Kimi API Platform)

• Kimi K3.1 Pro (Standard):
  • Input Tokens: $2.00 per 1M tokens
  • Cached Input Tokens: $0.20 per 1M tokens (90% discount on cache hits)
  • Output Tokens: $10.00 per 1M tokens
  • Context Cache Retention: $2.00 / 1M tokens for 5-minute TTL; $4.00 / 1M tokens for 1-hour TTL
• Kimi K3.1 Pro Highspeed (2–3× faster generation for interactive code execution):
  • Input Tokens: $4.00 per 1M tokens
  • Output Tokens: $20.00 per 1M tokens

Global Aggregation via OpenRouter

For international development teams seeking unified billing without domestic Chinese SMS or payment verification, Kimi K3.1 Pro is available directly on OpenRouter (moonshotai/kimi-k3.1-pro) matching the official standard rates:

With OpenRouter, developers can integrate Kimi K3.1 Pro alongside existing enterprise LLM routers using the standard OpenAI client:

# Calling Kimi K3.1 Pro via OpenRouter
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ.get("OPENROUTER_API_KEY", "your-openrouter-key"),
)

response = client.chat.completions.create(
    model="moonshotai/kimi-k3.1-pro",
    messages=[
        {"role": "user", "content": "Analyze this 200,000-line codebase for architectural bottlenecks."}
    ],
)
print(response.choices[0].message.content)

For organizations ingesting multi-hundred-page documents or continuous codebase repositories, prompt caching reduces the effective cost of million-token context queries to pennies, making large-scale agentic analysis economically viable.

Conclusion & What’s Ahead

With Kimi K3.1 Pro, Moonshot AI has demonstrated that open-weight architectures can successfully bridge the gap to top-tier proprietary frontier models. By pioneering a dual KDA + Gated DSA attention backbone, scaling LatentMoE to 896 fine-grained experts, and integrating native MoonViT-V2 vision encoding, K3.1 Pro provides a flexible foundation for long-context reasoning, terminal-level agentic workflows, and high-precision technical research.

Whether deployed via hosted endpoints at $2.00 per million tokens or served on private GPU infrastructure using vLLM and MXFP4 quantization, Kimi K3.1 Pro sets a formidable standard for what open multimodal thinking models can achieve.