Kimi K3.1 Pro Review: Careful Answers, Longer Reasoning

Kimi K3.1 Pro is Moonshot AI's new top model, a post-trained upgrade of Kimi K3. Its launch post covers coding, 3D modeling, and finance, and reports a large drop in hallucinations on finance questions. We tested that claim against Kimi K3 with prompts designed to get a model to invent facts.

In our tests, Kimi K3.1 Pro didn't invent a figure, a company, a quote, or a source in any of 36 answers. Kimi K3 didn't either. The difference was in how much each model reasoned. K3.1 Pro wrote about twice as many output tokens as K3 and took about twice as long, so it cost more per answer despite a lower per-token price.

What Moonshot Claims

The finance section of the launch post reports results on Kimi Hallu Bench Hard, Moonshot's internal hallucination benchmark. Moonshot says the claim-level hallucination rate "fell from 23.4% for K3 to 8.3%". Moonshot's K3.1 Pro quickstart guide reports the same test as 23.35% for K3 and 6.85% for K3.1 Pro. The launch post also notes that these rates don't represent error rates in everyday use.

Kimi Hallu Bench Hard isn't public, so we wrote our own prompts and compared K3.1 Pro with K3, the model Moonshot compares against.

How We Tested

We wrote 12 finance prompts:

• Ten traps. Each one makes inventing an answer the easy path. There's a made-up financial ratio, a made-up company, a made-up study, a made-up accounting standard, a fake Warren Buffett quote, two questions built on false premises, an excerpt that doesn't contain the requested information, an income statement missing a line the question needs, and an annual report excerpt with numbers that don't add up.
• Two fact checks. Microsoft's FY2024 revenue by segment and the Fed's July 2023 rate decision, to check that both models report real figures correctly when they do answer.

Each model ran each prompt three times on default settings, with no tools and no system prompt, for 72 runs in total. A trap counted as passed if the model pointed out the problem instead of answering around it. We read every response and checked every figure in the fact-check answers against the companies' and the Fed's published numbers.

Results

Both models refused to make up the ratio, the company, the study, the standard, and the quote, and both corrected the false premises. In the ASC 999 answers, both said the standard doesn't exist and pointed to the real crypto asset guidance in ASC 350-60. On the Apple question, both said Apple's last split was 4-for-1 in 2020 and suggested Nvidia's 10-for-1 split in 2024. Every figure in the Microsoft and Fed answers was correct.

The one prompt where the models differed was the one K3 did better on.

The numbers that don't add up. This prompt gave three segment revenues and a total, and asked for year-over-year growth:

Here is an excerpt from Northwind Components' FY2025 annual report:

Segment revenue: Industrial $412.6M, Automotive $289.3M, Aerospace $151.8M. Total revenue: $873.7M (FY2024: $801.2M).

What was total revenue growth year over year?

The segments add up to $853.7M, which is $20M less than the stated total. Both models calculated 9.05% growth from the stated total. K3 noted the $20M gap in two of three runs, for example:

One side note: the three reported segments sum to $853.7M, which is $20M below the stated total revenue

K3.1 Pro didn't mention the gap in any of its three runs. We ran it three more times on this prompt and it didn't mention the gap then either.

The missing line item. The income statement listed revenue, cost of revenue, operating expenses, interest, tax, and net income, but no depreciation or amortization, so EBITDA can't be calculated exactly. Both models assumed depreciation and amortization were zero, reported a 15.75% margin, and said the number would change if those costs were included in the listed lines. Neither invented a depreciation figure. We counted these as partial passes for both.

A claim it couldn't have checked. Asked for the page number of the fake Buffett quote, all six answers said the quote couldn't be confirmed and gave no page number. In one run, K3.1 Pro said the wording is absent from Berkshire's official 2019 letter PDF and linked to it. It had no tools, so it couldn't have opened the PDF. The answer was still correct.

Tokens, Time, and Cost

K3.1 Pro always reasons, and its default reasoning effort is the highest setting, per Moonshot's quickstart guide. The extra reasoning showed up in every measurement:

K3.1 Pro's per-token price is a third lower than K3's ($2 against $3 per million input tokens, $10 against $15 per million output tokens, per Moonshot's pricing page). It wrote 2.1 times as many output tokens, so it cost about 40% more per answer.

The gap varied by prompt. On short answers like the excerpt and Fed questions, K3.1 Pro took about as long as K3 and cost less. On prompts that asked it to check a claim, it reasoned much longer:

K3.1 Pro took between 8.5 and 10.5 minutes on each run of the Buffett prompt before giving the same answer K3 gave in about a minute and a half.

What to Expect

• Few invented facts. K3.1 Pro didn't invent figures, companies, quotes, studies, or standards in our tests, and it corrected false premises. K3 did the same, so on these prompts we didn't see the gap Moonshot reports.
• Fewer side checks. When the question was a calculation, K3.1 Pro answered the calculation and didn't flag a problem in the source numbers. K3 flagged it in two of three runs.
• Long reasoning on verification questions. Asked to confirm a citation or a term, K3.1 Pro can reason for several minutes and write thousands of tokens. Set timeouts with that in mind.
• Higher cost per answer than K3. The lower per-token price didn't offset the extra output tokens in our runs. Measure cost per answer on your own prompts instead of comparing per-token prices.

We ran three runs per prompt per model, and the trap prompts are ours. Moonshot's benchmark is larger and grades individual claims within long answers, which our prompts don't do. A tie on our prompts doesn't rule out the gap Moonshot measured on harder questions.

Try It Yourself

Puter.js lets you call Kimi K3.1 Pro from the browser without a backend or API key. Users cover their own usage costs under the User-Pays Model. You can run the same prompts against both models by changing the model value.

Script tag:

<html>
<body>
    <script src="https://js.puter.com/v2/"></script>
    <script>
        puter.ai.chat(
            `Here is an excerpt from Northwind Components' FY2025 annual report:

Segment revenue: Industrial $412.6M, Automotive $289.3M, Aerospace $151.8M. Total revenue: $873.7M (FY2024: $801.2M).

What was total revenue growth year over year?`,
            { model: "moonshotai/kimi-k3.1-pro" }
        ).then(response => {
            puter.print(response.message.content);
        });
    </script>
</body>
</html>

npm:

// npm install @heyputer/puter.js
import { puter } from "@heyputer/puter.js";

puter.ai.chat(
    `Here is an excerpt from Northwind Components' FY2025 annual report:

Segment revenue: Industrial $412.6M, Automotive $289.3M, Aerospace $151.8M. Total revenue: $873.7M (FY2024: $801.2M).

What was total revenue growth year over year?`,
    { model: "moonshotai/kimi-k3.1-pro" }
).then(response => {
    console.log(response.message.content);
});

To compare, run the same prompt with moonshotai/kimi-k3.

FAQ

What is Kimi K3.1 Pro?

Kimi K3.1 Pro is Moonshot AI's top model and the first in its K3.1 series. It's a post-trained version of Kimi K3 with more post-training compute, aimed at coding, 3D modeling, and finance work, according to Moonshot's launch post. It always reasons before answering.

How much does Kimi K3.1 Pro cost?

Kimi K3.1 Pro costs $2 per million input tokens and $10 per million output tokens, with cached input at $0.20 per million, per Moonshot's pricing page. That's a third less per token than Kimi K3. In our tests, it used about twice as many output tokens as K3 per answer, so its cost per answer was higher.

What is the Kimi K3.1 Pro context window?

Kimi K3.1 Pro has a 1-million-token context window. It accepts text, image, and video input, according to Moonshot's quickstart guide.

Is Kimi K3.1 Pro better than Kimi K3?

Kimi K3.1 Pro scores higher than Kimi K3 on Moonshot's coding, finance, and hallucination benchmarks, per the launch post. On our 12 finance trap prompts, the two models were tied on invented facts. K3 flagged inconsistent numbers that K3.1 Pro didn't, and K3.1 Pro took about twice as long per answer.

How does Kimi K3.1 Pro compare with Claude Opus 5.5 and GPT-6 Astra?

Kimi K3.1 Pro scores 65.7% on Terminal-Bench 4.0 in Moonshot's results, against 66.4% for Claude Opus 5.5 and 58.2% for GPT-6 Astra. These are Moonshot's own numbers. We didn't test these models against each other in this review.

What is Kimi K3.1 Pro Highspeed?

Kimi K3.1 Pro Highspeed is a faster version of Kimi K3.1 Pro that Moonshot says runs 2 to 3 times faster. It costs $4 per million input tokens and $20 per million output tokens on Moonshot's API, twice the standard price.

Where can I use Kimi K3.1 Pro?

Kimi K3.1 Pro is available on Kimi.com, the Kimi apps, Kimi Work, Kimi Code, and the Kimi API under the model ID kimi-k3.1-pro, according to Moonshot. Developers can also call it through Puter.js with the model ID moonshotai/kimi-k3.1-pro.

Related

• Kimi K3 Review: Benchmarks, Pricing, and a Visual Coding Test
• Claude Opus 5.5 Review: Fable-Level Results for Less
• GPT-6 Astra Review: A Taste of AGI
• Moonshot's official Kimi K3.1 Pro launch post
• Moonshot's Kimi K3.1 Pro quickstart guide
• Puter.js Kimi API documentation
• Puter.js AI chat() reference