Back to case studies
AI/MLProduct StrategyPricingRAGTeardown

Paying by the run — how Notion AI moved to metered pricing

Aug 2026

Notion moved AI to pay-per-run before it made AI reliable — so users now pay for its retrieval failures. A pricing teardown built from public sources

Paying by the run — how Notion AI moved to metered pricing

Why I looked at this

I use Notion daily, so I noticed the pricing changing around me rather than reading about it. In the space of a year, the same feature set was sold three different ways: as an add-on you could attach to any plan, then as something bundled into a higher tier, then as a meter you top up.

Most write-ups about that treat it as a pricing story — is it worth $20 a month or not. I think the more interesting question is what happens when you put a meter on a system that doesn't succeed every time. That's the teardown.

What changed, and when

Notion retired its standalone AI add-on in May 2025 and folded full AI into the Business plan. Agents arrived with Notion 3.0 that September. Custom Agents shipped in February 2026 with a free two-month beta, and on 4 May 2026 that beta ended — agents now consume Notion credits, sold at $10 per 1,000, pooled across the workspace, reset monthly with no roll-over.

Commercially it worked. Notion has reported passing $600M in ARR with more than half tied to AI-enabled customers, an $11B valuation at a January 2026 employee tender, and over a million Custom Agents created during the beta. This isn't a piece arguing the strategy failed.

The thesis

Notion moved AI to pay-per-run before it made AI reliable — so users now pay for its retrieval failures.

The chain is short:

  1. Retrieval is probabilistic. A workspace assistant does not answer correctly every time; some share of runs miss content that is genuinely there.

  2. A miss invites a retry. The natural human response to a bad answer is to ask again — which is a second billable run.

  3. So the meter charges for error. The cost of unreliability moved off Notion's margin and onto the customer's invoice.

Under the old flat add-on, a failed retrieval cost Notion compute. The vendor absorbed its own error rate. Under credits, the customer does.

The evidence

There are two accounts of how well this product works, and they don't agree.

On one side, Notion's launch materials carry a customer report of 95%+ ticket triage accuracy, 25%+ of tickets resolved autonomously, and around 20 hours saved per week. That's a real result, but it's a single customer's number published by the vendor.

On the other, public reviews and community posts from late 2025 through mid 2026 keep returning to four themes: answers that miss content sitting in the workspace, stale or expired links cited as sources, quality degrading on large workspaces, and credit burn that's hard to predict. That's a qualitative sample, not a representative one, and I make no frequency claims from it.

I'm not adjudicating between them. The point is that a buyer pricing this product only ever sees the first column.

The strongest thing I found

It's in Notion's own documentation. In the section advising admins how to control credit spend, the help centre tells them to use Insights to spot runs that take extra steps, fail and retry, or use a more expensive model than the job requires.

That's the vendor telling customers that failed-and-retried runs are a cost line they should go hunting for. I didn't need to prove that failures are billable — the docs assume it.

The user-facing consequence is worse than the accounting one: a failed run and a good run look identical until somebody opens a dashboard. There's no signal at the point of use.

What the meter actually costs

Notion publishes per-run estimates. A Q&A agent runs about $0.03–0.11; a daily brief agent about $0.10–0.30. So the meter isn't flat across the product — a daily brief can cost ten times a Q&A run. Credits are pooled at workspace level regardless of who triggered the agent, don't roll over, and agents pause automatically when the balance hits zero.

Who absorbs it

The individual user didn't get a worse price. They got a variable one.

Before May 2025, any plan could add AI for a predictable per-seat fee that didn't move with usage. Today, meaningful AI requires the Business plan, Custom Agents draw metered credits on top of that, and Free and Plus users get a capped trial allowance that ends in a wall.

A team that automates more work doesn't just pay more — it pays an amount it can't forecast at the start of the month.

What I'd ship

Don't bill for ungrounded runs. If a run returns no answer grounded in workspace content, it shouldn't consume credits. Retrieval failure is the vendor's problem to price, not the customer's to absorb.

Show cost before the run, not after. An estimate at trigger time turns a surprise into a decision.

Publish the fair use limit. When credits run out, other AI features continue "up to the fair use limit." That limit isn't defined anywhere in the documentation I could find.

Give Plus a metered option. A capped trial that ends in a wall converts fewer people than a small paid meter would.

What I couldn't do, and what I'd do next

This teardown is built entirely on public sources. I ran no controlled tests, and the deck says so on the title slide.

There are three things I'd need to answer properly, and none of them are visible from outside: the real failure rate of retrieval, what share of credit spend goes to retries rather than first attempts, and whether retrieval quality varies with workspace size the way users report.

With a Business seat and a few hours, I'd run a black-box test to get at the first one. Plant facts with invented identifiers in a workspace page, ask about them, remove them, ask again, and record whether the system admits the gap or answers anyway. Nonsense identifiers matter — if the answer is correct, it can only have come from retrieval, not from the model's training. Repeat each probe several times, because variance is the finding as much as the average.

I'd also time how long an edit takes to reach the index, and look for the case where a citation points at a page whose current text no longer supports the claim. That would be the sharpest artifact available: proof that a citation attests to retrieval, not to grounding.

Sources

Primary

  • Notion blog — "Introducing Custom Agents," 24 February 2026

  • Notion Help Center — "Buy & track Notion credits for Custom Agents"

  • Notion Help Center — Enterprise Search

Secondary

  • Reworked — Custom Agents general availability, May 2026

  • Published 2026 product reviews and public community posts