AI cost is now the second-largest controllable spend at most AI-first startups I work with, after payroll. It is also the one founders look at least often, because for most of 2023 and 2024 it was small enough to ignore and the tooling to break it apart was underbuilt. That has changed. At seed stage in 2026, the model bill is a first-class engineering problem, and the levers to control it are small in number and mostly boring.

The good news: none of them requires a dedicated ML platform team, a fine-tune, or self-hosted inference. Below is the shortlist I keep coming back to, ordered by effort-to-impact, along with the ones I explicitly tell founders to skip until later.

Why AI Cost Became a First-Class Problem

Three things happened at once. Retrieval-heavy and agent-heavy patterns became the default shape of AI features, so a single user action can trigger five to fifty model calls. Context windows got large enough that teams stopped worrying about token counts, so system prompts and pasted context grew without anyone noticing. And the price gap between the cheapest and most expensive frontier models widened, so where you route a request now matters more than which provider you picked.

The net effect at seed stage is that a modest number of active users can produce a monthly model bill that is small enough to not trigger investor questions and large enough to eat a meaningful slice of runway. The instinct is to wait until the number is scary and then panic-optimize. That works badly, because by the time it is scary, the shape of the product depends on the unoptimized behavior.

The Levers That Actually Move the Number

1. Prompt caching (first, always)

Both Anthropic and OpenAI now support prompt caching in ways that are easy to enable and hard to misuse. For any flow where the same system prompt, retrieved context, or tool schema is sent across multiple requests within a short window (which is most agent flows, most chat flows, and most multi-turn retrieval flows), turning caching on typically cuts input token cost by fifty to ninety percent for one afternoon of engineering.

The reason to do this first is that it is nearly free, the savings are large, and it does not change product behavior. Every other lever below moves the product in some way. This one moves only the bill.

2. Model routing (one weekend)

Almost every AI product mixes easy queries and hard queries. Routing every request to the top-of-the-line model is like sending every customer email to a partner-track lawyer. Some of them are worth it. Most are not.

A simple router (a lightweight classifier, or even a rules table on request shape) that sends easy queries to a cheaper model, and only escalates on ambiguity or when the cheap model’s confidence is low, is one weekend of engineering and typically thirty to sixty percent off the total bill without any measurable quality drop. The trap is over-engineering the router itself. Start with three tiers: cheap, mid, expensive. Ship. Measure.

3. Prompt hygiene (free)

Most AI features accumulate context the way legacy codebases accumulate config: nobody wants to be the one who deletes something. System prompts grow to five thousand tokens. Tool schemas include fields nobody uses. Retrieved context is pasted in full when a summary would work.

Once a quarter, someone should read every system prompt and tool schema in the product with a red pen. There is no framework here. It is a chore. The savings compound the same way trimming a bloated dependency list does: nothing dramatic on any single request, an obvious reduction across millions of them.

4. Batch APIs for non-realtime work

If a job does not need to answer in the current user session (nightly enrichment, backfills, eval runs, offline analysis), it should almost always run through a batch API. Both major providers charge roughly half price for batch work. One code path in your inference layer to switch between sync and batch pays for itself immediately for anything running in the background.

The specific mistake I see: teams put their eval runs on the sync API because “it feels faster.” Evals are the archetypal batch workload. Move them.

5. Retry and cascade hygiene

Two failure modes that quietly eat budget. First, unbounded retries on transient errors: a bad request that would fail regardless gets retried three times, each one billed. Second, cascading calls where an agent that runs into an ambiguous state calls the model again to “figure out what to do,” which produces another ambiguous state, and so on.

Both are cheap to fix. Bounded retries with exponential backoff for the first. A hard cap on chained model calls per user action for the second, plus a metric on how often the cap trips. The cap tripping is a bug signal, not just a budget signal.

6. Cache the non-LLM layer too

Retrieval results, tool outputs, and external API responses are often re-fetched request after request for the same session or user. A simple in-memory or Redis cache on the outputs of your retrieval and tool layer often saves as much as prompt caching on the model layer, because it removes the model call entirely rather than making it cheaper.

The One Dashboard to Build First

Cost per active user per week, split by feature. That is it. Not “total model spend.” Not “tokens per minute.” Cost per active user per week, split by feature.

This one number, when you have it, changes how every roadmap conversation goes. A feature that costs twelve cents per active user per week is a feature you scale to more users freely. A feature that costs four dollars per active user per week is a feature that is either premium-tier only or gets a hard efficiency pass before the next launch. Nobody has this argument well without the number, and everybody has it well once they do.

The mechanics are simple: log every model call with user id, feature tag, model, input tokens, output tokens. Roll up weekly. Divide by weekly active users per feature. Chart it. This is a two-day project and it changes how the team thinks about every AI feature it ships from that point forward.

A Diagnostic If Your AI Bill Is Already Uncomfortable

Before you optimize, find out which of these three shapes you have.

Shape one: a small number of features are almost all of the bill. Common. Fix by routing and caching those features, ignore the rest.

Shape two: a small number of users are almost all of the bill. Very common for agent-heavy products. Fix by capping chained calls per user action, then thinking about pricing.

Shape three: the bill is roughly uniform across features and users. Rare and usually indicates the whole product is over-modeled: everything is being answered by the most expensive model when most of it did not need to be. Fix with global model routing.

Do not start optimizing without knowing which shape you have. The wrong lever for the wrong shape is a week of engineering with a rounding-error savings.

What NOT to Spend Engineering Time On Yet

These are all real levers. They are just wrong for seed stage.

  • Fine-tuning. Not until you have a stable, high-volume pattern that a smaller model actually cannot handle with prompting alone. Almost every seed-stage fine-tune I have seen was a two-week detour that would have been better spent on prompt caching and routing.
  • Self-hosted inference. Great lever at real scale. At seed stage, GPU rental plus ops attention costs more than the model bill you were trying to reduce. Ignore until you are sure your workload is high enough to justify a full-time person’s attention on inference.
  • Chasing every new “cheaper” open model. Switching cost eats the savings. Wait until an open model is unambiguously better for your workload, not just marginally cheaper per token.
  • Building your own eval harness from scratch. Use one of the existing ones. Spend the engineering time on the actual evals.
  • Custom quantization. If you can pronounce it, you are early to it. Skip.

 

What Changes at Series A

Once volume grows, the levers change shape. Negotiated commit pricing with your provider becomes real. Custom fine-tunes may finally justify the engineering time. Dedicated inference infrastructure for a specific high-volume feature can pencil out. A part-time or full-time ML platform person becomes worth their salary. Batch inference at scale starts to matter more than sync inference.

None of that is what a seed-stage team should be doing now. All of it is what a team that did the seed-stage levers well will be able to do cleanly at the next stage. Teams that skipped the seed-stage levers arrive at Series A with a model bill that is scary, a codebase that assumes always-on frontier-model calls, and no cost-per-feature dashboard, which makes the Series A conversation about “our unit economics” the worst version of that conversation.

The Short Version

Turn on prompt caching this week. Ship a three-tier router this month. Have someone read every system prompt with a red pen this quarter. Build the one dashboard. Skip everything else on the list above until the numbers say otherwise.

Let’s Talk

If you are staring at an AI bill that is starting to feel like it might become a runway problem, or you are trying to figure out which of these levers to reach for first, that is exactly the kind of decision I help work through with founders. No pitch. Happy to look at your usage shape and tell you what I would do. Reach out.