$ ctrlaltroot
Field noteZürich

Open vs. closed models is a procurement argument wearing a technical costume

An evening of founders and cloud partners debating how the next generation of startups gets built. My position afterwards — not a report of what was said.

event: Open vs. Closed Models · VentureLust × Snowflake × AWS × DataMax

The question on the invitation was the one every AI-native founder has been arguing about for two years: build on open models or closed ones, and what does that mean for how the next generation of startups gets built?

What I want to write down here is my position after the conversation — not a summary of what was said in the room. Panels blur; the argument that survives the walk home is the useful artefact.

The framing is usually wrong

The debate is normally staged as capability: which model is better at the task. That question has a half-life of about four months, which makes it a poor basis for an architecture decision you’ll live with for three years.

The question that actually decides it is: what do you need to own, and what can you rent?

Closed models let you rent the frontier. You get capability you could never build, immediately, and your team can spend its limited attention on the product instead of the model. The cost is that you do not own the upgrade curve, the pricing, the deprecation schedule or the data path — and in a regulated market, the data path is often the deal-breaker.

Open models let you own the inference. You get control of cost at volume, deployment inside a customer’s perimeter, and the ability to fine-tune on a domain. The cost is that “owning” it means owning an operations problem — serving, capacity, evaluation, upgrades — which is precisely the work most AI-native teams underestimate by a factor of two.

Where the real cost sits

Nobody in the argument ever costs the migration. A hybrid architecture is not a compromise; it’s a second system, and it’s where I’ve seen more startups lose a quarter than anywhere else. If you route between models, you now need evaluation good enough to prove that the cheap path is acceptable for each class of request — and if you don’t have that, you have a cost optimisation that occasionally produces a wrong answer with no way to notice.

That’s the part I keep coming back to: the model choice is downstream of the eval capability. Teams with a real regression suite can move between open and closed, or route per-request, or downshift to a smaller model for the easy 80%, because they can prove they didn’t break anything. Teams without one are locked into whatever they chose first, by fear.

What it means for anyone selling into these companies

Three practical implications, and they’re the reason I go to these evenings:

  1. Nobody is loyal to a model, but everybody is loyal to an eval suite. If you’re a platform or infrastructure partner, the durable relationship is the one that makes switching safe — not the one that makes switching unnecessary.
  2. The buyer is increasingly the platform team, not the ML team. At the AI-native companies I meet, the person who owns the deployment decision is someone with a Kubernetes and CI background who inherited inference. That is a very specific kind of buyer, and most partner programmes still address a different one.
  3. Sovereignty is a Swiss sales motion, not a technical one. Open models win deals here for reasons that have nothing to do with benchmarks: the data stays in the perimeter, the regulator is satisfied, the audit is simple. If your product can’t answer that question, the capability argument never gets heard.

My own position, for what it’s worth: for an AI-native company in year one, rent the frontier and spend everything on evals. In year two, once you know which 80% of your traffic is boring, own that part. The mistake is doing it in the other order — optimising cost before you can measure quality, which is how you end up cheaper and worse with no way to tell.

← all notes · subscribe via RSS