Rescuing an AI Product From a Dying Platform in Four Weeks

Details anonymized. The shape of the problem is the point, not the names.

The month-of-runway problem

A founder I work with had built his product's AI engine on a no-code LLM orchestration platform. Good choice at the time: fast to prototype, visual flows, no infra to run. The product drafts compliant, on-voice text for medical practices, so every request can carry PHI and every output is regulated speech.

Then the platform announced it was shutting down. About a month of notice.

Everything that made the product the product lived inside that vendor: the prompt flows, the compliance rules, years of voice engineering. Not exportable to anything runnable. This is the LLM era's version of the agency-built website you can't edit, except the agency is also turning off the servers.

What four weeks bought

I rebuilt the system solo on Azure, part-time, against the vendor's shutoff date. The cutover landed on the deadline with zero downtime, verified by running the old and new systems against the same inputs until the outputs matched.

The rebuild, in rough order of how much it mattered:

The engine. The platform's proprietary flow format turned out to be event documents: 900+ blocks with branch gates and nested subflows. I reverse-engineered the format and reimplemented it as a faithful executor behind a FastAPI service on Azure Container Apps. Faithful matters. The goal of a migration is not "better", it's "provably the same", because the client's team can verify sameness and they cannot verify better. Better comes after.

Compliance as step zero, not a checkbox. De-identification runs before any model sees text, and every model in the fleet is pinned to the US data zone. One bug class was instructive: naive de-id masked names the response legitimately needed, garbling the output. The fix was a context-aware allowlist that keeps exactly the names the request declares and masks everything else. Compliance-grade LLM systems are an architecture problem. The paperwork describes the architecture; it doesn't substitute for it.

Here is that allowlist doing its job, on invented data:

Honest failure contracts. Early on, an unreachable data source caused a model to write apology text into structured fields, which then landed in a database. That's the quiet failure mode of LLM systems: they don't crash, they confabulate. The redesign made failure a first-class result. Unreadable source means empty fields plus a needs_manual_entry status, with a refusal detector and one grounded retry. The system never apologizes into data.

A control room, not a black box. The founder doesn't write code. He now edits every production prompt, picks the model per flow node, and replays real payloads in a web control room. Edits go to a staging store, get tested against a staging environment, then promote to production with history and one-click rollback. Prompt changes went from "file a developer request" to "founder ships it himself." For a non-technical owner, the ops surface is the product.

Boring deployment discipline. One repo, trunk-based. Every build is an immutable image tagged with its commit SHA. New builds hit a UAT environment first; the byte-identical image then promotes to production, with the previous revision parked as an instant rollback. Every promote gets a git tag, so the tag history is the audit trail. And after two supply-chain incidents in one week (a pip resolver meltdown and a transitive minor-version bump that broke the model API), all ~161 dependencies got frozen to a verified set. Both incidents died in staging. Users never saw either.

A real model fleet. Migrating off a platform also meant escaping its model menu. The system now runs the current frontier (multiple GPT lines, DeepSeek, Kimi) with per-node model selection, and the ops panel shows published per-model pricing next to actual month-to-date spend. Cost as a visible dial, not a surprise invoice.

What I'd tell anyone building on an AI platform

1. Ask the exit question before you build. If this vendor died tomorrow, what artifact do you walk away with? If the answer is "a JSON export nothing can run", you are renting your own product.
2. Your prompts are your IP. Treat them like source code. Version them, stage them, promote them, and keep them somewhere you control.
3. Design the failure states first. An LLM system's worst outputs come from its error paths, because that's where models improvise. Make failure structured and boring.
4. Byte-identical promotion beats clever branching. Prod should run the exact artifact staging verified. Tags are cheaper than branches and drift less.
5. One engineer with modern AI leverage covers what used to take a team, but only if the discipline comes along: parity testing, pinned dependencies, rollback ladders. The leverage multiplies the discipline. It doesn't replace it.

Platform risk is not an argument against building on platforms. It's an argument for knowing, on day one, what the rebuild would take. In this case the answer turned out to be four weeks. The founder now owns his stack, ships his own prompt changes, and the next version of the product is being built on rails he can see into.

That's the trade worth making: from renting to owning.