AI Product Development
Calling a model is easy. Shipping an AI capability your customers rely on is an engineering problem — evaluation, cost per call path, fallbacks, data boundaries, and an experience designed around output that is sometimes wrong.
- Who
- Delivered by a senior team assembled for the engagement, against a defined scope.
What you're seeing
- The AI feature launched and support volume rose with it.
- Usually means Real users found the inputs the build was never tested against. There was no graded set covering the awkward cases, so the first systematic quality signal is arriving through the support queue.
- Nobody can say what the AI feature costs per active user.
- Usually means Inference is priced per request and nothing else in the stack behaves that way, so it does not show up as a capacity problem. It shows up on the bill, usually two quarters after the design decisions that caused it.
- AI is on the roadmap and no one owns the architecture.
- Usually means It is being treated as a feature rather than as a subsystem with its own evaluation, cost model and failure behaviour. That works until the second feature, at which point everything gets rebuilt.
- An enterprise security review has stalled the release.
- Usually means The data boundary was decided implicitly by whoever picked the provider. Answering it now means either re-architecting or explaining to a customer why you cannot answer, and both are more expensive than deciding it at design time.
- Usage of the AI feature is not instrumented separately.
- Usually means There is no way to tell whether people use it twice and stop, which is the normal shape for AI features that do not work. Aggregate product analytics will not surface it because the numbers are small relative to everything else.
The gap between a demo and a feature
A demo needs the model to respond well to inputs somebody chose. A feature needs it to behave acceptably on inputs nobody anticipated, at a cost that works at scale, with a defined outcome when it is wrong, and with a data story that survives a customer’s security review.
Everything between those two states is engineering, and it is consistently underestimated because the demo is the part that looks like the product.
What gets built alongside the feature
The evaluation set comes first — real inputs with graded outputs, written with the people who know what a good answer looks like. It is what turns a prompt or model change from an argument into a measurement, and it is the single most skipped step.
Then the architecture: which model serves which call path, what context each one needs, where caching sits, what the fallback is when a provider is slow or unavailable, and where the data boundary runs. Then the product design work around probabilistic output — how the interface communicates uncertainty, where a person confirms, and how a user corrects the system in a way you can learn from.
Cost is a design output
Inference spend grows quietly because nothing else in a normal stack is priced per request. There is no connection pool to exhaust and no server to fall over; the bill simply moves.
Three things account for most of it. Context sent that nobody needed, usually because retrieval returns ten documents when two would do. The same answer bought repeatedly because nothing caches at the semantic level. And a frontier model answering questions a smaller model answers identically, which only becomes visible once there is a graded set to compare against.
All three are architectural, and all three are much cheaper to decide before the feature has traffic than after.
Where it sits
This is the product-facing shape within AI & Automation. The distinction from the rest of the category is who uses the result: these features ship to your customers, and everything else on this list serves your own operation.
Where the AI capability is one part of a larger build, it runs inside software development with the evaluation and cost work folded into the same engagement rather than split across two. Where the question is whether the feature is worth building, or whether an existing one is worth keeping, that read is AI consulting and it is a much smaller commitment.
What we will not do
We will not hand over a prototype and describe it as a foundation. The difficult work in this category is entirely in the parts a prototype omits — the graded set, the cost model, the fallback behaviour, the boundary, the instrumentation — and leaving those to the client’s team while calling the engagement complete is the most common way this work is sold.
We will also not build a feature nobody has said how to judge. If there is no answer to what a good output looks like and who decides, the feature cannot be improved after launch and cannot be retired on evidence, which means it will stay shipped and mediocre. Getting that written down is part of discovery, and where it turns out the honest answer is that the capability is not worth adding, that is a useful outcome delivered early.
What this covers
Each of these is a capability with its own page, its own order of work and its own outputs.
| Capability | What it means |
|---|---|
| AI App Development | LLM features that hold up in production. An evaluation set before the feature ships, cost controlled by design, and data boundaries settled before a provider is chosen. |
| Discovery Sprint | A short, bounded piece of work that turns an intention into a scope both sides can hold. Cheaper than the first month of building the wrong thing, and it ends with a decision including the decision not to proceed. |
| Backend & API | Server-side systems and the interfaces other things depend on. Built with the contract written first, because an API is a promise and changing one after it has callers is expensive. |
| Web Development | Web applications that stay fast and usable as they grow. Rendering strategy chosen against what the page actually needs, and accessibility treated as a requirement rather than a later audit. |
| QA and Test Strategy | A test suite people trust and run. Coverage placed where failure is expensive rather than spread evenly, and flakiness treated as a defect rather than as weather. |
How we decide
Unit economics are modelled before general availability
Costs It delays the wide release and sometimes forces a less capable model into the design.
An AI feature has a marginal cost per use that scales with adoption, which inverts the usual software assumption that the next user is nearly free. A feature that is profitable at pilot volume and unprofitable at scale is a discovery best made on a spreadsheet rather than on an invoice, and the levers — context size, model tier, caching — are all architectural.
The data boundary is a product decision, not an infrastructure one
Costs It can rule out the strongest model before anyone has tried it.
Which customer fields may leave your infrastructure, what a provider retains, and whether an EU-resident or self-hosted deployment is required determines which customers can buy the feature. That makes it a commercial constraint that belongs in the product definition, and retrofitting it means carving the feature out of contracts.
The feature is instrumented separately from the rest of the product
Costs More analytics work for a feature that may not survive.
AI features fail quietly. People try them, get an answer that is nearly right, and go back to the old path without complaining. Without per-feature retention, acceptance rates and correction rates, that looks identical to success in aggregate metrics, and the decision to fix or remove never gets made.
Four routes to an AI capability in your product
The trade is between speed, control and how much of the difficult part you end up owning.
| Route | Time to first release | What you own afterwards | When it wins |
|---|---|---|---|
| Build with an embedded team | Weeks, once scope is set | The code, the harness and the operating knowledge | The capability is core to the product and has to keep changing |
| Hire an in-house AI team | Months, dominated by hiring | Everything, including the hiring risk | A long roadmap that justifies permanent capacity |
| An AI feature vendor | Days | An integration and a dependency | A commodity capability that is not your differentiation |
| A prototype from an agency | Days | A demo, and the production work | Validating whether the idea is worth building at all |
The fourth is a legitimate first step and a poor last one. The failure mode is a prototype handed over as though it were a foundation, with evaluation, cost control and failure handling left as an exercise for the client's team.
Is this you?
- A real product with users and an AI roadmap that needs building
- An AI feature shipped and its behaviour under real traffic is unpredictable
- The capability is needed sooner than a specialist team can be hired
- Pre-product companies looking for an AI prototype to raise on
- Research programmes with no product on the other side
- Teams who need the conclusion to be that AI is the answer
How we run it
The wrong-answer path ships with the feature
Before release we specify what happens when the output is wrong — the confidence threshold, the review step for consequential actions, the fallback, and the record of what was generated. These systems fail by being confidently incorrect rather than by being unavailable, and a feature without that path finds its own in front of a customer.
