When A Proof Of Concept Is Really Research

I was at the OpenAI Build Week Community Builders Lounge in London today. I went there to build. I came away thinking about how we price uncertainty.

The room at Encode, near Old Street, was full of people working in very different ways. Some were just starting with Codex. Some were committed to Linux. One person was working with eight Mac minis. People compared what they were building, where the tools were helping, and where they were still getting stuck.

It felt adventurous.

OpenAI describes Build Week as a global week for exploring what is possible with Codex, supported by community events and a build challenge. That was exactly how the London room felt: not a polished industry pretending it had reached the answer, but a group of people discovering the answer together.

Tony Wood at the OpenAI Build Week Community Builders Lounge in London, with other builders working behind him. — At the OpenAI Build Week Community Builders Lounge in London. I went to work on Codex and came away thinking about how we commission work when the reliable pattern does not exist yet.

I had just completed a proof of concept, alongside the continuing work of building the Chief Agentic Officer business. Sitting in that room, something became very clear to me.

When we are building genuinely new agentic systems, we are often not implementing a known solution. We are doing research.

The estimate I would have got wrong

If a client had asked me at the beginning to create the Chief Agentic Officer briefing business, I think I would have massively underestimated the work.

Not because the visible output is impossibly complicated. A briefing is a briefing. A podcast is a podcast. A website is a website.

The difficulty sits underneath.

How do the agents gain access to the right information? How do they know what they are allowed to do? How do they hand work to one another? How do I see what is happening? What happens when a publishing platform has no useful API? How do we preserve evidence, recover from failure, control costs, and keep a human judgement point where it matters?

There was no complete, proven pattern I could simply install.

Making the work visible has turned out to be a constant effort. Each working part reveals the next question. Each blocked route teaches us something about the system we actually need.

That changes how the work should be commissioned.

Implementation or research?

There is a useful test.

Can a competent team point to somewhere this has already been done under similar conditions? Can they describe the architecture, dependencies, controls, delivery route, and likely failure modes? Can they reasonably say, "We will use this model, in this structure, and it will work"?

If they can, the work is largely implementation. It may still be difficult, but the path is known well enough to estimate, plan, and price.

If they cannot, the situation is different.

UK government guidance on research and development uses a surprisingly practical definition of technological uncertainty. It exists where whether something is possible, or how to make it work in practice, is not readily available or readily deducible by a competent professional. It can also exist where the challenge is how to combine known components into a system that achieves the intended result.

That is close to what I am seeing in agentic proofs of concept.

I am not saying that every AI pilot qualifies as formal R&D or for tax relief. That is a specific legal and accounting question. I am saying the boundary gives us a useful way to think.

Known implementation	Research-like proof of concept
A comparable pattern exists.	The system combination is materially new.
The route can be described before work begins.	The route must be discovered through experiments.
Unknowns are mainly delivery risks.	Whether or how it will work is itself uncertain.
Scope can be priced around a known outcome.	Work should be bounded around hypotheses, evidence, and learning.

It feels like the early browser years

I am old enough to remember the early commercial web.

Netscape Navigator 1.0 arrived in December 1994. During 1995 and 1996, browsers acquired tables, frames, plug-ins, JavaScript and competing scripting models. HTML 2.0 was published as an RFC in November 1995. HTTP/1.0 followed in May 1996. Microsoft announced JScript and VBScript support through ActiveX Scripting for Internet Explorer 3 in July 1996.

The detail matters because it corrects my memory slightly. HTML was the markup language; HTTP was the protocol. But the feeling was right: the standards, browser capabilities, commercial players, and accepted ways of building were moving underneath us.

A solution designed around one browser generation could feel old remarkably quickly.

Agentic work feels like that again.

Models are improving. Harnesses are changing. Tool access is changing. Long-running agents are becoming more capable. The way we give an agent identity, memory, permissions, evidence, and a place to work is still being invented in public.

Good engineering principles have not disappeared. We still need clear outcomes, sound system design, security, resilience, observability, backups, tests, and honest governance.

What is changing quickly is the way those parts are assembled into an agentic organisation.

Decision diagram distinguishing a repeatable implementation path from a research-like proof-of-concept loop based on whether a competent team can identify a reliable route. — If the route is known, implement it. If the route is part of the unknown, commission bounded research sprints that produce evidence and a decision.

Do not fixed-price an unknown route

A fixed price makes sense when the route to the outcome is understood.

It becomes dangerous when the route is the thing being discovered.

The supplier is encouraged to pretend the uncertainty is smaller than it is. The client is encouraged to treat every learning as a failure to deliver the original scope. Both sides start protecting their position instead of examining the evidence.

I think the better model is a partnership built around bounded research sprints.

Each sprint should be clear about:

what we currently believe;
which uncertainty matters most;
what we will build or test;
what evidence would support or reject the idea;
which systems, data, and people are in bounds;
which stop-lines must not be crossed;
what we learned;
what becomes safe and sensible to do next.

You can still fix the time, cost, team, and decision date for a sprint. What you should not do is guarantee an outcome that no competent person yet knows how to produce.

That is not an excuse for endless experimentation. Research needs more discipline, not less. The OECD's Frascati guidance describes R&D as novel, creative, uncertain, systematic, and transferable or reproducible. In practical terms: write down the question, run the test, preserve the evidence, and make the learning useful beyond the person who happened to discover it.

A proof of concept should prove something

We sometimes treat a proof of concept as a small version of the finished product.

I think that is often the wrong mental model.

A good proof of concept should retire one important uncertainty.

Can the agent find the right evidence? Can it maintain state across a long task? Can it work inside the permission boundary? Can the output be reviewed quickly enough? Can failures be made visible? Can a second agent safely continue the work? Can the operating cost support the business model?

A POC that answers one of those questions honestly may be more valuable than a beautiful demo that quietly avoids all of them.

The output is not only software.

It is evidence, a decision, a better map, and sometimes the confidence to stop.

The client has to be part of the learning

If I am working with a company on something genuinely new, I would now start with a very direct conversation.

Can you show me where this has been done before, with comparable data, controls, users, systems, and constraints?

If the answer is yes, excellent. Let us learn from it and price the implementation properly.

If the answer is no, then we should agree that we are entering a discovery partnership. The client cannot stand at the end of the process waiting for the promised machine. Their knowledge, access, decisions, risk appetite, and feedback are part of the experiment.

That relationship needs honesty from both sides.

The builder must not use novelty to avoid accountability. The client must not use procurement language to pretend the uncertainty has disappeared.

Adventure needs operating discipline

There is a kind of faith involved in research.

You have to believe that the idea is worth following before you can prove that it works. You need enough confidence to continue when the route is not obvious.

But it should not be blind faith.

It is faith disciplined by evidence, review points, stop-lines, records, and the willingness to change your mind.

That was the thought I brought home from the Build Week room in London.

We are all learning how to build this new layer of work. Some people are doing it with a laptop. Some with Linux. Some with an extraordinary number of Mac minis. The variety is part of the fun.

I commend OpenAI for helping those communities find one another. It was lovely to spend time with people who were willing to share what they knew and equally willing to admit what they were still discovering.

There is a real sense of adventure.

Just remember to price the map when drawing the map is part of the work.