You Cannot Vibe What You Do Not Know

I had a slightly embarrassing moment this morning.

I shared my article You Are The A In AI in a WhatsApp group. Everybody else's links arrived with a lovely image, title, and description. Mine arrived as a plain link.

I looked at it and thought: what is going on?

The article existed. The hero image existed. The HTML worked. The site loaded. I had spent a lot of time building rules for how my agents create and publish an article.

But I had missed something.

The page did not contain all the metadata a social platform needs to create a rich preview. In particular, it did not properly declare the image, title, description, and canonical URL through the Open Graph protocol.

Once I knew the rule, the fix was straightforward.

The difficult part was knowing that the rule existed.

And that made me realise something important:

You cannot vibe what you do not know.

The code was not the problem

This was not a dramatic software failure. Nothing had crashed. Nobody had lost data. The page looked fine when I opened it in a browser.

That is precisely why the omission survived.

The requirement only became visible when the page moved into another context: a messaging platform trying to understand it. My system had passed the tests I had given it. I had simply not given it the test that mattered in that moment.

AI did not forget a rule I had clearly defined. It built from an incomplete understanding of what good looked like.

That distinction matters.

We often talk as though a frontier model should be able to inspect a task, infer every relevant discipline, recover every hidden requirement, and produce the complete professional result.

Sometimes it can infer an astonishing amount.

But it cannot reliably tell you about every important thing that neither you nor the context has made visible.

The invisible requirement

Every profession contains visible tasks and invisible requirements.

The visible task might be: build a web page.

The invisible requirements include how it behaves on mobile, how a screen reader understands it, how a search engine indexes it, how WhatsApp previews it, how it is cached, how it is secured, how it is monitored, and what happens when somebody changes it six months later.

The IEEE Computer Society's current guide to the software engineering body of knowledge contains 18 knowledge areas. Construction is only one of them. Requirements, architecture, testing, operations, maintenance, quality, security, configuration management, economics, and professional practice all sit around the code.

That is why an expert's AI is likely to be better in their field than a non-expert's AI.

Not because the expert owns a more magical model.

Because the expert knows which questions to ask, which failure modes to expect, which evidence matters, and what good looks like when the work leaves the demonstration and meets the world.

Expertise is partly made of things we no longer notice

Michael Polanyi's work on tacit knowledge is useful here. People can possess practical understanding that is difficult to state completely. Experience changes what we notice. It teaches us which detail feels wrong, which shortcut is dangerous, and which apparently small omission will become expensive later.

A skilled person may not begin a task by reciting every rule they have accumulated. Many of those rules have become part of how they see the work.

When we use an AI system, however, that hidden understanding needs a route into the work. It may enter through good instructions, examples, standards, tests, review, tools, organisational context, or the expert watching the output.

If it enters through none of those routes, the model has to infer it.

Inference is useful. It is not the same thing as assurance.

About that confidence curve

There is a popular chart often attached to the Dunning-Kruger effect. It shows confidence racing up to a grand peak, collapsing into a valley, and slowly recovering as wisdom develops.

It is a memorable picture. It is not the actual finding from the original research.

Justin Kruger and David Dunning's 1999 studies found that people who performed poorly in particular tasks also had difficulty assessing their performance. The underlying idea is relevant: some of the knowledge required to perform well is also required to recognise what good performance looks like.

But we should not turn that into a smug story about stupid beginners. Later researchers have argued that the familiar effect can be inflated by statistical artefacts such as regression to the mean and the better-than-average effect. The size and interpretation of the effect remain debated.

An illustrative confidence curve with an early peak, a dip as complexity becomes visible, and a gradual recovery, accompanied by notes explaining that this is not the original Dunning-Kruger research result. — The familiar confidence curve is a useful illustration, not the original experimental result. The practical warning is that capability can arrive before our judgement is well calibrated.

The practical point is simpler.

When we first become able to make something work, our output improves faster than our ability to see everything that might be wrong with it.

AI can make that gap wider because it gives us borrowed fluency. The application looks polished. The report sounds confident. The code runs. We experience the capability of the model and can easily mistake it for the completeness of our own understanding.

AI may improve the answer while weakening our judgement of it

A recent study gives this an important twist.

In two studies involving 246 and 452 participants, researchers examined people using AI for logical-reasoning questions. AI improved performance, but participants still overestimated how well they had done. In the first study, performance improved by three points against the comparison population while participants overestimated their score by four points.

More strikingly, higher AI literacy was associated with less accurate self-assessment. The usual Dunning-Kruger pattern flattened: overconfidence was not confined to the least skilled.

That feels much closer to the risk we are facing.

The problem is not simply that beginners will build bad things.

The problem is that all of us can use AI to produce something better than we could have produced alone, while becoming less able to judge what is missing.

The model can make us more capable without automatically making us wiser.

Disposable code and shared systems are different things

I am not arguing against vibe coding.

For disposable code, experiments, personal tools, and quick exploration, it is extraordinary. Build the thing. Try the idea. Throw it away if it does not work.

The boundary changes when somebody else relies on it.

A public website has users, search engines, social crawlers, accessibility needs, security concerns, and a history. An internal tool has data, permissions, colleagues, backups, and an owner. A customer system has promises attached to it.

At that point, the work needs more than successful generation. It needs a body of knowledge around it.

NIST's Secure Software Development Framework makes a similar point in a more formal way. Secure development is not one final security check. It requires preparation, protection, secure production, and a way to respond when vulnerabilities are found. The framework is deliberately outcome-based and risk-based rather than a universal checklist.

That is professional work: knowing which body of knowledge applies, then adapting it to the situation in front of you.

Turn every surprise into a rule

The answer is not to wait until you know everything. You never will.

The answer is to build a better learning loop.

State the outcome. What should work, for whom, and in which real setting?
Name the known constraints. Data, security, access, cost, platforms, users, law, culture, language, and operations.
Ask which disciplines are hiding inside the task. A web page may also be an accessibility, identity, search, social, security, and publishing problem.
Bring in standards and expertise. Use the model to find the relevant body of knowledge, then verify it against authoritative sources and experienced people.
Test in the real context. Do not only test whether the page opens. Test whether the actual crawler, user, colleague, or downstream system can use it.
Capture the surprise. Turn the failure into a rule, example, test, or reusable skill.
Run the loop again. Expertise grows through the accumulation of corrected assumptions.

A countryside route diagram showing how an unexpected failure becomes a discovered rule, a test, shared knowledge, and safer future work. — The useful response to an unknown is not embarrassment. It is to convert the surprise into a durable rule and test.

That is what I have done with this small mistake.

We did not just fix one article. We created a public-page standard, added social metadata to every relevant page, generated correctly sized preview images, added structured data and browser identity assets, and built automated checks so the omission cannot quietly return.

The missing knowledge became part of the system.

Your expertise still matters

We have spent decades sharing code, examples, answers, templates, and techniques with each other. That generosity helped create the body of material from which modern models learned.

There is an irony in that. The knowledge we shared to help one another now helps automate parts of the work we used to do.

But a model's broad knowledge does not erase local expertise.

Every company has its own language, customers, permissions, history, risks, and ways of working. Every culture contains meaning that does not fit neatly into a generic instruction. Every serious system eventually meets a circumstance that was not in the demonstration.

So focus on your field. Learn what good looks like. Build your rules. Share examples. Ask the AI what you may have missed. Ask an expert too. Test the output where it will really live.

Vibe boldly when the work is cheap to discard.

Slow down when other people will depend on it.

And when something surprises you, do not pretend you should have known everything.

Learn the rule.

Then make sure your system knows it next time.