Critical Thinking for Humans and Bots

You do not need to win the argument. You need to make a better decision.

That applies to the human in the meeting and the bot producing the briefing. A polished answer is useful. A polished answer that is wrong is still wrong, just considerably easier to circulate.

I would start with a small habit: before accepting the conclusion, ask what is carrying it. What is the evidence? What has been assumed? What would change the answer?

This is the first of two practical lessons. Work through one invented business example, then try the same questions on something you actually need to decide. Allow about 25 minutes as a suggested learning slot, not a promise of mastery. The second lesson turns the method into a checking brief for a bot.

What critical thinking actually means

It does not mean finding something negative to say. It does not mean memorising enough Latin to make the other person uncomfortable.

It means being careful about what you believe, why you believe it and what you are going to do about it.

The American Philosophical Association's Delphi work identified six core skills: interpretation, analysis, evaluation, inference, explanation and self-regulation. Those categories are reproduced in Appendix A of Abrami and colleagues' research paper. They are a useful human-learning framework, not a test proving that a bot thinks like a person.

For our purposes, the distinction is simple. Critical thinking checks the claim. Good judgement decides what to do with it, given the uncertainty, consequences and choices available. That is my working distinction for these lessons, not a new scientific definition.

You might have good evidence that a tool saves time and still decide not to buy it because the information it needs is too sensitive. The facts and the decision are connected. They are not the same thing.

Five things to keep separate

When you read a proposal or ask a bot for advice, sort its contents into these buckets.

Notice the hidden journey from "this happened" to "therefore buy it". That is where I would spend my attention.

A source can accurately report its own pilot without establishing what will happen in your business. Equally, a commercial source is not automatically wrong. Inspect the method and the limits, not just the logo.

A six-question check

This is my suggested working checklist, drawing on the research, not a validated assessment or certification.

1. What exactly are we deciding?

"Is AI useful?" is rather a large job for a Tuesday morning.

"Should our ten-person team trial this tool on one weekly admin task?" gives you something you can investigate.

Write down the task, people, time period and outcome that matter. Agree what would count as unacceptable, too. A faster answer with important details missing is not automatically an improvement.

2. What is the evidence, and where did it come from?

Open the source. Do not stop at the headline or the bot's summary. Find the passage, table or record that supports the particular claim.

Look for the date, sample, comparison, units and exclusions. Was it a measurement, an estimate, a testimonial or a sales promise? Do three articles provide three independent observations, or all repeat the same announcement?

If you cannot inspect the underlying evidence, label that limitation. "The supplier reports" is different from "we have independently verified".

3. What have we quietly assumed?

Ask what must be true for the result to apply here. Similar tasks? Similar staff? The same quality standard? No extra review work?

Put those assumptions next to the conclusion. Do not bury them in the appendix and then circulate the exciting bit.

4. What else could explain it?

Perhaps the tool helped. Perhaps the pilot used unusually experienced volunteers. Perhaps work moved to someone else, or quality was checked less thoroughly.

These are hypotheses to investigate, not accusations. The point is to find evidence that distinguishes explanations, not invent objections forever.

5. How sure should we be, and what would change our minds?

Be specific about the uncertainty. You can be confident in a calculation while remaining unsure about the inputs.

Ask for reasons and evidence, rather than a decorative confidence percentage. A bot saying "87% confident" is not a measured reliability rate unless someone has actually validated that use of the number.

Decide in advance what result would make you continue, change course or stop. For a low-risk trial that might be a quality threshold and an observed net time saving. For a consequential decision, get appropriate domain expertise and a stronger review.

6. What is the smallest sensible next step?

Sometimes that is an approved trial. Sometimes it is obtaining one missing document. Sometimes the sensible answer is no.

Record the decision, the person authorised to make it, and when to review it. Good analysis does not grant permission to send an email, change a system or spend the money.

Worked example: the impressive 40% saving

Here is an entirely invented scenario. These figures are arithmetic for learning, not a product benchmark.

A supplier says its tool cuts a particular admin task's time by 40%. Our team has ten people. Each has a 40-hour week and spends two hours on that task.

Before reading on, write your own answer: does that mean the team needs 40% fewer people?

That is a possible gross task-time saving equivalent to 2% of the team's scheduled hours, not evidence for removing 40% of the team. It is before setup, checking, rework and other overheads. The supplier's result has not yet been shown to apply to us.

Even a genuine saving does not automatically become cash. You would still need to decide whether that capacity improves service, clears a backlog or reduces a cost that can actually be avoided.

My next step would be a small, authorised comparison: similar tasks, the same quality checks, and all relevant time counted. Agree the success and stop criteria first. Do not cherry-pick the fastest examples afterwards.

There is also a related language trap. 40% more output per hour is not the same statement as 40% less time per unit. At unchanged output and quality, the first implies time per unit of 1 divided by 1.4: about 71.4% of the original, or about 28.6% less. Always ask what the percentage is a percentage of.

Ready-to-copy: check a claim beside me

Try this with an approved assistant, using invented or non-sensitive material. This is a request for an inspectable answer, not hidden internal reasoning.

You are not asking it to agree with you. You are not asking it to disagree for sport. You are asking it to help test the case.

Your exercise: move the method to a different problem

Use another invented claim: "Our newsletter's clicks rose after we changed the subject line, so the new wording caused the improvement."

Before asking the bot, list two other explanations and one comparison you would want. Then compare your answer with its answer.

Answer check: the audience, list size, sending time or offer might also have changed. A comparable, properly randomised subject-line test would give stronger evidence about that change than this simple before-and-after observation. One comparison would not establish a universal rule for every audience or future campaign.

Now choose a low-risk question from your own work. Can you recognise the same gap between an observation and a causal claim without being told where it is? For company material, check the assistant's approval and data-sharing rules first.

Practise judgement, not just the vocabulary

Abrami and colleagues' 2015 meta-analysis found that critical-thinking instruction can help, with dialogue, real or situated problems and mentoring among the useful features. That is evidence about education, not proof that these two short guides improve a bot's reasoning. Research and scope.

Nor would I turn this into a rule that slow thinking is always good and intuition always bad. Kahneman and Klein's 2009 analysis connects skilled intuition with learnable patterns and adequate practice and feedback. Feeling certain is not, by itself, evidence of expertise.

For a bot, an articulate explanation also needs checking. NIST's 2024 Generative AI Profile, section 2.2, describes erroneous answers supported by erroneous explanations or citations. It is voluntary US guidance, not a certification of a particular product.

My suggestion is to keep a small decision record: the question, evidence, assumptions, expectation, owner and review date. When the result arrives, compare it with what you wrote before you knew the outcome. Discuss a mistake with someone who understands the work. Try the method again in a different setting.

The point is not to make every choice take all afternoon. It is to notice when an attractive answer is doing more work than the evidence can carry.

Next: Teach Your Bot to Check Its Work. For the related discussion about performance versus evidence, read Winning the Debate Is Not the Same as Being Right.

Sources and limits

• Abrami and colleagues, 2015: the accessible abstract and Appendix A support the teaching summary and reproduced Delphi categories. The separately hosted original Delphi PDF was inaccessible during this check; it is not presented as independently read.
• Kahneman and Klein, 2009: original journal paper hosted by a university; a human-expertise synthesis, not an AI evaluation.
• NIST AI 600-1, July 2024: sections 2.2 and MEASURE 2.5 support the distinction between plausible output and verified evidence.
• The checklist, prompts, learning slots and examples are proposed teaching tools. They are not validated tests, professional qualifications or guarantees of reliable judgement. No real supplier, staffing decision or newsletter experiment is represented here.