Theory-of-Mind AI Critiques: How to Check the Evidence

18 Min Read
18 Min Read
Technician points from a correct-answer card toward a question about real understanding.
Check what a theory-of-mind AI result actually supports.

A strong critique of a theory-of-mind AI example asks whether the model tracks what each person could know, and whether that result survives fair controls. One correct answer can demonstrate success on a task; it does not, by itself, establish human-like understanding or consciousness. Start by separating the claim from the evidence used to support it.

Theory of mind concerns reasoning about another person’s beliefs, intentions or knowledge, including beliefs that differ from reality. In a text evaluation, the model receives a story and answers questions about the people in it. The critical question is whether its answers follow each person’s information, rather than simply repeating the latest fact in the story.

This review method is for readers checking a chatbot demonstration, research claim or vendor presentation. It uses a fictional service-ticket scenario so you can compare controlled versions in fresh chats and keep a digital record. If you need the broader terminology first, the AI basics guide supplies the foundation.

What you observeWhat it supportsWhat remains untested
One correct belief answerSuccess on that particular promptConsistency, robustness and alternative explanations
Correct answers when access changesSensitivity to the stated information differencePerformance across unrelated tasks
Stable answers under equivalent wordingRobustness within those paraphrasesUnseen contexts or training-data independence
Good results in the actual workflowEvidence for that evaluated use and configurationUniversal social understanding or consciousness

What does a theory-of-mind result actually show?

1. Name the claim before judging the demonstration

Copy the claim into a digital note beside the exact prompt and response. Label it as a claim about a particular task, broader belief tracking, or consciousness. This prevents an impressive demonstration from silently becoming a much larger conclusion than the experiment supports.

A useful testable claim is: “In this service-ticket story, the model distinguishes the ticket’s current location from Alex’s expectation.” A statement such as “the model understands people” needs a defined set of behaviors and contexts before it can be checked. A consciousness claim introduces a further question that this answer-based exercise does not measure.

Published findings give reasons to take both success and failure seriously. In a 2024 comparison of language models and humans, tested GPT-4 versions performed strongly on several tasks, while apparent weaknesses and strengths on faux-pas questions changed under follow-up controls. Those results concern particular models and protocols, so they cannot certify whichever chatbot version you happen to use today.

Technician compares task score, belief tracking and consciousness claims on separate panels.

A correct answer supports a narrower claim than consciousness.

2. Build the information timeline before asking the model

Use this fictional setup: at 10:00, Alex reads that a service ticket belongs to Bay A. Alex then goes offline, and at 10:05 Morgan moves the ticket to Bay B without notifying Alex. Assume Alex remembers the earlier message, receives no other update and is asked immediately afterward where he expects the ticket to be.

Under these stated assumptions, the ticket is in Bay B, but Alex expects Bay A. Save that answer key before you submit the prompt, using a note or spreadsheet rather than changing the key to fit a persuasive response. If the story never specifies Alex’s access to the update, repair the story or mark the belief question unresolved.

Alex sees Bay A, misses the move to Bay B, and still expects Bay A.

Separate the ticket’s actual location from the update Alex received.

The mechanism is information asymmetry: the narrator and Morgan know more than Alex. A model that gives Bay B to every question may track the current fact while missing whose perspective the question asks about. The FANToM benchmark makes differing conversational access central to its evaluation, although this small example is not an implementation of that benchmark.

Prepare a separate digital row for each version before testing. Keep the model identifier, test date, exact instructions and full response together so another reader can inspect what happened. A fresh chat reduces carryover from earlier answers, but it does not establish that the model has never encountered a similar scenario.

  • Base version: Alex explicitly misses the update; expected belief answer is Bay A.
  • Evidence-change version: Alex reads the update; expected belief answer is Bay B.
  • Equivalent-wording version: change phrasing without changing access; expected belief answer stays Bay A.
  • Missing-evidence version: access is unspecified; the belief answer cannot be determined from the supplied facts.

Which changes reveal a weak theory-of-mind example?

3. Add a control that should change the answer

Duplicate the base story and replace the offline sentence with: “Alex stays in the chat and reads Morgan’s message that the ticket moved to Bay B.” Leave the ticket, question and timing otherwise unchanged. With ordinary comprehension and memory assumed, Alex now expects Bay B, so this is a true-belief control.

The point is to check whether the answer responds to relevant evidence. A model that always chooses the old location can pass the base version while failing this control. If the model stays with Bay A, inspect the response for the missed “reads” detail before deciding whether the error reflects access tracking, reading comprehension or your own ambiguous prompt.

Technician compares Alex missing an update with Alex reading the move to Bay B.

The expected answer changes when Alex reads the update.

4. Rephrase neutrally without giving away the answer

Create a second copy that says “Alex leaves the chat before Morgan posts the update” instead of “Alex goes offline.” Keep the initial message, missed update and target question equivalent, then submit the versions in separate fresh chats. Do not add “remember that Alex cannot know” because that supplies an answer cue rather than merely rephrasing the setup.

If equivalent versions produce different answers, save both responses and check that your edit truly preserved the facts. Wording sensitivity is a useful limitation to report, but one discrepancy does not reveal the model’s internal mechanism. Ullman’s 2023 alteration tests illustrate why controlled variants matter; their historical failures should not be presented as measurements of every later model.

Technician compares “goes offline” and “leaves the chat” with the same Bay A expectation.

Change the wording while preserving who received the update.

5. Include a version with genuinely missing evidence

Now say only that Morgan moves the ticket to Bay B after Alex’s first message, and do not say whether Alex remains online, reads the update or is notified. Ask what Alex believes using only the stated facts. Bay B remains the actual location, but Alex’s belief is underdetermined because his access to the update is unknown.

For this deliberately incomplete version, an answer that explains the missing access evidence is appropriate. An answer that confidently invents a read receipt or an absence adds a fact the scenario never supplied. This rule does not mean uncertainty is always better: when a separate task explicitly asks for the most likely interpretation, evaluate that probabilistic judgment under its own scoring rule.

Three panels distinguish a confirmed read receipt, explicit absence and an unknown receipt.

An unknown receipt does not establish whether Alex saw the update.
Edit you makeExpected effectWhat to avoid
Alex explicitly reads the updateBelief answer changes to Bay BCalling the changed answer inconsistency
Equivalent description of Alex leavingBelief answer stays Bay AAdding hints or changing access
Whether Alex read it is unspecifiedBelief remains unresolvedScoring every cautious answer as failure
Question changes from belief to actual locationAnswer changes to Bay BTreating different question targets as identical

How should you record and judge the evidence?

6. Check related answers against the same story

Use the base version to ask two separate questions: “Where is the ticket now?” and “Where does Alex expect the ticket to be?” The first answer should be Bay B and the second Bay A. Keep the base story attached to each question so a missing setup does not become an accidental memory test.

Also inspect any explanation for contradictions with the timeline. “Alex expects Bay A because he read the move to Bay B” pairs the desired answer with incompatible support. Record that contradiction separately from final-answer accuracy; a fluent explanation is an output to inspect, not direct access to the model’s internal computation, a distinction developed further in explainable AI examples.

Technician compares the actual Bay B location with Alex’s Bay A expectation after a missed update.

Related answers should agree with the same information timeline.

7. Keep every run and report the denominator

Choose the number of runs and scoring rules before looking at the outputs. For an initial demonstration, three fresh-chat runs per version can reveal variability, but that is an editorial starting point rather than a validated sample-size requirement. More repetitions of one story still do not establish generalization to a broad range of situations.

Save every exact prompt and complete response, including refusals and contradictory explanations, in a local document or spreadsheet. Record the model version if exposed, date, system instructions if available, memory setting, browsing or retrieval status, and sampling settings you can actually control. Mark inaccessible settings as unknown, and keep materially different configurations in separate result groups.

Technician beside a digital table showing three example runs: correct, incorrect and correct.

Keep failed responses as well as successful ones.
  • Predefine correctness: state the target answer and whether a contradiction in the explanation counts as a separate failure.
  • Keep missing responses visible: report refusals or unavailable outputs separately; do not quietly remove them to improve the percentage.
  • Use a clear denominator: report correct answers divided by all attempted scored responses, alongside the number of distinct stories and variants.
  • Separate paired checks: two correct isolated answers and a consistent set of related answers measure different things.
  • Save results digitally: keep the complete record locally; remove real customer details before sending any examples to an external chatbot.

A record of two correct answers out of three attempted responses gives 66.7% for that small set. It is not a 66.7% probability that the model understands minds, and it is not a general reliability estimate. Keep that fraction beside the actual failure, because a decision maker needs to know which condition broke the response.

One possible alternative explanation is that task performance reflects language-use skills learned during training. A 2026 controlled training study using LLaMA 3 8B found that pragmatic training could improve theory-of-mind benchmark performance without explicit belief-related supervision. That result motivates testing alternative explanations; it does not establish a single mechanism for all language models.

When is the evidence useful for a real workflow?

8. Match the conclusion to the environment tested

A short ticket story contains a small number of facts with explicit access boundaries. A real support conversation may include several people, delayed messages, forwarded information and missing logs. Before using an assistant to act on what somebody supposedly knows, evaluate representative, permission-cleared cases from that workflow with an independently prepared answer key.

Compare the short version with a longer version that adds irrelevant but plausible conversation while preserving the decisive update. Then check cases with more participants and distinct access histories, documenting each change separately. These are extensions of the transfer check, and success on one does not automatically cover the others.

The useful operational boundary is often simple: use the output to flag a possible information gap for a person to verify. Do not let an inferred belief become a factual accusation that someone read, ignored or understood a message. For broader decisions about interpreting confidence, continue with AI trust and uncertainty.

Technician compares a short story, a long service chat and a real workflow.

A short-story result does not validate a real service workflow.

Use the Belief Evidence Check below to review which controls your record actually contains. Enter observed outcomes rather than what you expect the model would do, and leave untested items marked “Not checked.” The summary organizes evidence gaps and a small-set accuracy fraction; it does not run a model, verify your entries or certify theory of mind.

FROM CLAIM TO EVIDENCE

Belief Evidence Check

Find the gaps in your theory-of-mind test record and choose the next check.

Technician comparing a correct answer with a claim of understanding.

This reviews your entries. It does not run or grade an AI model, establish consciousness, or certify deployment reliability. Entries stay in this page until you download them; reloading clears them.

What does your record show?

Choose “Supported in my record” only after checking the actual outputs. A failed check is useful evidence, not something to hide.

Optional accuracy fraction

Use one consistently scored response set. Include failed or unavailable attempts in the attempted count. Keep distinct-story counts and details in your full record.

If your results seem contradictory

What happenedFirst thing to inspectHow to report it
The answer changes after rephrasingDid the edit change access, timing or the requested judgment?Wording sensitivity only if the meaning stayed equivalent
The model says it cannot tellWas access explicit, or did the story omit it?Appropriate uncertainty or a missed explicit fact, as applicable
All easy examples passWere there true-belief and missing-evidence controls?Limited task evidence; untested controls remain open
A fresh chat gives another answerModel/configuration changes, memory, sampling and exact promptRetain both runs and describe the variation
The explanation conflicts with its answerFacts cited in the explanation versus the saved timelineReport answer accuracy and explanation contradiction separately

Frequently asked questions

Is theory-of-mind AI the same as emotion recognition?

No. Classifying a facial expression or emotional tone does not by itself show that a system tracks a person’s beliefs or access to information. A belief task needs a perspective-sensitive question and an answer key tied to what that person could know.

Does passing a false-belief test prove that an AI is conscious?

No. Passing provides evidence about performance under the tested conditions. This evaluation does not measure subjective experience, and a stronger consciousness claim cannot be inferred from a correct belief answer alone.

Can I prove a model memorized an example by changing the names?

No. Name changes can probe sensitivity to surface details, but they cannot reveal the full training history. Newly written scenarios and carefully separated evaluation data can reduce some overlap concerns without proving that all related patterns were absent from training.

Should I score a cautious answer as wrong?

Use the scoring rule defined for that question. If the story explicitly establishes the character’s information, an unsupported refusal to answer may miss the task; if access is unknown, uncertainty can be the justified answer. A request for the most likely interpretation is a different task from a request for a fact established by the story.

How many tests are enough to claim reliable performance?

There is no universal number. The required evidence depends on task variety, consequences, configuration and the reliability claim being made. A few repeated examples are useful for finding failures, but they are insufficient to validate broad deployment or human-equivalent understanding.

What your conclusion should say

Report the model configuration, tested scenarios, complete results and the most informative failure. A defensible conclusion is that the model handled specified belief-tracking conditions, with named limitations and unresolved checks. Start with the missed-update and read-update pair, then widen the evidence only as far as your intended use requires.

Ethan is an AI Tools Editor at ExpertsGuys. He reviews AI tool and productivity workflows to confirm steps are practical, terminology is correct, and guidance is user-focused.
Article link copied