Mistaking What AI Knows for What I Understand

An explanation can feel clear while it is in front of us. Its sentences follow one another, the examples seem reasonable, and the conclusion arrives neatly. Then someone asks us to explain the same idea without looking, and a space opens where our confidence had been.

That space is not a reason for shame. It is useful information. In an age when AI can supply explanations almost continuously, learning to recognise it may be one of the most important things we learn.

The boundary becomes harder to see

In the first part, I distinguished receiving an answer from learning through it. The difficulty is that these experiences can feel remarkably similar. When the answer is articulate and responsive, we may feel as though we have participated in the reasoning more fully than we have.

Suppose a learner asks why a particular policy affects prices. AI offers a plausible explanation with familiar examples. The learner can repeat the conclusion, but when asked what would happen under a different assumption, they no longer know where to begin. This is a hypothetical situation, but it illustrates a very ordinary distinction between recognising a claim and understanding its structure.

The issue is not simply whether the answer is correct. An accurate explanation can remain external to the learner. Equally, a learner may understand the reasoning in an answer that contains an error. We need to examine reliability and understanding separately, because improving one does not automatically improve the other.

We sometimes know less than we think we know

Rozenblit and Keil's research on the illusion of explanatory depth offers a helpful lens. In their 2002 study, people rated their understanding of how things worked, then attempted detailed explanations and related tasks. Having to explain exposed gaps, and participants revised their estimates of their understanding downwards.

The study predates today's generative AI and does not establish that AI causes this illusion. What it gives us is a reminder: confidence in understanding can be greater than our ability to produce an explanation. The feeling is human, not a defect belonging to a particular generation of learners.

I find this a gentler starting point than accusing people of pretending to know. Often, we have mistaken familiarity for understanding without realising it. We recognise the vocabulary, remember the conclusion, and assume the connections between them are available to us as well.

AI's fluency gives us another reason to check that assumption. We can encounter an explanation so well organised that its order temporarily carries us. The useful question is what remains available when we need to choose the next step ourselves.

Learning to notice our own uncertainty

Metacognition refers to thinking about and monitoring our own thinking. In learning, that includes noticing what we understand, choosing an approach, and checking whether the approach is working. It sounds abstract until we translate it into a small question: which part could I explain, and which part am I merely repeating?

We do not need a precise score for every piece of knowledge. We need enough awareness to decide whether to continue, practise, ask for help, or verify a claim. “I can follow this example, but I cannot yet solve another one” is a much more useful description than “I am bad at this.”

A learning environment should make such distinctions safe to express. If uncertainty is treated as an embarrassment, learners have an incentive to conceal it. If uncertainty is treated as a starting point, it becomes something a teacher or a tool can help address.

This also matters in professional work. Saying that I can prepare a first draft but need someone qualified to check the legal interpretation is an act of responsibility. It locates the boundary of my knowledge instead of hiding it behind a polished document.

Let AI help reveal the gap

AI can be useful here if we give it a role beyond answering. We might ask it to present a small problem, listen to our explanation, and identify a step that needs clarification. Or we might ask for two apparently similar examples and try to explain why the same method does not apply to both.

Consider studying compound interest. Instead of requesting another summary, a learner could explain why the timing of a deposit matters, then apply the idea to a changed example. AI can help generate practice material and offer feedback, while a reliable textbook or teacher remains important when the feedback itself is uncertain.

The sequence matters. Allowing the learner to attempt something before receiving the model answer creates an opportunity to see their own reasoning. If the full solution arrives first, they may unknowingly build their response around it and mistake that support for independent understanding.

None of this makes the chatbot an infallible examiner. It can miss a mistake or confidently praise a weak explanation. Its feedback is an aid to reflection, not final certification that we have learned. For consequential subjects, a qualified human or dependable source may still be necessary.

Understanding does not require refusing assistance

It would be unreasonable to insist that genuine learning means doing everything unaided. We learn with books, demonstrations, conversation, calculators, and many other forms of support. AI can join that collection without turning every assisted achievement into something suspect.

The question is what the support allows us to do next. Does it help us identify a principle, make a connection, or improve a decision? Or does it repeatedly perform the very thinking we were hoping to practise? The answer can differ according to the learner and the purpose.

For an unfamiliar subject, a simplified explanation may be exactly the right first step. As understanding develops, it can help to revisit the simplification and ask what was left out. Support can change with the learner, rather than remain fixed at either complete assistance or complete independence.

I would like us to become more comfortable with provisional statements: I understand this part; I am unsure about that assumption; I can use the method here but have not tested it elsewhere. Such language makes collaboration more dependable because other people can see where help is needed.

A boundary we can redraw together

The boundary between knowing and not knowing is rarely permanent. It changes as we practise, receive feedback, and encounter situations that challenge an earlier explanation. Recognising the boundary is therefore not an admission that learning has failed. It is part of learning's progress.

AI can help us redraw it, but someone still needs the time, confidence, and support to engage in that process. Two people can use the same tool and have very different opportunities to discover what they do not yet understand. That difference leads to a larger educational question.

Next: “The Same AI Does Not Mean the Same Opportunity to Learn,” available on 20 October 2026 at 9:00 AM KST. If the tool is available to everyone, why might the benefits of learning still be distributed unequally?

Research note: Rozenblit and Keil, “The misunderstood limits of folk science: an illusion of explanatory depth,” Cognitive Science (2002). This research concerns self-estimates of explanatory understanding, not the effects of generative AI. https://pmc.ncbi.nlm.nih.gov/articles/PMC3062901/