In their paper Combining Psychology with Artificial Intelligence: What could possibly go wrong?, cognitive scientists Iris van Rooij and Olivia Guest explore what happens when AI systems are treated as if they think like people. They examine how psychological research changes when these systems are used not just to mimic behaviour, but to explain it and what that shift reveals about the assumptions shaping both fields.
Their argument matters because it’s becoming easy to assume that if a system talks, writes, or predicts like a person, it must understand like one too. This paper unpacks why that assumption is flawed, and what it reveals about the kinds of reasoning science is beginning to accept.
Why the fusion of psychology and AI is epistemically dangerous
The fusion of psychology and AI, when approached without careful consideration, can disrupt our understanding of knowledge: how we formulate questions, construct theories, and determine what constitutes an explanation.
The authors contend that this convergence can lead to errors that are more insidious than straightforward methodological mistakes. The core issue lies in how each field defines understanding and what types of outputs they consider as evidence. When these standards become blurred or diminished, distinguishing between a theory and a mere placeholder, or between a tool and the subject it is intended to study, becomes increasingly challenging.
Psychology's research habits lower the bar for explanation
Psychology, particularly in its mainstream experimental form, has been grappling with inherent structural weaknesses. The replication crisis is the most visible symptom, but there are deeper issues influencing research practices:
Hyperempiricism refers to the tendency to prioritise data collection and the identification of statistically significant effects, often at the expense of developing robust theories. The mere presence of an effect is often considered informative, even without an accompanying explanation.
Theory-light science describes a trend where researchers focus on how individuals perform specific tasks, without considering whether these tasks genuinely reflect broader cognitive capacities. The emphasis is on measurable outcomes rather than explanatory depth.
Statistical proceduralism reflects the field's inclination to address crises by implementing stricter protocols and enhancing statistical rigour, rather than pursuing conceptual reform. Practices such as pre-registration and replication enhance methodological rigour but fail to tackle fundamental questions about what constitutes a meaningful theory.
These tendencies render the field vulnerable to what the authors term an "epistemic shortcut" - a shift in how knowledge claims are justified. Rather than developing and testing theoretical assumptions, researchers may start to treat system outputs as inherently explanatory. Consequently, if an AI system produces behaviour resembling human responses, it might be mistakenly viewed as a substitute for genuine cognition, even if the underlying mechanisms remain unexplored..
AI imports assumptions that favour performance over understanding
AI introduces its own assumptions that influence its approach to understanding, often rooted in engineering where success is measured by performance rather than explanation:
Makeism suggests that building something is key to understanding it. In practice, if an AI system replicates a behaviour, it's often assumed that the behaviour is explained. However, replication doesn't confirm the same underlying process.
Treating high-performing models as if they reveal the mechanisms behind behaviours is a common mistake. Even if a system performs well, it may not capture the essence of the phenomenon it mimics.
Performance metrics like benchmark results and predictive accuracy are frequently equated with scientific insight. High-scoring models are often deemed valid, even if their success is unclear or irrelevant to cognitive theory.
Hype cycles exacerbate these issues: commercial and reputational incentives encourage overstatement, making it easy to overlook constraints like computational intractability or multiple realisability, where different systems produce similar outputs differently.
These factors foster a reasoning pattern where systems with superficially human-like behaviours are assumed to be cognitively equivalent to humans, often without examining the assumptions behind this equivalence.
What goes wrong when these patterns reinforce each other
When psychology and AI are brought together without challenging these habits, their weaknesses can amplify each other. This can result in a number of epistemic errors:
Category errors, where AI systems are treated as if they are minds or cognitive agents.
Success-to-truth inferences, where good performance is taken as evidence that a system is cognitively plausible.
Theory-laundering, where the outputs of machine learning systems are framed as if they reveal new insights about the mind, even when no theoretical commitments are made or tested.
Automated epistemology, where theory-building is quietly outsourced to systems that were not designed for that purpose, and are not capable of the kinds of reasoning theory-building requires.
The consequence is that explanation becomes harder to evaluate. Systems that simulate thought are mistaken for systems that explain it. Outputs are mistaken for insight and the role of human judgement in making sense of complex phenomena is quietly displaced.
A slower, more theory-led approach to using AI in science
The authors advocate for a thoughtful integration of psychology and AI, rejecting a superficial merger that overlooks each field's limitations. They suggest using AI as a tool for developing and testing theories, not as a means to bypass understanding.
Central to this approach is recognizing computational intractability: problems that, although theoretically solvable, are practically infeasible due to excessive time or resource demands. Many cognitive tasks like planning and reasoning fall into this category and become too complex to compute reliably.
When AI systems efficiently handle such tasks, they might be simplifying the problem or using shortcuts, which makes them useful tools but not replicas of human cognition. This perspective emphasizes human-led theory-building, supported by formal tools without being reduced to them.
The authors term this approach "non-makeist", viewing AI as an aid to explore the boundaries of cognitive understanding rather than a method to recreate minds. It involves careful progress, improved questioning, and skepticism toward systems that seem to over-explain.
The three big traps
1. AI systems are not minds
This trap revolves around a category error: confusing behavioural mimicry with mental equivalence. Systems like large language models generate outputs that resemble human responses, but resemblance does not equate to equivalence. These systems lack essential features of human cognition, such as intentionality, understanding, flexible generalisation, and embodied experience. Yet, many researchers and commentators discuss AI systems as if they possess "intelligence," "theory of mind," or even "cognitive abilities."
Here's why this is a mistake:
Domain-general cognition is computationally intractable: Human cognition involves more than pattern matching; it includes flexibly navigating entirely new situations, integrating context, and grounding concepts in lived experience. There is no known feasible method to construct such a system using machine learning.
AI systems are decoys: they imitate human responses without replicating the underlying mechanisms. Their plausibility is performative, not structural, making them compelling yet potentially misleading.
Replacing human participants with AI systems is a scientific error. While using AI models instead of human subjects may appear cost-effective or innovative, it circumvents the very phenomenon psychology aims to study: people.
This trap fosters a fundamental misunderstanding of the nature of minds. It blurs the distinction between external performance and internal processes, risking the development of a behavioural science that no longer understands behaviour as inherently human.
2. AI systems are not theories
Even if we avoid anthropomorphising AI, can we at least consider it a tool for theorising about the mind? The authors caution that we should proceed with extreme care. There's a growing assumption that if a model accurately predicts human data, it provides a valid explanation of human cognition. However, predictive performance does not equate to explanatory adequacy.
Key distinctions:
Prediction ≠ Explanation: A model can accurately forecast an outcome without explaining why or how it occurs. This is similar to using a tide table: it predicts tides but doesn't explain the underlying forces.
Correlation ≠ Cognition: When internal parameters of an AI system align with brain data, it's tempting to infer mechanistic similarity. However, this overlooks multiple realisability—the concept that different systems can produce the same outputs through varied processes.
Tasks ≠ Capacities: Excelling at a specific task (e.g., classifying images or solving puzzles) doesn't mean a model captures the broader cognitive capacity (e.g., vision, reasoning). Tasks are often narrow and designed for tractability, whereas capacities are holistic and generalisable.
Scaling Fails: Even if a model succeeds in individual tasks, it likely cannot scale to the open-ended, context-rich, compositional nature of real-world cognition. The limitations here are formal, not just practical.
This trap encourages what might be termed epistemic laundering: using the superficial success of AI models to mask the absence of genuine understanding. It turns theory into a retrospective narrative applied to high-performing tools, rather than a generative framework grounded in psychological plausibility.
3. Cognitive science cannot be automated
The third trap pertains to the conduct of science itself. In the wake of the replication crisis, psychology has become highly procedural, heavily reliant on statistical rigour and data-driven methodologies. Into this environment steps AI, offering the alluring promise of further automation.
Some researchers now view machine learning as a substitute for crucial aspects of scientific work, such as theory generation, hypothesis testing, or insight discovery. This reflects a belief that cognition—and science itself—can be formalised, optimised, and outsourced. Here's why this approach fails:
There is no algorithmic shortcut to developing good theory. The space of possible explanatory models is too vast, and no efficient search strategy guarantees success.
P-values and predictive models cannot capture the conceptual framing, explanatory scope, or counterfactual reasoning necessary for robust theory-building.
When tools provide seemingly coherent outputs without requiring conceptual clarity, the capacity for deep theoretical reflection diminishes and we lose the ability to recognise what we don't know.
As AI-generated outputs inundate scientific discourse, there's a growing risk of epistemic artefacts and pseudoscience by automation - outputs that appear insightful but are structurally hollow, misleadingly consistent, or misaligned with scientific goals.
Science is fundamentally a cognitive activity, so if we outsource too much of its interpretive labour, we risk losing the very capacities that make science generative: creativity, doubt, conceptual reframing, and deep explanation. This trap, therefore, isn't just about AI - it's about the human cost of mistaking automation for understanding.
The way forward
Although the paper is highly critical of how psychology and AI are often integrated, it does not advocate for their separation. Instead, the authors propose a different kind of relationship where AI is treated as a tool for thinking, not as a substitute for the mind itself—an approach they refer to as non-makeism.
In this perspective, AI can support theory-building but should not be treated as if it captures how cognition actually works. It can be used to formalise ideas, clarify assumptions, or test the internal logic of a theory. For example, a simulation might demonstrate what happens when a certain process is scaled up or whether two explanations lead to different predictions. These methods help explore the structure of a theory but do not replace the need to ask how people think, decide, or make sense of the world.
This approach also requires acknowledging that some cognitive processes are too complex to model directly. In computer science terms, they are intractable—even if a system can describe them, it cannot execute them within realistic time or resource limits. If an AI system appears to solve these problems easily, it is likely doing something different, and we need to understand what has been simplified or omitted.
Finally, the authors argue that progress in this area will not be rapid. The goal is not to automate theory but to improve it. This means slowing down, asking better questions, and being clear about the difference between a useful output and a genuine explanation. Used in this way, AI becomes a tool for clearer thinking, not a shortcut to understanding.
Final thoughts
Bringing AI into psychology isn’t the problem; the concern is how quickly we start treating outputs as if they carry meaning, and tools as if they think. AI can support theory-building, but it can’t replace the need for conceptual clarity or interpretive judgement.
Predictive performance isn’t the same as explanation. Systems that produce plausible results often do so without telling us anything about how cognition works, and that distinction is easy to overlook when the output feels familiar.
Theories give structure to what we’re trying to explain and AI can certainly us help formalise and test them, but it can’t define what matters or decide what counts as understanding.
Scientific understanding fundamentally still depends on human judgement - on us noticing the gaps, asking better questions, and staying with uncertainty long enough to make sense of it. When too much of that work is delegated, the reflective habits that drive theory risk falling away.
This argument fits closely with a recurring theme in Generative Friction: that when systems remove too much resistance, they also remove the need to think carefully. Slowness, in science as in everyday cognition, isn’t always a problem to fix - sometimes it’s the condition for insight.
About the authors:
Iris van Rooij is a professor of computational cognitive science at Radboud University and principal investigator at the Donders Institute. Her work focuses on the formal limits of cognition and the role of theory in psychological science.
Olivia Guest is an assistant professor at the Donders Institute, known for her work on computational modelling, conceptual clarity in cognitive science, and critical perspectives on AI. Together, they write and collaborate at the intersection of psychology, philosophy, and theoretical computer science.
van Rooij, I., & Guest, O. (2025, May 14). Combining Psychology with Artificial Intelligence: What could possibly go wrong? (pre-print)
Share this post