Doing Without Asking Why
Why that's not great for science, and how to overcome it
SIGNAL
A recent analysis in Nature reveals that between 2012 and 2022, the proportion of scientific papers engaging with AI across 20 fields has quadrupled, testament to the rapid adoption of AI-driven modeling in research. Yet, this surge comes with a warning: without clear protocols to catch errors, an overreliance on AI modeling risks doing more harm than good, potentially undermining core epistemic values of science like scrutiny, understanding, and replication.
Meanwhile, Yale News highlights a deeper worry: AI might enable scientists to “produce more while understanding less.” Anthropologist Lisa Messeri and cognitive scientist M. J. Crockett caution that AI could narrow the breadth of questions posed, experiments undertaken, and perspectives considered. And so, falling into an “illusion of understanding” where productivity masks ignorance.
STORY
Let’s consider a real‑world scenario, turning abstract concerns into a concrete narrative.
In a mid‑2024 episode, a peer‑reviewed journal published a paper filled with grotesquely wrong AI‑generated figures: a rat with impossibly large and anatomically bizarre features; images annotated with gibberish like iollotte sserotgomar. The absurdity dubbed “the rat‑penis paper” exploded across social media and expert circles.
Why did this catastrophe happen? Because the authors used generative AI to generate visuals without asking why. They needed figures, AI offered them; they accepted them. The scientific process collapsed into how-to-produce, leaving hypotheses, reasoning, and validation (the why) abandoned.
This mirrors a broader inversion: in traditional natural sciences, inquiry begins with why: why is the sky blue, why does the particle behave thus. Technological devices—microscopes, particle accelerators, digital tools—emerged as byproducts to support the why, either to reproduce how nature works, look where the human eye couldn’t see or to validate hypotheses.
But with modern AI, that pathway is flipped. AI delivers results (the how) without grounding them in understanding. It becomes a seductive black box: fast, scalable, but hollow.
This dynamic played out again in several fields. Two researchers demonstrated the use of AI in fabricating 288 complete fake academic finance papers that predict stock returns. Their purpose was demonstrative, but the ease with which AI can generate plausible yet fraudulent science underscores how doing without asking why enables deception as much as discovery.
These real stories illustrate the peril: AI, untethered from why, may produce impressive outputs, yet hollow. It inverts the scientific chain of meaning by introducing technology before understanding, process before purpose.
THE HUMAN OVERRIDE
To reclaim the primacy of why and keep science and technology balanced, here’s a pragmatic, zero‑BS framework.
1. Ask “Why?” first, before “How?”
Always start with hypotheses, motivations, and purpose. AI must serve the why and never replace it.
Action 1.1: Mandate that every AI-assisted study begin with a clear statement of the research question and rationale.
Action 1.2: In peer review, require explicit documentation of how AI tools contributed and to clarify their purpose.
2. Embed epistemic humility
Recognise that AI models are fallible and often opaque. Guard against illusions of understanding.
Action 2.1: Introduce evaluation protocols that test AI outputs against foundational scientific reasoning.
Action 2.2: Regularly audit AI outputs for hallucinations or fabrication, especially in figures or data.
3. Maintain transparency and replicability
AI’s black‑box nature must be countered with traceability.
Action 3.1: Require complete log disclosures (prompt, model version, date) for AI-generated content.
Action 3.2: Store raw outputs, reasoning steps, and human edits in appendices or repositories for replication.
4. Preserve methodological pluralism
Resist monocultures of AI-only approaches. Keep diverse ways of knowing alive.
Action 4.1: Encourage studies that combine AI-driven and traditional methods—celebrating both paths.
Action 4.2: Journals and funding bodies to require justification for AI use compared to classical approaches.
5. Install oversight and accountability
Ensure AI tools don’t undermine scientific integrity.
Action 5.1: Establish oversight committees (e.g., “AI Ethics and Reliability Board”) in research institutions.
Action 5.2: Utilize tools such as SciGuard, a proposed system designed to mitigate AI misuse in scientific domains.
6. Foster continual critical discourse
Build shared epistemic vocabularies to discuss AI risks.
Action 6.1: Host regular workshops where scientists interrogate AI’s blind spots together.
Action 6.2: Publish accessible guidance (e.g., from NAS or similarly influential bodies) laying out norms for AI in science.
This framework centers the human: the questioner, the validator, the skeptic. (We shall start talking about ‘skeptic-in-the-loop’!)
AI takes care of scaffolding, we’re the architect.
SPARK
Provocative question: If tomorrow’s labs are run by AI assistants, guided solely by “how” not “why,” what becomes of human curiosity, moral imagination, and the beauty of reflection that science offers? Do we risk building a future of knowledge production that is efficient but soulless? Well… we could argue, without the why, there’s no knowledge at all, right?
Further reading:
Control Risk for Potential Misuse of Artificial Intelligence in Science
Realizing the Promise and Minimizing the Perils of AI for Science and the Scientific Community
Explore policy side: European Commission, Artificial intelligence in science – Promises or perils for creativity
Humans First, Center, and Last: Don’t let the tools choose the problem (the first contributed piece on Honest AI!)
P.S. If you’re feeling overwhelmed by the AI noise, its claimed capabilities, and the plethora of tools available, or if you're struggling to make AI work for your specific context, I can help.
I offer limited consulting slots to executives and professionals seeking to transition from AI confusion to discernible results. No
generic frameworks. Just personalized roadmaps that match your skills, job-to-be-done, and timeline.
Only 1 slot in August and 2 in September remaining. They fill fast.
Book a discovery call - takes 2 minutes.