Humans First, Center, and Last
Don’t let the tools choose the problem
Welcome to Honest AI’s first contributed article!
As Honest AI has grown in trust and readership, we’ve started receiving our first contributor pitches. I’m excited for this, which is precisely the kind of community this publication is built to nurture.
If you’d like to write for Honest AI, you can share your idea here.
It’s 2007.
You’re reading a paperback copy of a book you love: ‘The Naked Ape’ by Desmond Morris. This new edition promises updates.
To your sheer disappointment, not only is this an automatically generated reprint, but the production slops are obvious. A regular reader might think the typos are original, but you can trace them to a systematic mistake in the process. Somewhere along the line, someone chose to skip (at least) one round of professional proofreading. The cutting costs logic was simple: the author was famous and the book a proven seller, so commercial success was low risk. Still, manually typing it in again into a digital system would be labour-intensive. Why retype an out-of-print copy when we could just scan it?
Computers, after all, are 96% to 98% accurate these days.
So they did. They scanned the text and ‘read’ it using Optical Character Recognition (OCR) —essentially the same technology that scans QR codes. Every once in a while, though, the algorithm gives you ‘m’ where ‘rn’ should be, or exclamation marks (!) where the letter ‘l’ or number ‘1’ belongs. It sometimes swaps a colon for a semicolon, a dot for a comma, or vice versa. To be honest, the whole thing looks like a mess.
How is that possible? Can’t the computer ‘think’?
Well, it couldn’t, and it still can’t. The machine won’t check for the meaning of words, syntax, or any sort of coherence between clauses, paragraphs or chapters. It’s not a person, and it doesn’t know any of the rules you think it knows. It simply matches patterns of straight lines, curves, and dots. It doesn’t even know ‘2S’ is not the same as ‘25’. So chapter ‘2S’ is perfectly valid for it. You get the idea.
Lucky for publishers, most readers can fill in gaps and guess words even when letters are scrambled. That’s because they interpret the text’s meaning at a higher level than mere strings of characters, syllables, and sentences. But that doesn’t mean readers should have to do extra work.
After all, they paid for Desmond Morris’s essay ‘The Naked Ape’, not a funny, scrambled version of it.
It would be fair to say that the editors oversold the capabilities of a pattern recognition machine.
Two objections
Fast forward to 2022.
The same overconfidence is playing out again, this time with Large Language Models (LLMs). The heralds of ‘AI’ extended the game.
Fans all over the internet rushed to preach their gospel to the world: soon, not even experts would notice the difference between the product and the imitation. And they'll love the imitation. It takes away the need for all the drudgery. It appeals to our fundamental ability to dream and wish. And this would apply to images, video, sound and text alike!
If you think of it that way, bypassing tedious work is natural from an evolutionary perspective. Our brains are wired to love that: quick rewards, no waiting. But here’s where my objections begin. They fall into two categories:
Moral – First, let’s suppose we ask a Large Language Model for a finished book and we get it. Do we want that output? The answer depends on how much we decide we want machines to exploit that evolutionary loophole into our wishing and dreaming. It is a matter of agency and responsibility. As adults, we learn to delay gratification and usually work to meet some quality standard instead of rushing to the outcome. If we press the AI button, we miss all of the checkpoints, decisions, assessments and revisions a typical job requires. The fact that everyone else is doing it shouldn’t be an acceptable excuse, but that’s what many actors in the text production industry are doing.
Technical – The problem isn’t just taste. In machine translation—long before LLMs—post-editing an automated draft was often harder than starting fresh. Most errors are minor, so users think all is fine. But critical mistakes slip through: the “descend” that should have been “ascend” in aviation manuals. We should be catching such errors by design, not luck.
From OCR to Science
Writing and the printing press gave us a memory more reliable than our own, preserving culture across centuries. But in science, the stakes are higher.
The natural sciences matured by producing not just results but methods: transparent, repeatable steps to test claims against reality. In this analogy, reality is the “original,” and a scientific model is the “copy.” When in doubt, the model—not reality—is wrong. Safeguards like peer review, benchmarking, and transparency evolved to keep bias, partisanship, and sloppy reasoning from winning by force instead of merit.
An LLM, however, offers only the copy—without the process that created it. If we were to trust LLM ‘science,’ we would be abdicating all principles of intellectual rigor, ethics, and best practices required to even start free inquiry into such a delicate kind of knowledge, which constitutes the foundation of industrialised societies.
What good is a replica of scientific knowledge that just ‘sounds’ right?
Isn’t that a hypothesis?
Remember how the proofreader's work was crucial, comparing the original to the copy?
The fact that a layperson, a six-year-old or a sloppy scientist can use an LLM to generate text that looks like valid science only proves that hypotheses (wishing something was true) are easier to generate than to test (finding out whether something is actually true). Scientists can’t afford to look the other way when their tools and institutions are challenged, even if we’re accused of gatekeeping knowledge or worse. Having methodological standards and not trusting that reality bends to our wishes was the whole point, wasn't it?
Say two scientific colleagues are discussing some laboratory result or a theoretical model of a natural phenomenon. It is by meticulously filtering out all of the basic mistakes and clearly specifying as many of the details as they can that they suggest a valuable contribution to the community —usually in the form of a paper. They can (and should) be called out by peers if there’s anything wrong in their work at all.
For that to happen, it is key that the process is reproducible. That way, no matter if the authors’ claims are right or wrong, we get to know why they are right or wrong, and we can test them for ourselves. You get none of that from LLMs, because they’re a black box, so they are opaque to scrutiny.
I contend that now more than ever is the time to stand our ground and judge the merit of scientific ideas and projects, not only on their outcomes, but first and foremost, on the methods section: the how we got there guarantees reproducibility and is therefore indispensable.
If scientists surrender to outcome-only thinking, we abandon the intellectual rigor that makes knowledge trustworthy. We risk replacing inquiry with imitation.
That boring, inconvenient, step-by-step section you often skip to jump to impactful results? It won’t make your clickbaity headlines and it’s plain old news for insiders, but it's the trail of decisions about design, development, and testing that tells the whole story.
The Lesson
Scientific progress depends on the how, not just the what. If you skip the “Methods” section in a paper, you’ll be missing the part where the real value lies. Without it, nothing can be verified, reproduced, or trusted.
I’ll say it in case this needs to be repeated: LLMs (or ‘agents’ based on them) can't make their own decisions, nor can they tell you how they reached conclusions, not even by the infamous chain-of-thought. They only perform the tasks for which they have been given permission. When overwhelmed or in doubt, just remember: humans should be in charge. Trust them to make mistakes and own them.
If you’re considering an AI-first strategy, you might have been gravely misled, and you might want to reconsider: you can pick two of cheap, fast, and good —not all three. You may save time, money and trouble if you ask your team, ‘Do we need automation in the first place?’
Influential figures who actually listen to people’s and organizations' needs will be leading innovation instead of chasing the next sloppy imitation.
__
P.S. Thank you for reading till the end (today’s most valuable skill!). If you found it interesting, please consider sharing it with people in your network
 | A guest post by
|