Technical Scale ≠ Social Scale
Why Bigger AI Models Aren't Always Better
SIGNAL
What if the conventional wisdom "more data = better models" is actually a myth?
Or, with better nuance, what if it holds technically, but is fundamentally wrong socially?
Researchers Diaz & Madaio, in their paper Scaling Laws Do Not Scale, dismantle this belief: common performance metrics (like accuracy or BLEU*) don’t reflect universal improvements, but rather the vantage of dominant dataset populations. They argue that as datasets grow, metrics become increasingly blind to diverse or marginalized user groups (Diaz & Madaio, 2024).
This blind faith in scale risks sidelining communities already overlooked. Optimizing models for averaged metrics can mask serious declines in performance or fairness for linguistic, cultural, or demographic minorities.
This critique hits just as tech giants race toward trillion-parameter AI systems and city-sized data centers—a gold‑rush of scale—but the key question is: who is benefitting from this race?
It’s time to break free from equating scale with quality. We need evaluations that are contextual, culturally aware, and human-centric, not a one-size-fits-all metric.
*BLEU is a score that measures how close a machine translation is to a human one.
STORY
When translation tools fail real communities: the Hmong example
A telling report from the South China Morning Post chronicles how automated translation tools, perceived as universally helpful, actually fail linguistic minorities such as the Hmong community in the US during critical civic activities like voter registration (PostMag).
In 2020, Jennifer Xiong, a volunteer with the Hmong Innovating Politics group in Fresno, California, spent her summer mobilising Hmong speakers to register for the U.S. presidential election. Despite her efforts, many community members remained disengaged. It turned out to be a language problem.
Xiong notes the translation tools caused confusion: "This was an entirely new thing for me to see," she reflects, referencing how translations were often vague or inaccurate, failing to convey registration instructions effectively.
Even more recently, MPR News relayed how the Minnesota Department of Human Services published Facebook notices—translated into Hmong and Somali—that were riddled with errors. Hmong speakers described the translations as "entirely too literal," making them virtually unintelligible. The agency later admitted the errors were due to inferior contractor translations and not tools like Google Translate. However, that didn’t mitigate the confusion caused by the flawed communication (MPRnews).
Together, these stories highlight the human consequences behind models that appear “good” on average. The Hmong, despite being one of the largest Southeast Asian communities in the U.S., remain underserved by linguistic technology. What looks like a technical failure here reveals itself more like a civic, cultural, and deeply consequential one.
This aligns directly with Diaz & Madaio’s warning: Aggregate metrics may look fine, but individual communities suffer poor performance or miscommunication. Bigger models won't solve this, and they may never solve it even if we come up with new evaluations that are re‑grounded in community realities.
THE HUMAN OVERRIDE
If scaling laws are breaking down, what can practitioners, policymakers, and executives do? Here is a pragmatic framework—The Four R’s of Responsible Evaluation—to move beyond blind scaling.
1. Revisit Metrics Regularly
Metrics age like software, not like wine. What worked in 2020 may be irrelevant in 2025. Translation BLEU, image F1-scores, or toxicity benchmarks degrade as culture and language evolve. Organizations should audit key metrics every 12–18 months. Questions to ask: Whose reality is still being captured? Whose isn’t?
2. Recognize Divergent Values
Scaling expands user diversity. A hiring algorithm used in both Germany and Brazil will face different expectations about fairness. Instead of one global metric, define a portfolio: speed for some contexts, accuracy for others, fairness-weighted scores where social stakes are high. This requires admitting that metrics are political, not just technical.
3. Resist Metric Monoculture
No single KPI should dictate go/no-go decisions. Adopt multi-objective dashboards, where trade-offs are visible rather than hidden. For instance, a medical AI could report sensitivity, specificity, and demographic parity side by side. Leaders must resist the comfort of a single “magic number.”
4. Root Evaluation in Communities
Involve end-users directly in evaluation. Community juries, participatory design workshops, or feedback loops can surface blind spots that benchmarks miss. The Canadian government recently piloted citizen panels to assess AI systems for immigration processing—an approach that should be copied elsewhere.
These four R’s shift evaluation from being a technocratic exercise to a democratic one. They also acknowledge that AI is no longer a lab curiosity; it mediates work, justice, and health. As such, evaluation cannot remain a closed conversation among engineers.
For executives, the call is clear: stop funding model size as a proxy for quality. Instead, fund evaluation as a first-class research agenda. For policymakers: require plural metrics in regulation, just as financial disclosure requires multiple balance sheet indicators. For professionals: demand that AI tools report not just performance, but for whom that performance holds.
In short: override the scaling reflex with human judgment.
SPARK
Scaling laws are seductive because they mirror an old industrial dream: growth as progress. More steel, more oil, more code—always more. But if Diaz and Madaio are right, AI progress will not be linear or universal. It will be plural, contested, and local.
The uncomfortable question is: What if the pursuit of ever-bigger models is not technological destiny, but a cultural addiction to size?
Investors love scaling because it simplifies due diligence: bigger model, higher moat. Engineers love it because it transforms research into engineering. Politicians love it because it signals national ambition. Yet none of these stakeholders is asking the Basque journalist or Yoruba storyteller whether the system actually works for them.
To break free, we might need to treat evaluation not as an afterthought, but as the core of AI research itself. Imagine if every leaderboard included a “community satisfaction index,” or if scaling competitions were replaced with “inclusivity challenges.” Would the race look the same?
Food for thought as we enter another season of trillion-parameter hype.
Further Reading:
UK AI Safety Institute, Early lessons from evaluating frontier AI systems (2024)
Selbst & Barocas, The Intuitive Appeal of Explainable Machines (2018)
Why do we work when we don’t have to? My latest essay (only for paid subscribers)
Take a look at the first contributed article on Honest AI. If you’d like to write for Honest AI, you can share your idea here.
P.S. If you’re feeling overwhelmed by the AI noise, its claimed capabilities, and the plethora of tools available, or if you're struggling to make AI work for your specific context, I can help.
I offer limited consulting slots to executives and professionals seeking to transition from AI confusion to discernible results. No
generic frameworks. Just personalized roadmaps that match your skills, job-to-be-done, and timeline.
Only 2 slots in September remaining. They fill fast.
Book a discovery call - takes 2 minutes.