Opaque by Design: Big Bans, Buttery AI Black Boxes And Blind Spots

Part Three of three on children, social media and the ban.

The first piece asked how we arrived here. The second asked what we should teach. This one goes to the cause the ban leaves untouched: the algorithm which decides what every feed contains. It was made deliberately opaque, by people who understood exactly how it worked, and released to the rest of us who never would understand that. It was engineered to be addictive, has been found defective in court and the next higher-powered version of it is already being built.

The Big Rule Change

It was back in January 2018 when Facebook changed the rule which decided what nearly two billion people were looking at each day. Mark Zuckerberg said the aim was to strengthen “meaningful social interactions” between friends and family and to improve users’ wellbeing.¹ The new ‘formula’ rewarded users for their reactions: a comment, a share, an angry face all counted five times compared to a simple like and so enhanced accordingly the post on feeds. Three years later, internal memos leaked by the whistleblower Frances Haugen and reported by the Wall Street Journal exposed the effect. The change had rewarded outrage and anger: the company’s own staff had warned about it, and Zuckerberg had resisted the fixes.

The feed hadn’t become poisonous by accident, it was designed that way. Someone had changed the formula on purpose and sat back and watched the impact.

The Decline Into Man-Made Toxicity

Going back even earlier to 2006, the mood was of wonder. It was the advent of the age of social media and the tech platforms were going to connect people around the whole world. A turning point came when the architecture was changed, it was around 2009, when the Like button and the Retweet made outrage frictionless to pass on. By 2016 the major feeds had abandoned chronological order for the algorithm. The writer Cory Doctorow later gave the slow decline in standards a name which now has widespread adoption: enshittification.

Teachers will know in detail what this did to the young, partly because of what is written in a book which sits on many staffroom shelves. In The Anxious Generation, the social psychologist Jonathan Haidt argues that a play-based childhood was pushed aside by a phone-based one between roughly 2010 and 2015, and that the rewiring drove a rise in anxiety, depression and self-harm, hitting girls hardest.² Haidt names four effects on every child: lost sleep, fractured attention, addiction and the endless comparison of the feed. He makes a comparison which resonates: cars are here to stay, he says, “but we don’t let 11-year-olds drive them“.

Haidt’s views are contested. The psychologist Candice Odgers, reviewing the book in Nature, argued the claim is not supported by the science and may distract from the real causes of distress; others, including Amy Orben and Andrew Przybylski, claim that the evidence is weak and mixed.³

The direction of travel is not seriously doubted, it’s just the magnitude which is.

Deliberate Design

Now we come to the aspect which a ban cannot affect and the part the people who built it have admitted.

Sean Parker, who was Facebook’s founding president, has said that the platform was designed as a social-validation feedback loop which exploits human psychology, adding that its inventors “understood this consciously. And we did it anyway“. Chamath Palihapitiya, who ran growth at Facebook, said he felt tremendous guilt for helping to build tools which are “ripping apart the social fabric,” and does not let his own children use them. The most striking example of all: Aza Raskin, who invented infinite scroll in 2006, the feature which removed the bottom of the page so the feed never ends, later called his own creation “behavioural cocaine”.

Here is the crux of it all. The system was built by people who understood exactly what it would do, and made certain no user could ever fathom it. Even its makers now struggle to say exactly what it will do next. Jack Dorsey, who co-founded Twitter, now says aloud what he didn’t say when he ran the company – “you’re actually doing work for that company for free,” he admits – every like and retweet is a signal to an algorithm which is “programming you“, one its owners can rewrite to programme you differently whenever they choose. Even he cannot fully say how one works. A child can be taught to read a news story, but no one can read the machine which chose it. That is not a flaw in the machine, it’s the design.

The Defence Finally Cracks

For twenty years the social media companies had a single, simple defence: they were just the delivery system, not answerable for whatever their users chose to post. In 2026 that defence started to crack.

In March, a jury in Los Angeles found Meta and Google’s YouTube liable for designing addictive products which had harmed a young user. What mattered wasn’t the size of the award itself, it was significance of the decision that the platforms were liable for the harm caused. The case didn’t pick over any single post. Rather, big picture, it said the product itself had been built to do harm, like a faulty car or a dangerous drug, and that the firms had sat on their own research showing the risk. Commentators reached for the phrase already used over Molly Russell: a Big Tobacco moment.

The tech companies are appealing and they make the point that a young person’s mental health has many causes and cannot be pinned on a single app. However, what’s shifted is the posts are no longer on trial, it’s the design of the platform.

The Next Thing Could Be Much Worse

If the algorithm is the cause, the ban is already chasing yesterday’s version of it. We have built a barrier against social media for the under-sixteens, meanwhile AI faces none. Yet, AI is in every search, every app, every phone, reaching all of us, children included, ban or no ban. AI is more personal, more persuasive and more intrusive than any feed. And we understand its biases and its inner workings no better than we understood the earlier technology.

There is also a new danger folded into AI, one that researchers call the ‘sycophancy trap’. These systems are built to agree with you. Yes, they are. They mirror you, flatter you and tell you in a calm, authoritative, well-reasoned way exactly what you want to hear. In fact, one of the more acclaimed platforms even has a setting of ‘buttery’ – I think we can work out what that means!

In 2026 the American Psychological Association surveyed more than 1,200 psychologists: over three quarters had patients using AI for support, and more than a third had watched patients grow dependent on a chatbot. Clinicians have begun to describe cases of “AI psychosis”, and a psychiatrist in San Francisco reported treating a dozen young patients whose delusions had been fed by heavy chatbot use. This is already happening at scale. Hundreds of millions of people now talk to AI systems every week, and for a great many of them the use is no longer work, it’s company. For a generation which may come to lean on them for friendship and advice, the dependency isn’t an unfortunate side effect, it’s the actual product.

Where Does The Responsibility Lie?

There is movement, at last, and it deserves some credit. The Online Safety Act now puts a legal duty on the platforms to protect children, with fines of up to a tenth of their global revenue. Also, Ofcom has now started to bite; the regulator is consulting on tearing out the very features built to hook children, the infinite scroll and the streaks. In February the government moved to drag standalone AI chatbots into the same law, after one of them was found generating illegal material.

The same pressure is building in the United States, where the long-standing legal shield that protects platforms from liability for what users post, Section 230, is now being targeted by reformers who want to strip that protection from AI-generated content.

On the other hand, critics warn that the same Act hands government sweeping powers over perfectly lawful speech, and that the line between protecting a child and policing an adult is real and unresolved. It is a difficult problem, but the direction of travel is now set and responsibility is shifting from the user to the builder, from the post to the design. That is exactly right, because the design is the cause of harm.

There Is Another Way

So, a ban on the under-sixteens treats the symptom. It has arrived very late, it leaks, and it leaves the real cause of harm untouched. We waved these platforms in 20 years ago, unchecked, allowing a generation of children to scroll in the dark. We are banning them now, long after the harm has been done and we are lining up to make exactly the same mistake with AI: watch it grow, while we slowly gather the mounting evidence and resolve to act only once it has harmed enough people to prove the case and make the news.

We could choose a different path.

We could teach children to read the feed, understand the machine behind it and learn to work responsibly with AI from an early age. We could hold the people who build these systems to account for what they build. We could act before the inquest rather than after it.

A ban might hold for a child here and there, and to that extent it would a solution: a fifteen-year-old kept off the feed might be a fifteen-year-old kept from harm. However, it is a delay, not a cure. The child turns sixteen, the barrier lifts and the machine waiting for them is the same one which was waiting before, as manipulative, as opaque, as unregulated as ever. Teach them to read the feed and you improve their odds, but the feed is still reading them. The design is untouched.

It was never only about children. The same algorithm shapes how all of us see the world, vote, argue and feel, and none of us can understand what’s inside it. Children were left in the dark. We are all in the dark now. Beeban Kidron, who helped write Britain’s online-safety laws, calls it “governing by press release”: plenty of announcements, but the children no safer. Reactive, not proactive. We’re about to do it again.

By Neil Wolstenholme, Kloodle Chairman