I have a friend with a habit of posting twenty photos at a time of her hiking trips. Photos she took in the middle of the Alps, among inhospitable, snow-capped peaks, landscapes speaking of wonder, adventure, ancient times. A place I would visit in a heartbeat.
I flick through them; three or four of them look exactly the same to me. Colours are faded, composition is off. Some are slightly tilted. I give a like but quickly move on. I’m on a quest. Something more polished, pleasing, perfect, surely will come up soon.
I remember the early days when we joined Instagram to see photos posted by our friends. Within the span of a decade or so, this platform transformed from a place of connection to one of consumption. The friend content has not disappeared but rather pushed to the sidelines. Content creators dominate now, providing us with an infinite supply of tailor-made entertainment.
Information has never been more central to our lives, technological innovations since the 1950s constantly improving access to and increasing the quantity of it. At one point, however, the link between information and the utility we get from it, might have turned from positive to negative: when there was a finite amount of relevant information to each of us – say, through the daily paper, news channel or magazines – each new piece of information had a reasonable chance at increasing our knowledge, understanding of the world, literature, culture, history, local events. This new information chunk might have then contributed to raising our intellect, status, ability to make conversation or open-mindedness.
Today’s daily flood of information, however, can feel increasingly paralysing. Where do you begin when everything seems popular, everything seems urgent and the speed of publishing breaking news seems to have become a race to the bottom?
On top of that, of course, come the uglier outgrowths of an increasingly unimpeded online world, among the most salient being misinformation and AI-generated content.
Researchers found the exposure to both misinformation and AI content to have left people more uncertain in general: While we become better at spotting fabricated material, we now are also more likely to wrongly distrust genuine images and information as a result.
In this new world of abundant information, is it still the consumer who wins? Maybe. Let’s say you disable the algorithm on your timeline, you follow a limited number of people and sites on your preferred platform that strictly share content valuable to you. You have the self-discipline to open and close your phone at predetermined times. Within the span of fifteen minutes, you might learn about new investment hacks, skincare tips and budget travel destinations. Not a piece of AI slop snuck in.
There is a reason why most of us can’t relate to the above scenario. The hurdles we have to climb, the defaults we have to disable, the mental power we need to exert every single time we open or do not open our phones, is excessive in light of all the other challenges we face in our offline lives. Plus, if we so strictly limit the content we see, we might miss out on other interesting things. Better things.
In this process, utility increases more for those controlling and capitalising on the flow of information, rather than for its consumers. This happens in more ways than one. Big Tech sells our data and encroaches on our privacy in doing so. Governments have introduced regulations on this across the world.
But it’s not our data that’s the most valuable thing we give away on these platforms. Instead, it’s our attention, an asset we that may be harder to protect through privacy laws. And it isn’t just social media firms that want our attention.
Political parties have also learned to make this information-overabundance work for them, often using misinformation (false information) and disinformation (false information deliberately intended to mislead) to their advantage to influence their popularity and election results. What seems a new problem of our times, is merely the incarnation of a permanent conflicting relationship between factual truth and politics: in 1967, Hannah Arendt declared both to be on “rather bad terms with each other”, facts precluding debate, the latter “constituting the very essence of political life”. Today, this conflict materialises in the form of bots, poor quality news sites and AI-generated videos. Ad revenue makes all of this work.
Retailers, by placing their ads where our attention is, have become an unlikely pawn in this system. Content is what gets people’s attention. Platforms and sites with content often finance themselves through ads. Typically, “programmatic advertising” is applied, ads placed automatically based on user demographics and buying interest. An estimated $2.6 billion in such ad revenue is being sent to publishers of mis- and disinformation each year.
To maximise profitability, there are sites that have now turned into “content farms”. They’re places where low-quality, AI-generated content imitates news. Previous to AI, some sites simply copy-pasted articles from other news outlets, and occasionally fed in a new piece, fabricated stuff – playing the long game. Artificial generation of content has made this much simpler. These sites often originate from unlikely destinations such as Vietnam, Pakistan, Sri Lanka and Venezuela. But they aim at Western audiences because ads bring in more money for each dollar spent. This has created the bizarre situation where AI videos are churned out in Sri Lanka to create outrage among the French or British population on whichever topic is currently politically heated.
The problem with making attention the core of a business model is that, to maximise it, one must target and exploit human biases as much as possible. While this seems morally ambiguous, does it necessarily have to lead to a negative outcome? Who was responsible for the proliferation of misinformation, disinformation, and content farms? We’ve seen the news about Russian bots and Cambridge Analytica. Could it have been them?
The jury is still out on why mis- and disinformation spreads on social media; partly because we would need to have a close look at what exactly the algorithm does under the hood (access for researchers to these algorithms is almost unheard of). But the answer is most likely a blend of algorithm, the people we follow and our own biases.
Researchers at Northwestern University in the US, began looking into this. After finding emotion to be a driver of sharing information, they looked further. They found that “moral-emotional content” explains the spread even better: When motivated to identify with a group, users become more likely to share this type of content. It combines statements of strong emotion with moralised ones, the latter defined by the authors as referring to ideas interpreted in terms of the interests of a unit larger than the individual. They find that this content is especially likely to capture our attention, and that platform design – the algorithm, the reactions we can give and so on – amplifies our natural tendencies to spread it.
In my own research, I looked at how much more likely misinformation spread on Twitter (before it was X) if a tweet contained emotional language. Not very surprisingly, I found that people especially liked to retweet when “high arousal” emotions were involved. Those span joy, anger and fear, and less so feelings like sadness or affection. On top of that, people were more likely to retweet low-quality information if there were more words in the tweet that related to the outgroup, the group opposing the author’s views.
American journalist Bill Buford, when living in England, began following British football hooligans in 1982, hooliganism hitting its peak in the eighties and nineties. Over eight years, he befriended them, went to games and was in the crowd when it turned violent. In his 1991 book “Among the Thugs”, he writes this about the crowd:
“We have no illusions about the potential depravities of our nature, except that rarely, despite our modern sophistication, do we admit that these depravities are genuinely our own: yours, mine. […] Here, now, in England, in London, down a side-street, not far from the centre, there is a crowd assembling: but that crowd, we will insist, is not us.”
What makes us human, including the traits that pit us against each other, has always been there. We cannot solely blame the algorithm for this, even if it sets the stage for what type of behaviour is rewarded most.
“Was it not because ringleaders, of dubious politics, infiltrated and influenced the crowd?” In British football, there were “firms” with leaders who often instigated the violence – but once the crowd formed, nobody controlled it. Much more, it was fuelled by an “us versus them”, a sense of belonging that tipped into anger, things started by one, two, three people that took on lives of their own. Went viral.
So, what can be done? Football crowd violence eventually reduced because of large-scale police crackdowns. Online, various actions have been taken: platforms fact-check and take down misinformation or flag it, while governments can block or order the takedown of sites creating the content in the first place. But government regulation of such content can tip into censorship – even if well-meaning – and thus comes with risks: within just one year, internet freedom deteriorated in 28 countries according to Freedom House, an NGO, in 2025.
The focus on stifling misinformation spread alone may also be too narrow. It is the current algorithmic design that maximises for attention and engagement, which must change. This may mean introducing anti-addictive platform designs, and it may mean prioritising content that is popular on both sides of a political spectrum. These ideas aren’t exactly aligned with current incentives though: Facebook, in 2021, piloted an algorithm that boosted posts considered to be “good for the world” by users. As a result, people opened Facebook less; the New York Times called this a “trade-off between civility and growth”.
There is, then, a need to fundamentally rethink systems, before such platform changes have a fair chance to succeed. A need for roadblocks that disable, or at least soften, the attention-maximisation mechanism. I think we can already see the beginnings of this. An example: The EU’s GDPR, by increasing user privacy, was found to have had a side effect: After it reduced the degree of ad personalisation, the accuracy of content recommendations also faltered, distorting incentives toward more content creation. This happened because of the complex web of monetary incentives between ads and content creators.
What happens if we take such changes further, is anyone’s guess. We might eventually spend less time on social media. We might become more civil online. We might move to new platforms where we aren’t just consuming, but connecting and creating again.
Maybe the online world would then begin reflecting the offline one again – not the other way around.