AI bots access websites without paying a subscription while real people have to pay

I have just been doing some research to find out if Paris cleaned up the streets by removing stray cats and dogs before the Olympics using Bing’s search engine. China did this before their hosting of the Olympics in the cruellest way. And Russia before the World Cup.

Internet search has changed dramatically by Michael Broad

Side by side with the usual search results is the summarised result of Bing’s Copilot (AI chatbot). It refers to three websites that it has ‘scarped’ to find the information. One of them is www.thelocal.fr. I visited that site and my access was blocked by a subscription demand.

I can’t access www.thelocal.fr to read the source of Copilot’s information and expand on it but Bing’s Copilot did have access to the www.thelocal.fr article. The AI bot goes behind the paywall.

This is another example of unfairness on the internet at the moment. AI is pretty well destroying the viability of millions of independent websites. They are going to disappear eventually as AI dominates.

I believe that already around half of user searches are made through AI bots. What I mean by that is that around 50% of internet users looking for information now use these AI bots such as ChatGPT or Google Gemini rather than using the convention search engine method and a list of websites.

The reason is that it is much easier to let the AI bots do all the work in searching for the information. I do it myself. But AI summarised search results means that real people completely bypass these websites. Clearly, they are losing traffic bigtime.

That means they are losing advertising income which makes them less financially viable. The vast majority of independent, smallish websites are already unviable financially anyway. This is another nail in the coffin for these sites. The final nail for many I’d argue.

The shape of the internet is changing rapidly. Back in the day there were millions of individuals creating useful and interesting blogs and conventional websites providing good information. That’s all disappearing. Today there really isn’t the motivation to create a content website. It’ll be drowned out by competition and ‘stillborn’ by AI’s looming presence.

ChatGPT are about to launch their own search engine to compete with Google which by the way has a near monopoly which is illegal but they use an army of high-powered lawyers to circumvent efforts to reduce its dominance.

Bing’s search engine is better than Google. It is more accurate and relevant but Bing has a fraction of the users enjoyed by Google.

The internet used to be open for individual entrepreneurs wishing to make a bit of money through a website. Those days are disappearing rapidly. It is all about dominance and big business. Even when Google search works normally, the big websites are favoured like Quora and Reddit.

The small guys are metaphorically speaking feeding on the crumbs from the high table. Scavenging for a bit of activity. It is sad. It is capitalism in action. Unregulated capitalism. The internet has always been largely unregulated in the West. Russia and China control access to the internet by contrast, which is even worse.

In the future an internet search will use AI only and it will provide all the information. The AI bot will refer to the source websites and provide links but few users will click on those links.

These changes are an existential threat to millions of websites, I argue, and even big ones such as mumsnet.com are suffering. It decided to sue OpenAI the majority owner and creator of ChatGPT for compensation for stealing their information and a court order to desist and delete the stolen information.

Billion-dollar litigation is being spawned by this seismic change to the internet but all of it revolves around the dire unfairness of these developments and breaches of laws governing competition.

RELATED: Pets Radar is an Internet menace

Mumsnet

Here is a summary of the mumsnet.com litigation from Phind another AI search engine; new on the market.

The litigation between Mumsnet and OpenAI revolves around allegations of content scraping by OpenAI from Mumsnet’s platform. Mumsnet, a UK-based online community and parenting forum, initiated legal action against OpenAI, accusing the AI company of scraping billions of words and content from its site without permission. This action marks the first legal challenge against OpenAI in the UK concerning copyright infringement, breach of terms of service, and database right infringement. Mumsnet’s complaint demands that OpenAI delete all Mumsnet data it possesses and halt any further usage.

The dispute began when Mumsnet discovered that OpenAI was scraping its content, allegedly to train its Large Language Model (LLM), ChatGPT. Mumsnet’s terms of use explicitly forbid any form of distribution, scraping, or copying of its content without prior approval. Despite this, OpenAI reportedly showed little interest in licensing Mumsnet’s content, preferring datasets that are less accessible online. Mumsnet’s founder, Justine Roberts, highlighted the importance of protecting online publishers’ content and warned of the potential harm to communities like Mumsnet if AI developers continue to scrape content without permission.

Mumsnet’s stance is part of a broader trend among news publishers and online communities facing similar issues with AI companies using their content without proper agreements or compensation. Other entities, including The Center for Investigative Reporting, The Intercept, Raw Story, and Alter Net, have also taken legal action against OpenAI and Microsoft for copyright infringement and exploitative practices related to the use of their content for AI model training.

OpenAI responded to Mumsnet’s complaint by confirming receipt of the letter and expressing commitment to working within the bounds of the law. The company also indicated willingness to assist publishers in preventing bot scraping of their sites, suggesting measures beyond legal requirements to address creators’ concerns.

This litigation underscores the growing tensions between AI companies and content creators over the use of online content for AI model training. It highlights the need for clearer guidelines and agreements regarding the use of web content by AI technologies, especially in contexts where consent and compensation are crucial for maintaining the integrity and sustainability of online publishing platforms.

The sources for the above include Tech Radar, Mumsnet and Press Gazette to name three.

You see, I used an AI bot for that information. It’s convenient. It is good but it means websites that are the source of the information are losing out. Their hard work is being abused.

Leave a Comment

follow it link and logo