How does Google treat texts written by AI? [Experiment results]

Content created by AI – Google rating

On behalf of the Content Writer, we conducted an experiment aimed at evaluating the treatment of AI content by Google search engines.

The purpose of the experiment

Checking the visibility of articles created by artificial intelligence.

Duration

From 30.05.23 to 30.06.23.

Methodology

We used two Polish content-related websites for testing: content-seo.pl and contentmarketingowy.pl. On the first one, we published 100 AI articles every other day, resulting in a total of 1400 entries. On the second website, we focused on quality through additional editing, so we only inserted 6 articles per week, resulting in a total of 50 entries.

The AI engine used for the experiment is GPT-3-5-turbo. The prompt looked as follows:

“Create a substantive Polish article of at least 1000 words on the topic of “[title].” Organize the article using H2 headings, without an H1 heading and without a title. Add an introduction. Make sure the content is informative and provides valuable insights to the reader. Bold the entire first paragraph with a tag <strong>. Frequently use unordered lists with tags <ul> and <li>. Conclude with a tag <h2> that summarizes the key takeaways from the article. The writing style should be accessible and suitable for a blog”.

All topics and content were related to the copywriting industry, in line with the themes of both websites. They were 100% developed by AI – during editing on the second portal, we only corrected language errors. Thanks to the Cron Job configuration, articles were created automatically “in the background”, without our involvement.

Results

A visible increase in the number of views appeared on both portals in the first days of publication – then the results quickly returned to the initial value.

(For context, the results from the beginning of May were shown, so one month before the experiment started).

Content-seo.pl

Contentmarketingowy.pl

The articles are consistently indexed, although practically all of them are positioned outside of the TOP100, which does not translate into clickability.

Content-seo.pl

Contentmarketingowy.pl

Conclusions

Articles created by AI are indexed in Google, but they do not generate any traffic on the website. We have not observed any impact on click-through rates, even despite the large number of articles (1400).

Such an amount of content created by humans usually results in tangible outcomes that attract thousands of users. Apparently, Google negatively evaluated articles written by AI – even when they were additionally edited.

A large number of articles outside of the TOP100 have an additional side effect: it lowers the average position of the website in search results. This limits the possibility of achieving high positions in the future. It is therefore worth considering the consequences of such actions, which can negatively impact positioning for years to come.

However, it is worth noting that SEO is an ongoing process, and our experiment has only just ended. The results may change, and in that case, we will promptly update this publication.

Update [3.01.2024]

After 6 months, still nothing, silence everywhere, quiet everywhere, only the sound of crickets in the grass can be heard… The SEO results remain in their places, which shows that in the long term we will also not experience any surprises.

Content-seo.pl

Contentmarketingowy.pl

We consider the experiment to be finished. Without a doubt, we will not apply similar strategies either in ourselves or in our clients. The creator must be a human, and artificial intelligence remains a support – and only a support.

Update [22.07.2024]

The pages experienced a permanent decrease in traffic, so we decided to remove all the content created as part of the experiment in order to start building domain trust again.

It is worth mentioning in this place about the growing penalization of AI content by Google, of which the flagship example is the downfall of the Funky Media website. The portal published several hundred articles per day – and seemingly “well-maintained” ones at that. The posts included a table of contents, graphics, as well as automatic linking. However, the Google algorithm update has resulted in the visibility chart of the website looking like this now:

The page shone like a star, taking advantage of a momentary oversight of indexing robots. However, the search engine put an end to this practice, limiting the reach of the page by over 99%.

For comparison, this is how the visibility chart of the website contentwriter.pl looks like, where we have been consistently posting content created entirely by experts from the very beginning.

It is also worth noting that in content marketing, it is not just about numbers, but primarily about acquiring customers. Sharing knowledge generates valuable traffic, which cannot be said about generic AI texts.

An extreme example here is the article by Funky Media, where artificial intelligence was supposed to generate a post with the keyword “marketing raport.” Unfortunately, a space was mistakenly inserted after the word “rap,” which led to the AI publishing… an extensive article about marketing among rappers.

Today, we can with certainty state that the potential of artificial intelligence should be utilized where it truly finds its application. However, we must maintain moderation and a healthy approach, as not everything can be entrusted to machines.

The content writing industry has proven to be surprisingly resistant to the AI revolution. This has happened not only due to Google’s firm stance against AI-generated content but also thanks to ordinary readers who prefer texts created by humans.