{"id":15549,"date":"2022-11-07T11:02:28","date_gmt":"2022-11-07T16:02:28","guid":{"rendered":"https:\/\/ipullrank.com\/?p=15549"},"modified":"2025-07-31T15:59:46","modified_gmt":"2025-07-31T19:59:46","slug":"ai-content-not-seo-threat","status":"publish","type":"post","link":"https:\/\/ipullrank.com\/ai-content-not-seo-threat","title":{"rendered":"AI Content is not the SEO threat they want you to think it is"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Get iPullRank’s AI in Content and SEO Free Guide\u00a0–\u00a0DOWNLOAD<\/a><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In 2018 I made a prediction at TechSEOBoost that we were 5 years away from any random script kiddie being able to leverage Natural Language Generation (NLG) to generate perfectly optimized content at scale using open source libraries. At that point, I\u2019d been keeping a close eye on what was happening in the Natural Language Processing (NLP) space with Large Language Models (LLMs) beginning to mature. Conversations that I had when I got offstage after my keynote suggested that we were actually much closer to perfect automated content being deployed by cargo coders than I thought.\u00a0<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

<\/p>\n

The next year I judged, hosted, and iPullRank provided the grand prize for the TechSEOBoost technical SEO competition<\/a>. Ultimately, Tomek Rudzi would take home the prize, but there were two strong entries that demonstrated natural language generation in multiple languages from the late great Hamlet Batista<\/a> and Vincent Terrasi from OnCrawl<\/a> respectively. Personally, I found these to be the most compelling submissions, but the judging was done by committee.<\/p>\n

Let\u2019s take a step back though. What I imagine as \u201cperfectly optimized content\u201d is the combination of what content optimization tools (Frase, SurferSEO, Searchmetrics Content Experience, Ryte\u2019s Content Success, WordLift, Inlinks, etc) do with what natural language generation tools (CopyAI, Jasper, etc) are doing. A key benefit of the latter set of tools is in how they look at relationships across the graph rather than just vertically down the SERP.\u00a0<\/p>\n

In practice, a user would submit a keyword with some instructions to a system that would derive entities, term co-occurrence, and questions as inputs from the SERP and then generate relevant and optimized copy with semantic markup and internal linking baked in. With the emergence of Generative Adversarial Networks (GANs)<\/a> and the ubiquity of text-to-speech<\/a>, the system could cook up related imagery and video transcripts as well.<\/p>\n

If that sounds a bit like science fiction, it\u2019s not. The components exist and innovative companies like HuggingFace and OpenAI are driving us there, but the elements have not been tied together into one system yet. However, there\u2019s still a year left in my prediction and iPullRank has an engineering team, so make of that what you will.<\/p>\n

In the meantime, user-friendly tools built from large language models that generate copy virtually indistinguishable from what a human may write are beginning to grab the attention of marketers and would-be spammers. The implications for a search engine that has been caught on the back foot by a social video platform<\/a> and has a growing symphony of users drawing the conclusion that results limited to Reddit as a source are better than core search quality<\/a> are potentially immense. Whether or not the two are related, we’re seeing the beginnings of a campaign to discourage the use of this type of generated content.\u00a0<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

If it's a problem, Google created it (or What is a Large Language Model?)<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Make no mistake, all the technologies that I am referencing herein are incredible albeit somewhat problematic for a number of reasons that we\u2019ll discuss shortly. Google\u2019s AI Research team has driven a quantum leap in the NLP\/NLU\/NLG fields with a few key innovations that have happened in recent years.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

While the fledgling concepts behind them stretch back to prehistoric types of Computer Science theory, the Large Language Models, as they are called, were developed based on the concept of \u201cTransformers.\u201d These marvels of computational linguistics examine a colossal collection of documents and, basically, discover the probability for one word to succeed another word based on the likelihood of words appearing in a given sequence. I say \u201cbasically,\u201d because word embeddings play a huge role here and there are also language modeling forms based on \u201cmasking\u201d<\/a> which allows the model to use the context of the surrounding words and not just the preceding words. Transformers were introduced by Google engineers and presented in a paper in 2017 called “All You Need is Attention<\/a>,” explained in something closer to layman\u2019s terms in a post called \u201cTransformer: A Novel Neural Network Architecture for Language Understanding<\/a>\u201d The concepts would be brought to life in a way that SEOs took notice of with Bidirectional Encoder Representations from Transformers (BERT)<\/a>.\u00a0<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In August 2018, just prior to the open sourcing of BERT, the OpenAI team published their \u201cImproving Language Understanding with Unsupervised Learning\u201d<\/a> post and the accompanying \u201cImproving Language Understanding with Generative Pre-Training\u201d paper<\/a> and code<\/a> wherein they introduced the concept of Generative Pre-Training Transformer or GPT-1.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

How do Generative Language Models Work?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In practice, given a prompt, a large generative language model such as GPT-1 and its successors (GPT-2, GPT-3, T-5, etc.) extrapolate the copy to the length a user specifies using the probabilities of what is most likely to be the next token (word, punctuation, or even code) to appear in the sequence. In other words, if I tell the language model to finish the sentence \u201cMary had a little\u201d it is likely that the highest probability for the next word is \u201clamb\u201d due to how often that sequence is present in the texts from the Internet that it has learned from.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In effect, language models aren\u2019t actually \u201cwriting\u201d anything. They are emulating copy that they have encountered based on the number of parameters generated in their training. You can play with this live in a variety of ways, but Write With Transformer<\/a> by HuggingFace has a variety of models that illustrate the concept.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In this screenshot, you\u2019re seeing the three different directions that the model feels confident about taking a sentence that starts with \u201cJay Z is.\u201d The longer it goes further unprompted the more it will begin to meander into topics of disinterest.\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

<\/p>

This is the same functionality at play when Gmail or Google Docs attempts to autocomplete your sentence. The email or document is the prompt and Google\u2019s language models are predicting your next word, phrase, or response as you write. Note: I suspect that it will be a bit discombobulating to see a portion of the same paragraph in the screenshot above, so I\u2019m just writing an additional sentence to make this area visually easier to differentiate so readers don\u2019t think it was a mistake.<\/em><\/p>

While large language models will generate content just fine \u201cout-of-the-box,\u201d there is also an opportunity to \u201cfine-tune\u201d their output by feeding them additional content from which they can learn more parameters. This is valuable for brands because you can feed it all of your site\u2019s content and the language model will improve its ability to mimic your brand voice. For anyone that is serious about leveraging this technology at scale, fine-tuning needs to be a consideration.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

The Large Language Model Explosion<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

<\/p>

As with all types of machine learning models, a series of values that represent the relationship between variables known as \u201cparameters\u201d are learned in the training of a language model. Parameters in language models represent the pre-trained understanding of probabilities of words in a sequence. What has been found is that, generally, the more parameters, or the more word relationships examined using this process, the stronger the language model is at its various tasks. For instance, language generation substantially improved between GPT-2 and GPT-3.<\/span><\/p>

<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

<\/p>

Despite projects like <\/span>DeepMind\u2019s RETRO being said to outperform GPT-3 with only 7 billion parameters<\/span><\/a>, companies such as Google, Nvidia, Microsoft, Facebook, and the curiously named OpenAI, have been in proverbial arms race to build bigger models. Google themselves have a series of interesting models such as <\/span>GLaM<\/span><\/a>, <\/span>LaMDA<\/span><\/a>, and <\/span>PaLM<\/span><\/a>; they\u2019ve also recently leaped into the lead with their 1.6 trillion parameter model Switch-C.\u00a0<\/span><\/p>

All of this technology is incredible, but these innovations certainly come at a cost.<\/span><\/p>

<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

What are Some Problems with Large Language Models?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

<\/p>

When I say that these language models are trained from colossal collections of documents, I mean that the engineers building them take huge publicly available data stores from well-known corpora such as the Common Crawl, Wikipedia, the Internet Archive as well as WordPress, Blogspot (aka Spam City), New York Times, eBay, GitHub, CNN and (yikes) Reddit among other sources.<\/p>

<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Naturally inherent in this is a series of biases, hate speech, and any number of potentially problematic elements that come from training anything on a dataset that does not take the time to filter such things out. Surely, I don\u2019t have to reference the Microsoft chatbot that swiftly went Kanye once it was unleashed on Twitter, right?<\/em><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Of course, there have been warnings from academics about the potential problems. Wrongfully terminated former Googler and founder of <\/span>DAIR.AI<\/span><\/a> Timnit Gebru along with Emily Bender, Angelina McMillan-Major, Shmargaret Mitchell, and other researchers that were not allowed to be named all questioned the ethics behind LLMs in their research paper “<\/span>On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? \ud83e\udd9c<\/span><\/a>“<\/span><\/p>\n

In the paper they highlight a series of problems:<\/span>
<\/span><\/p>\n