Eric Gilley, Author at iPullRank

1 Important Lesson Omar Little Taught Me About Writing Page Titles

Eric Gilley — Thu, 22 Jan 2026 12:00:00 +0000

I got my start in content by editing articles for a site that lived and died by clicks. We didn’t use the word “clickbait” (not out loud), but that’s exactly what it was. I’m not proud of headlines like “3 Frustrating Toyota Tacoma Problems That Send Drivers Into a Rage” or “Aaron Rodgers Sends Packers Fan a Clear Message With Latest Podcast Appearance”, but hey, I had to pay the bills.

And in the wise words of Omar Little from the TV show The Wire:

Clicks equaled money, and making people feel something was how you got them to click. That was the incentive. That was the game.

These days, we’re seeing a new game forming. The incentive structure is shifting.

For many types of content, visibility now comes from being useful to something that isn’t human at all; not a reader, but an amalgamation of three different LLMs and a swarm of search, retrieval, grounding systems, summarizers, re-rankers, and answer engines I just (incorrectly) lump under the catch-all: “AI.”

It doesn’t feel outrage. It doesn’t get curious. It just scans, slices, and decides in a fraction of a second whether your page contains an answer. And what it sees first, namely your title, url slug, and a short text snippet, is the first and most decisive signal it has to go on. If those don’t scream relevance, you probably don’t get cited.

This means the headline’s job isn’t to tease; it’s to plainly declare what you know and who you’re useful to. It’s not clickbait; it’s AI bait. And once again, the game is rewarding those who know how to write to the system.

How an AI "Sees" Your Page: The Snippet-First World

Contrary to what many assume, an AI doesn’t “see” your webpage in its entirety. It doesn’t load the full HTML, parse the layout, or admire your design. Instead, As Dan Petrovic explains in How GPT Sees the Web, its first interaction is with a small, ruthlessly efficient packet of data: the page title, the URL, and a short text snippet.

When an LLM is grounding an answer via search, this is the initial context it’s given to decide whether your page is worth retrieving at all.

This is the core idea behind what we at iPullRank call Relevance Engineering: intentionally shaping content so its meaning is legible at the exact moment a system is deciding whether your page is useful. Before design, before UX, before persuasion — relevance has to be established.

Think of it as a high-stakes, single-chance audition. If that preview doesn’t scream “I have the answer!” for the user’s query, the AI will logically and instantly move on to a source with a stronger, more relevant signal.

It is an automated process of elimination designed for maximum efficiency. The AI’s decision to “click” and investigate your page further is won or lost in this initial glance, making the quality of that preview paramount.

Not to mention, even when a page is retrieved, the model still isn’t consuming it all at once. It’s evaluating the content in constrained windows, pulling only the portions most likely to resolve the question at hand.

Tools like Relevance Doctor exist to diagnose this exact moment: what signal your page sends in that first interaction, and whether it’s strong enough to survive the cut.

From Human Clickbait to "AI Clickbait"

Over the years as the web evolved, headlines changed as well.

You can think of headline evolution like this:

Early SEO (Late ‘90s to late ‘00s): Keywords signaled topical relevance (“Fixing Crawl Budget Issues”)
Content marketing era (Early to mid 2010s): Curiosity and emotion drove clicks
Social era (Late ‘10s to early ‘20s): Outrage and intrigue maximized engagement
AI Search era (today): Explicitness and semantic density determine inclusion

The incentive hasn’t changed. Visibility still rewards alignment. What’s changed is who’s doing the selecting.

You can see this incentive shift play out in real traffic patterns.

Below is an anonymized traffic chart from a content site that leaned heavily on curiosity-driven, emotionally charged headlines. For a few years, that approach performed extremely well by optimizing for human behavior: intrigue, open loops, and the promise of a payoff after the click.

That strategy wasn’t wrong. It was perfectly aligned with the distribution systems of the time. But as discovery began to move upstream into AI-powered summaries, re-rankers, and answer engines, the same ambiguity that once drove clicks became harder for machines to interpret.

When the selector changes, the headline has to change with it.

That same snippet-first, elimination-driven efficiency leads to a new way of thinking about titles and snippets: a concept we’ll call AI Clickbait. But unlike its human-focused cousin, it’s about radical clarity.

Let’s look at a real-world example of an old-school title and see how it would have to change for today’s game.

To do that, we’ll pull out a certified banger from the archives, written by none other than the founder of iPullRank, 2025 Search Marketer of the Year, and my boss (please don’t be mad at me): Mike King.

This 2016 headline was a masterclass in playing to the human incentive:

“How I Sped Up My Site 68% With One Line of Code.”

Certified banger. I want to click that right now.

But that headline would for sure be ignored by an LLM. A machine isn’t curious; it’s a logician. It can’t process the intrigue. The title offers a compelling result but hides the actual subject matter, giving the AI nothing concrete to latch onto.

For today’s AI-first world, you’d have to flip the formula and close that information gap instantly:

“How to Improve Page Speed Using the rel=prerender HTML Attribute”

Definitely not as catchy. However, that’s the version you write when the reader is a summarizer, not a person. Just topic, method, and outcome.

Human-Optimized	AI-Optimized
“This One Line of Code Changed Everything”	“How to Improve Page Speed Using rel=prerender”

Here are a couple of examples from the BBC news site. Do either of these inspire you to click and find out what they’re talking about?

You’re not trying to trick the AI; you’re trying to give it the most efficient signal possible. It proves that the core of the game remains the same: you have to know your audience. The only thing that’s changing is who (or what) you’re playing to.

Is It Time to Rethink Title Tag Length?

For years, SEOs, editors, and writers were taught to treat 60 characters like gospel. Not because Google cared about brevity, but because that’s where the snippet cut off in search results. Anything beyond that risked getting truncated.

In this guide to page titles created by Screaming Frog, these are the recommended title length maximums and minimums:

Title Maximum Length – 580 pixels or 60 characters.
Title Minimum Length – 200 pixels or 30 characters.

But AI doesn’t operate under a visual pixel constraint. It’s not laying out a search results page. It’s parsing meaning. Truncation doesn’t matter as much anymore; data loss does.

As written in this article by Krishna Madhaven at Microsoft, “Your page title, description, and H1 tag (the top-level HTML heading) are important signals AI systems use to interpret purpose and scope.”

The real risk isn’t your title getting cut off. It’s your most important concepts never making it into the snippet that gets passed to the LLM.

Take an old-school headline like:

“Fixing Crawl Budget Issues” (26 characters)

It’s short and technically accurate, but semantically flat. Instead try:

“How to Fix Crawl Budget Issues That Prevent Googlebot from Indexing Your Site” (77 characters)

It’s longer, but also denser: how-to intent, crawl budget, Googlebot, indexing problems. It gives the AI more reasons to believe your page directly addresses the user’s query. Not just “this is about SEO,” but “this is about exactly the issue you asked about.”

In a small internal analysis of one site, we compared high-ranking pages that were cited by AI systems against high-ranking pages that were not. While both groups performed well in traditional search, pages that were cited tended to use longer, more semantically dense titles on average.

This doesn’t prove that longer titles guarantee more AI visibility. But it does suggest something worth testing further: when relevance is evaluated programmatically, titles that surface more context upfront may have an advantage.

The takeaway isn’t “write longer titles.” It’s to stop treating length as the constraint and start treating meaning as the variable that matters.

The goal isn’t to jam in keywords; it’s to front-load meaning.

How to Craft a Headline for an AI-First Interface

Writing an AI-legible headline is about removing ambiguity at the moment of evaluation. Screaming Frog suggests following these best practices for page titles:

Concise
Descriptive & Relevant
Unique
Includes Brand
Optimal In Length
Enticing

A practical way for marketers to approach it:

Start with the question your page answers. If you can’t phrase it as a question, the page probably isn’t answer-shaped yet.
Name the core entity and problem explicitly. Don’t assume prior context.
Front-load meaning, not modifiers. Lead with what the page is about, not how impressive the result is.
Reduce intrigue that hides subject matter. Curiosity gaps work on humans; they slow down machines.
Sanity-check for isolation. If the title were stripped of its surrounding page, would its purpose still be obvious?

As Krishna writes, “Page titles should clearly summarize what the content delivers, using natural language that aligns with search intent.”

A New Title Length Best Practice Worth Testing

This isn’t about throwing out everything you know about SEO. It’s about recognizing that the rules of visibility are evolving, and the audience is changing.

You’re no longer writing just for humans. You’re also writing for models that don’t scroll, don’t skim, and don’t guess. They parse titles, weigh snippets, and choose what to cite based on how clearly your content signals that it’s an answer.

But let’s be real: not every headline needs to be AI Clickbait. If your content is built for humans (email, social, or storytelling) then cleverness still has its place. A good curiosity gap still works where curiosity lives.

The point isn’t to kill creativity but to optimize for the interface you’re speaking into.

When the goal is LLM visibility, AI Overviews, or zero-click citations, semantic clarity wins. In those cases, your headline isn’t a hook. It’s a claim.

So experiment. A/B test longer, clearer titles. Revisit old blog posts that rank well but get ignored by AI search. Give your best content a second chance by making it unmistakably answer-shaped.

Because like Omar said, it’s all in the game. And these days, the game picks the clearest answer.

Want to learn more about optimizing for LLMs?

Read our AI Search Manual

The post 1 Important Lesson Omar Little Taught Me About Writing Page Titles appeared first on iPullRank.

Relevance Engineering at Scale: Smarter Content Pruning with Embeddings & SEO

Eric Gilley — Wed, 14 May 2025 20:17:16 +0000

Let’s be honest. The words “content audit” rarely spark joy. Especially when you’re staring down a legacy blog archive with thousands of articles accumulated over years. Traditional content pruning often feels like a necessary evil. It’s time-consuming, subjective, and heavily reliant on SEO metrics that don’t always tell the full story about strategic alignment.

You know the drill: pull traffic data, check rankings, glance at backlinks, maybe skim a few posts, and make your best guess. But how do you really know if that low-traffic article from 2018 is truly irrelevant, or just poorly positioned but semantically aligned with a core business priority? How do you scale this judgment across 1,000+ pages without losing your mind or your entire quarter?

At iPullRank, we faced this exact challenge. We needed a more intelligent, scalable, and strategically grounded way to audit and prune large content libraries. Our solution? Combining the power of AI-driven semantic relevance analysis with essential SEO and content metadata. This article walks you through a Relevance Engineering framework we developed. This isn’t just cleanup; it’s about sharpening your site’s overall semantic signal, the identity that search engines increasingly recognize, by cutting through the noise of irrelevant content.

Read Francine's Relevance Engineering primer

An Introduction to Relevance Engineering: The Future of Search

Key Takeaway:

We used Relevance Engineering to quantify strategic alignment across 1,000+ blog posts, cut 500+ underperformers, and lift sitewide semantic relevance by 2–3% — without guessing.

The Problem: Why Legacy Content Pruning Techniques Fall Short at Scale

So, why ditch the tried-and-true processes? Because frankly, for large sites, it’s often neither tried nor true enough. It crumbles under its own weight.

Anyone who’s spent weeks drowning in spreadsheets, manually mapping keywords, and trying to eyeball relevance across 800 blog posts knows the first issue: manual slogs just don’t scale.

It’s a resource black hole. You either burn out your team, rush the job, or sample so lightly you miss critical insights. The sheer volume makes comprehensive, thoughtful analysis almost impossible.

Then there’s the data trap. Relying solely on traffic, rankings, or even conversions can tell an incomplete story. Sure, that post from 2019 might get decent organic traffic, but is it attracting the right audience? Does it align with your current product positioning or ideal customer profile? Or is it just ranking for some tangential query, pulling in looky-loos who bounce immediately?

Search and organic visibility in general is changing (just ask Mike). Traffic doesn’t always equal value, and performance data alone overlooks strategic alignment. It also struggles to identify consolidation opportunities.

Worst of all is the subjectivity quagmire. What is “relevant,” really? Without an objective yardstick, it often comes down to gut feel, internal politics (“But the VP of Sales liked that post!”), or inconsistent judgment calls between team members or over time.

You end up with debates based on opinion, not data, about whether a piece truly supports core business topics. Trying to maintain consistency across thousands of articles this way? Good luck.

We needed a better way. Something faster, more objective, and capable of assessing meaning alongside performance.

Our Solution: Objective Relevance Scoring with Embeddings and SEO Reality

Faced with the scaling and subjectivity problems, we knew we needed a system that could blend semantic understanding with real-world performance data. The objective was clear: systematically identify content that genuinely aligns with the company’s core expertise and business goals, flag the rest for pruning or revision, and do it efficiently across a massive library.

Our approach hinges on a hybrid model. We don’t just use AI embeddings, and we don’t just look at SEO stats. We fuse them together. Here’s the gist:

Ground Truth in Strategy: First, we defined the core topic areas the business actually cares about, directly linked to their products and target audience needs. Think of these as the strategic pillars for content.
Quantify Meaning with Embeddings: We used AI to generate numerical representations (embeddings) for:
- Each core topic area (by creating a “topic centroid” from relevant keywords).
- The overall business relevance (embedding a concise statement of strategic focus).
- Every single blog article (combining title and main body content).
Measure Alignment with Cosine Similarity: We then mathematically calculated the semantic “closeness” (cosine similarity) between each article’s embedding and the embeddings representing our core topics and business relevance. This gave us objective relevance scores for every post against every strategic pillar.
Layer in Performance Reality: Relevance scores alone aren’t enough. We integrated key SEO performance metrics (like recent clicks from GSC) and crucial content metadata (publish and last updated dates).
Make Data-Driven Decisions: Finally, we combined the semantic relevance scores, SEO data, and content age into a decision framework (our Kill / Keep / Review model) to categorize each piece of content logically and consistently.

This approach gives us a multi-dimensional view of each article’s value. With a quick glance, you could view an article’s semantic fit, its actual performance, and its freshness. This allows for much smarter, more defensible decisions at scale.

The Step-by-Step Workflow: How We Did It

All right, if you’ve made it this far, you’re ready to get into the weeds. Before diving into the individual steps, here’s a quick look at the toolkit we used for this project. Nothing too exotic, but having the right tools for crawling, data processing, embedding generation, and analysis is crucial.

To start, here are the tools we use and how we use them:

Screaming Frog SEO Spider
For crawling the blog and extracting URLs, titles, and main article content via custom XPath.
Ollama (with mxbai-embed-large model)
Running a local embedding model gave us control and kept costs down for generating semantic embeddings at scale. You could substitute this with API-based models (OpenAI, Cohere, Voyage, etc.).
Python (Pandas, NumPy, Scikit-learn)
The workhorse for cleaning text, generating embeddings in batches, calculating cosine similarity, and merging various data sources.
Google Sheets
For final analysis, filtering, applying the decision framework, and tracking manual reviews.
GA4/Google Search Console
Source for essential SEO performance data (clicks, impressions, potential conversions).

With that out of the way, here’s how we executed each phase.

Read Mike’s deep dive on using vector embeddings with Screaming Frog

Vector Embeddings is All You Need: SEO Use Cases for Vectorizing the Web with Screaming Frog

Step 1: Define Strategic Focus (The North Star)

You can’t measure relevance if you don’t know what you’re measuring against. This first step is critical and grounds the entire analysis in business reality, not just keyword vanity metrics.

Why do this: To ensure the audit aligns with current product/service offerings, target markets, and strategic content goals. This prevents pruning content that is valuable, just not yet performing, and ensures the remaining content strongly supports the business.
How we did it: We worked with the client to solidify their core solution areas. For this B2B SaaS provider, it boiled down to distinct categories based on their unique sales plays like ‘Media Management’. For each area, we developed representative keyword portfolios reflecting user intent and product capabilities. Crucially, we also drafted a concise business relevance statement (a short paragraph capturing the ideal focus and target audience for their content efforts going forward). This statement becomes its own benchmark later.

One of the first misalignments we spotted after this exercise? A whole series of posts about remote working tips and digital nomad life — timely during early COVID, but no longer aligned with the client’s current positioning in enterprise cloud infrastructure.

Step 2: Generate Topic & Business Relevance Centroids (Representing Meaning)

With the strategic pillars defined, we needed to translate them into a format the machines could understand: embeddings. The goal was to create a single, representative vector for each core topic cluster and for the overall business relevance.

Why do this: These ‘centroid’ embeddings act as quantitative benchmarks for semantic relevance. Calculating similarity against these is far more objective than a human guessing “how relevant” a post is.
How we did it: Using our chosen embedding model (Ollama with mxbai-embed-large, run locally for control and cost), we didn’t just embed the topic name. Instead:
- We generated an embedding for each individual keyword within a topic’s portfolio (from Step 1).
- We then calculated the average of all keyword embeddings within that cluster. This averaged vector became the topic centroid – a robust mathematical representation of the topic’s semantic space.
- We also generated a single embedding for the business relevance statement drafted in Step 1.
- (Tooling Note: Running a model locally, like with Ollama, is great for large jobs where API costs could skyrocket or where data privacy is paramount. API options are faster to set up if those aren’t concerns.)

Step 3: Generate Article Embeddings (Representing Each Post)

Now, we create a semantic vector for every single article in the library.

Why do this: To represent the meaning of each article numerically, allowing for mathematical comparison against our topic centroids.
How we did it: We ran the extracted content through some basic Python cleaning routines (removing excess whitespace, stray HTML tags missed by the crawl). Then, for each article, we added the Title text and the cleaned main body content and generated a single embedding using the same model (mxbai-embed-large) for consistency. These individual article embeddings were stored efficiently, typically in a NumPy array paired with their corresponding URLs.

Step 4: Calculate Similarity Scores (Measuring Alignment)

This is where the magic happens, comparing the meaning of each article to the meaning of our target topics.

Why do this: To get an objective, numerical score quantifying how semantically aligned each article is with each core topic cluster and the overall business relevance statement.
How we did it: Using Scikit-learn in Python, we calculated the cosine similarity between each article’s embedding (from Step 4) and each of the topic centroid embeddings (plus the business relevance embedding, all from Step 2). Cosine similarity is ideal here as it measures the orientation (i.e., topical direction) rather than magnitude of the vectors.

The output was essentially a matrix, added to our spreadsheet, showing each URL alongside columns like ‘Media Management Similarity‘, and ‘Business Relevance Similarity‘, with scores ranging from -1 to 1 (closer to 1 means more similar). For example, a blog post about free iPhone 4 wallpapers showed a very low cosine similarity score across every core topic — even though it had once performed decently in organic search. This quantifiable disconnect helped us move beyond “it once got traffic” and make a stronger case for pruning.

Step 5: Layer SEO Performance & Metadata (Adding Context)

Semantic relevance is powerful, but it lives in the real world. An article might be perfectly relevant but get zero traffic, or be ancient and outdated.

Why do this: To provide the necessary business and performance context to the relevance scores. Pruning shouldn’t happen in a vacuum.
How we did it: We pulled standard SEO metrics, primarily trailing 3-6 months of organic clicks from Google Search Console (using a recent window avoids rewarding historical performance that’s since decayed). We also pulled Publish Date and Last Modified Date from the CMS or crawl data. This performance and freshness data was then joined to our main spreadsheet containing the URLs and similarity scores. We now had a master file for analysis.

Step 6: Apply the Decision Framework (Making Informed Choices)

Mama, we made it. With all the data assembled, it was time to make the calls.

Why do this: To translate the combined data points into clear, actionable categories for each article, guiding the pruning and optimization efforts.
How we did it: We established data thresholds to categorize each article. This wasn’t purely algorithmic; it involved setting rules and then reviewing the output.
- KILL: Candidates typically had low similarity scores across all core topics, low recent GSC clicks, and were old (e.g., > 5 years with no significant updates). These offer little strategic or performance value.
- KEEP: Articles generally qualified if they had high similarity to at least one core topic OR had strong recent SEO performance, even if relevance was moderate. These are either strategically sound or proven performers.
- REVIEW/REVISE: This crucial category caught articles that were relevant but perhaps outdated, underperforming despite relevance, or highly similar to other posts (potential consolidation targets, like those >0.90 similarity). These need human judgment.
- Understandably, this process still included manual review. The automated categorization flagged candidates, but strategists reviewed edge cases, confirmed decisions, and identified specific actions (e.g., “redirect,” “update and repromote,” “consolidate into new pillar page”). High-relevance articles flagged for review due to age or performance became top priorities for content refresh efforts.

Okay, deep breath. That was the methodology. Now, let’s see if all that computational elbow grease actually moved the needle.

The Results: A Leaner, More Relevant Blog

So, did all that data wrangling and vector crunching pay off? For our client, the answer was a clear yes. The methodology provided the objective evidence needed to make significant, strategically sound changes.

Here’s a snapshot of the outcomes:

Quantifiable Lift in Overall Relevance: This was the headline metric. We went beyond individual page scores and actually measured the blog’s collective semantic alignment. By creating a single representative embedding for the entire blog’s content (think of it as a ‘mega embedding’) both before and after pruning, we could calculate the change in cosine similarity against the core business relevance target.

The result? A site-wide relevance lift of 2-3%. For a large, established content library, improving the overall semantic signal by that much is definitely nothing to sneeze at. It’s a direct measure of increased focus.

Significant Pruning Achieved: The analysis identified substantial deadweight. Based on the combined relevance, performance, and freshness scores, we flagged approximately 45% of the content library for pruning or consolidation. This meant strategically removing or redirecting over 500 articles that no longer served a purpose.

Foundation for Future Authority: The result is a more focused, coherent, and authoritative blog, better positioned to perform in search, resonate with the target audience, and drive meaningful business results. It established a cleaner baseline for future content development and topic cluster expansion.

This wasn’t about hitting a deletion target; it was about using a combination of semantic understanding and performance data to surgically refine the content library for maximum strategic impact.

How You Can Apply This Framework (And Why You Should)

This methodology isn’t just a one-off project; it’s a robust framework adaptable to several common strategic needs. Its real advantage lies in bringing objectivity and scalability to relevance assessment, moving beyond gut feelings and manual slogs. By integrating semantic analysis with concrete SEO data and content metadata, you get a holistic view of content value, ensuring decisions align with both search demand and business priorities.

Key Takeaways: Smarter Pruning, Stronger Strategy

Ultimately, tackling a sprawling content library requires more than just traffic analysis and gut instinct.

The core idea here is that AI-powered embeddings offer a genuinely scalable and objective way to measure semantic relevance, moving beyond subjective interpretations of what fits your strategy. However, relevance alone isn’t the full picture. True strategic content management emerges when you fuse that semantic understanding with concrete SEO performance data and content freshness, creating a holistic view of each asset’s real value and potential. Remember, this isn’t merely about deleting old pages; it’s about strategically focusing your content portfolio to sharpen your site’s overall thematic signal, making it clearer to both search engines and your target audience what you stand for.

Adopting this kind of data-driven, multi-faceted framework provides a defensible and far more effective approach to content audits and ongoing governance, especially when dealing with the complexities of large, legacy websites.

Want to find out about how Relevance Engineering can help your business?

Learn about iPullRank's Relevance Engineering Services

The post Relevance Engineering at Scale: Smarter Content Pruning with Embeddings & SEO appeared first on iPullRank.