Every AI visibility tool asks you the same question on day one: which prompts do you want to track?

It sounds simple. It isn't. Nobody outside OpenAI, Google and Anthropic knows what people actually type into AI tools, and real conversations rarely look like a neat keyword. As Linda Sallai, who co-leads LLM strategy at BrokerChooser, put it, a search used to be "best stock brokers." Now it's something like "what is the best broker for me that also gives fractional shares in India?"

At Ahrefs Evolve, we asked 12 in-house marketers how they handle this. Almost all of them track prompts. Almost none of them fully trust the data.

So if the data is directional anyway, the real question is how you choose prompts that point in the right direction. Here are the five approaches we heard, from the most human to the most technical.

1. Ask your customers how they talk

The strongest answer we heard came from Irina Jordan, VP of Marketing at Checkmate. Her buyers are restaurant executives at brands like Arby's, Wendy's and Buffalo Wild Wings, and they don't phrase questions the way marketers do. So she goes to their conferences and asks what's on their minds, what they're talking about and what triggers a conversation. Then she uses their words as her prompts.

"The curse of knowledge is real. If you make an assumption of what you think they're asking for, you will be so off."

Jonathan Rundle at ChemPoint faces a similar challenge. His buyers are chemical formulators asking very scientific questions: what a chemical will do in a formulation, how to solve a problem, where to source a product. His team builds prompts around those real problems.

Alexandre Teulade at ClearPoint Strategy adds one rule: keep your own brand out of most of your prompts. "Really put yourself into the customer journey, and like, what would someone in our ICP really search for."

2. Start from keywords, then rewrite in a human voice

Several teams start with the keyword research they already have. Linda's team at BrokerChooser takes traditional search queries and uses AI to turn each one into several conversational prompts.

Anton Holm at the International Rescue Committee does the same, building prompts from the IRC's established SEO keywords and priority areas like cash assistance, malnutrition and maternal support. He's fine with AI tools that expand a keyword into a full fan-out of prompts, but he adds a warning:

"Never lose sight of what it takes to create your own prompt and to really put yourself in the mindset of your buyer or your supporter."

Anton also flagged something many teams miss. The IRC runs sites for the US, Korea, Germany, Sweden, the UK and the EU, and AI answers vary by who's asking and where. A prompt that looks great in San Diego may look very different in Seoul.

3. Track topics, not exact prompts

Jordin Frey at U.S. Career Institute stopped chasing individual prompts altogether. "Everyone searches so differently, it's hard to pick a specific prompt and nail it down." Instead, her team tracks coverage of the broader fan-out topics AI pulls from.

Lance Holt at Morning Brew takes a similar approach with a topic matrix he credits to the Growth Memo. One side is the brand's tag taxonomy, or what they cover. The other is personas, or who they're writing for. Together they define the signals Morning Brew wants to send to AI.

4. Prioritize comparison prompts

Some of the most important prompts aren't about what you do. They're about how you stack up.

Dave learned this the hard way. While his company was pitching a project, someone asked an AI tool which of two competing vendors would be better, and it picked the competitor. "So from a prompt standpoint, we're going to be looking at how do we compare against our competitors."

Will Heany at Snowflake sees the same urgency. Head-to-head prompts are often "the final prompt that someone throws into Claude before they decide to make that buying decision."

Eden's reporting shows NRG's strengths and weaknesses in the eyes of AI next to those of competitors X, Y and Z, because "it's no longer about my website. It's about reviews and social and Reddit and everything that is on the internet."

5. Build your own system

The most technical teams are building their own prompt infrastructure.

Eden spent about 100 hours using an AI coding agent to write roughly 80,000 lines of Python. The result is an automated, interactive reporting suite built on Ahrefs Brand Radar data that anyone at NRG can open. Will's team is building prompt tracking inside Snowflake's own data warehouse, which they find "a lot more trustworthy than relying on someone who paid for prompt volume." BrokerChooser built a proprietary tool to measure how often it's cited.

At the other end, Stefanie took a simpler route. Her agency built a small baseline set of generic, brand and location-specific prompts, so the team can see whether its website work moves the needle.

How we'd build a prompt set

Pull the best of each approach together and you get a prompt set that's small, grounded and useful:

  1. Start with customer language. Mine sales calls, support tickets, reviews and conversations with real buyers. Our team found the same thing Irina did: talking to your audience is still the best prompt research there is.
  2. Map it to topics and personas. Use Lance's matrix: what you cover on one axis, who you're writing for on the other. Track coverage across the grid, not just a handful of exact phrases.
  3. Keep most prompts unbranded. You want to know whether you show up when buyers don't already know your name.
  4. Add your money prompts. Include head-to-head comparisons and "best X for Y" questions for your top competitors. Check them often.
  5. Test by market. If you sell in more than one country or region, check how answers differ by location.
  6. Use the data for direction, not promises. Watch trends, gaps and how AI describes you. Report revenue as the outcome.

For more on putting your audience first, read Want to know how to do keyword research in the AI era? Know your audience. For how to track what you find, see how to track brand mentions in AI search.

Better questions beat more prompts

The teams we talked to have tools that can track 25,000 prompts. The ones getting the most out of them aren't tracking the most. They're tracking the right ones: the questions their real buyers ask, in the words those buyers use.

No tool can hand you that list. Your customers can.

Want help building a prompt set grounded in your audience? See our GEO/AEO services or let's talk.

I recently was scrubbing a client site for some outdated information and kept running into issues. I knew it was on the site way more than my crawls were coming back as having found and yet no matter how I changed the settings or my search queries, it wasn’t coming back quite right. When I talked to my team, one of them asked if I was using Javascript rendering and it was an oh duh, lightbulb moment for me (I’m tired guys). Lo and behold, Javascript rendering made a night and day difference. Dozens of pages were found to have the outdated information. I dug deeper for my own learning and found that quite a few pages in a subfolder were about 90% different from just the HTML rendered version of the page. Mindblowing.

And while in the past, we could get away with that much Javascript, though maybe not ideal, in this AI age, Javascript is something we all need to be paying more attention to. 

I recently heard Sam Torres give a talk on Javascript and she has some really amazing insights on JS rendering.

There is a common framework for Javascript. Some elements on your site are critical and must be in the HTML your server sends: the title tag, meta robots, canonical, and structured data. Others, like navigation and headings, should be. And some, including most body copy and the links inside it, are usually fine to render with JavaScript in the browser.

That framework is sound for Google. But the same talk made the case that most AI crawlers never run JavaScript at all. Put those two ideas side by side and something doesn't add up. The content the framework calls "nice to have" is the content AI search can't see, and it's often exactly the content that gets cited.

This post is about closing that gap: why the old priorities made sense, what's changed, and how to re-rank them when AI visibility matters as much as rankings.

Every page has two versions

The single most useful idea in JavaScript SEO is that every URL exists in two forms.

The response HTML is what your server sends back when something requests the page. It's what you see with View Source (Ctrl+U) or a cURL request. Whatever is in it is there before any script runs.

The rendered HTML is the page after the browser executes JavaScript and builds the full DOM. It's what you see with Inspect Element, and it's what a person scrolling your site experiences.

On a well-built site, the two are nearly identical. On a site that builds content in the browser, the response can be a near-empty shell while the rendered page is full of copy, links, and product detail. Every crawler gets the response. Only some of them ever see the render.

Why the old priorities made sense

The critical, important, nice-to-have framework wasn't wrong. It was built for a world where Google was the reader that mattered most.

Googlebot renders JavaScript. It fetches the response first, then queues the page for rendering and processes the full DOM later. That delay is why the critical tier exists: if your canonical or meta robots tag only appears after rendering, Google may act on the wrong version before the render catches up.

Body copy is different. If it's built client-side, Google still sees it once the page is rendered. It might take longer, and it costs rendering resources, but it gets there. So treating body copy as lower risk was a reasonable trade-off when the main reader would eventually run your scripts.

What changed: crawlers that never render

AI search added a set of readers that skip rendering entirely. In December 2024, Vercel and MERJ analyzed AI crawler traffic across Vercel's network and found that none of the major AI crawlers execute JavaScript. That included OpenAI's GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic's ClaudeBot, and PerplexityBot. They do download JavaScript files. They just don't run them.

The exceptions are telling. Gemini relies on Google's rendering infrastructure, and Applebot renders pages with a browser-based crawler. Everyone else reads the response HTML and moves on.

There's a caveat worth being honest about. Some AI answers draw on search indexes that were built after rendering, so client-side content can occasionally surface indirectly. But you can't plan visibility around an occasional, indirect path. For the crawlers feeding ChatGPT, Claude and Perplexity directly, the response HTML is the whole page.

That breaks the assumption behind the nice-to-have tier. Body copy rendered client-side was lower risk because the reader would eventually render it. For these readers, eventually never comes.

Re-ranking the priorities for AI search

When AI visibility matters, the content you'd want cited moves to the top tier. Here's how the familiar framework shifts.

Title, meta robots, canonical, structured dataCriticalCriticalNo change. These were always non-negotiable.
Headings (H1, H2, H3)ImportantCriticalHeadings define the passages AI systems extract and attribute.
Main navigationImportantCriticalIt's how a non-rendering crawler finds the rest of your site.
Body copyNice to haveCriticalThis is the content that gets quoted and cited.
In-content linksNice to haveCriticalThey connect related pages for crawlers that never see the render.
FAQs, accordions, tabs, "load more" contentOften overlookedCriticalThese often load on interaction and hold the question-and-answer content AI answers draw on.
Images and alt textNice to haveImportantAlt text is readable context, and multimodal retrieval is growing.
Interactive widgets, chat, countersNice to haveNice to haveStill fine to render client-side.

The short version: if you want a page to be cited, everything that makes it worth citing needs to be in the response HTML.

Audit by template, not by page

Remember that crawl I mentioned? Where the raw response and the rendered response were completely different?  Results like that are a good reminder of why spot-checking a few URLs isn't enough.

The blog was fine. Most articles gained only 1 to 5 percent more words after rendering, and the gain was almost identical across very different posts. That consistency points to template elements loaded by JavaScript, not article content. The body copy was in the response.

The blog's hub page was not. About 42 percent of its words existed only after rendering. On a listing page, that's typically article cards and links, the paths a crawler follows to find everything else.

The product templates were nearly empty. Location-level pages returned about 9 words in the response HTML and 650 to 1,200 words once rendered. Detail pages under them returned about 62 words and 1,100 to 2,200 once rendered. To a crawler that doesn't run JavaScript, these high-intent commercial pages are shells.

Same site, same domain, completely different visibility, decided entirely by how each template renders. If I'd only checked a few blog posts, everything would have looked healthy. The lesson is to group your crawl by template or subfolder, sort by how much content depends on JavaScript, and look at the outliers.

Agents are a third audience

AI agents that browse on a user's behalf do run JavaScript, because they drive real browsers. They can see the rendered page, click, scroll, fill out forms, and complete transactions.

They fail in different ways. Slow pages cost them time and tokens, and they'll abandon a load that takes too long. Single-page apps that use hash-based routing give them no distinct URLs to work with. Modals, cookie banners and popups get in the way. And bot protection can flag them as bad actors and block them outright.

So a site can be readable by Googlebot, invisible to AI crawlers, and frustrating for agents, all at once. Server-rendered content with fast, stable pages serves all three.

How to check your own site

You don't need special access to see what AI crawlers see. Start with one page from each major template.

  1. Compare View Source with Inspect Element. Search the source for a sentence from the middle of the page. If it's only in Inspect, it's built by JavaScript.
  2. Fetch the page as an AI crawler. Run curl -A "GPTBot" https://example.com/page in a terminal. What comes back is exactly what a non-rendering crawler receives. It also shows whether a firewall or bot rule treats that user agent differently.
  3. Turn JavaScript off and reload. Whatever disappears was never in the response.
  4. Scale it with a crawler. Run a site crawl with JavaScript rendering enabled and compare word counts between the original and rendered HTML. Group the results by subfolder or template and sort by the share of content that depends on JavaScript.
  5. Read your server logs. Check whether GPTBot, ClaudeBot and PerplexityBot are actually requesting your important pages and getting 200 responses. Being crawlable and being crawled are different things.

What to ask your development team for

"Fix our JavaScript SEO" won't get you far. Specific requests will.

The response is the page

Sam closed her talk with a line I keep coming back to: JavaScript isn't the problem, but where your content lives might be.

For AI search, I'd take it one step further. The response HTML isn't one version of your page. For most AI crawlers, it's the only version. If your content isn't in the response, it isn't in the answer.

The good news is that you can see exactly what these crawlers see, and the fixes are well understood. Start by auditing your templates, re-rank what has to be in the response, and bring your developers specific asks.

Not sure what AI crawlers see on your site?

We’ll check your key templates the way GPTBot, ClaudeBot and PerplexityBot do, show you which pages are shells in the response HTML, and give you a prioritized list of fixes your developers can act on. Get your free AI audit →

Measurement continued to be a topic at Ahrefs Evolve after BrightonSEO. Everyone has heard of and is talking about the decreasing line in search console. Organic is down. People have said it’s disappearance has been tucked away in direct. The attribution is messier than ever, but that doesn’t mean outcomes are.

At Ahrefs Evolve, we interviewed 12 in-house marketers about how they're handling AI search. When we got to measurement, a surprising number of them told a version of the same story. Their traffic is down, but the business is fine. In several cases it's better.

The pattern makes sense. AI tools answer a lot of top-of-funnel questions on the spot. The people who still click through have usually already done their research. There are fewer of them, and they're closer to buying.

The problem is proving it. As Dave Kimball, marketing director at Invite Networks, put it: "Attribution is the billion-dollar whale. Everyone's trying to figure that out."

Why the old attribution model breaks

The people we talked to named  a couple reasons AI search is so hard to attribute, outside of the typical conversation of traffic hiding in direct. 

Sales cycles outlast the data. Dave's IT business has sales cycles of one to three years. Tying a first touch in ChatGPT to a deal that closes two years later is close to impossible with today's tools.

The systems don't talk to each other. Eden used the shift in search to push NRG's CMS from on-premises to the cloud, so she could finally connect analytics, journey data and AI visibility in one place. Until that's done, "I can see clicks went down, clicks are now up. I don't know why."

What leadership actually wants to see

We asked every marketer what their leadership looks at when they report on AI search. Seven of the 12 gave some version of the same answer: revenue, pipeline, leads or customers.

Irina Jordin, VP of Marketing at Checkmate, carries a pipeline number herself, so she didn't hesitate: "It is all about SQLs." Leadership asks where deals come from, "but they really don't care how or what. They just want to see the dollar signs at the end." Jonathan Rundle, head of marketing at ChemPoint, said his leadership cares about "revenue, margin impact, leads." Eden's leadership at NRG asks one question: does it add customers or help keep them?

Visibility metrics still have a place, but mostly as supporting evidence. Stefanie Maio at Evans Hotels reports mentions, citations and share of voice, framed explicitly as trends rather than precise numbers. James's leadership watches prompt visibility as a brand-health signal, "but ultimately it's where are the leads coming from and what's the conversion rate."

How in-house teams are connecting the dots

No one we talked to has solved attribution. But the most confident teams are combining several imperfect signals into one story:

Pipeline as the north star. Alexandre's team stopped arguing about which channel gets credit and made "pipeline generated and influenced by marketing" the main goal. They also track new signals, like time spent on money pages and conversion rates on those pages.

AI referral traffic, judged by conversion. James calls this "the metric I can really hang my hat on." The sessions are small. The conversion rate isn't. Jonathan's team is seeing ChatGPT referrals convert too, enough that they're now testing ChatGPT ads.

Self-reported attribution. Irina reads her form fields closely. Leads now name AI search as a source alongside direct and organic. James sees more people saying, "I found you through ChatGPT or Claude or a Google AI Overview."

Context with every number. This might be the biggest takeaway when reporting your success in IA Search to leadership who might understand the full picture. Eden's tip for reporting up: never show an AI metric on its own. When she compared NRG to its top five competitors, she pointed out that AI only recommended a provider in about 100 of 40,000 answers, and a specific mention of NRG was only a handful of times. "Every time you say a number, there's like four lines of context you have to give around it."

One trusted source of truth. Will Heany, SEO manager at Snowflake, has seen two managers send leadership AI numbers that say completely different things. His team is now building prompt tracking inside Snowflake's own data warehouse, which they find "a lot more trustworthy" than outside tools.

Rand Fishkin, who spoke at Ahrefs Evolve on zero-click marketing, recently pointed people to Eduard Blacquière's AI KPI model as "possibly the only thing I've seen comprehensive enough" to show how you're actually doing in AI visibility. It's worth a look if you're building a framework from scratch.

How we'd tell the story to leadership

If your traffic is down and your leads aren't, here's how we'd help you prove it:

  1. Report traffic, leads and revenue together. Put organic sessions, AI referrals, direct traffic and pipeline on the same page. Tracked separately, each looks small. Together, they tell the real story.
  2. Make conversion rate the headline. If AI referrals convert at several times the rate of organic, lead with that, not with session counts.
  3. Add a "How did you hear about us?" field. Make it open text on your high-intent forms. It catches AI influence your analytics can't.
  4. Use first-party AI data where it exists. Google Search Console now has a generative AI performance report for AI Overviews and AI Mode. Start there before buying more tools.
  5. Give every number its context. Explain what a metric can and can't tell you, the way Eden does. It builds more trust than a confident number that later falls apart.

For more on the measurement gap, read Measurement is AI search's biggest unanswered question, and listen to Rand Fishkin and Paxton Gray on Attribution Is Dead. Now What?

Fewer visitors, better buyers

A falling traffic chart used to mean something was broken. Today it often means AI is answering the easy questions, and the people who still show up are the ones ready to buy.

That's not a problem to explain away. It's a story to tell, with the numbers to back it up.

Need help proving what AI search is doing for your pipeline? See our GEO/AEO services or let's talk.

97th Floor CEO, Paxton Gray, and WordPress VIP CMO, Jodi Cerretani, closed out their three-part webinar series on AI discoverability. The final session covers where to invest, how to measure success, and how to prove ROI to your CFO.

Every CFO is asking the same question right now: what is AI visibility actually worth? It’s a fair question for a channel that’s hard to attribute. But the buyers have already moved.

Enterprise AI Discoverability Series

Benchmark how AI sees your brand, build a strategy to strengthen it, and measure the impact.

Buyers Pick Their Favorite Before They Ever Call You

Paxton shared that 68% of Google searches now end without a click. When full AI mode is present, that jumps to 92%. Buyers do their research inside AI, then show up later by typing in your URL or searching your name.

He calls this shift the move from the keyword era to the specificity era. The old customer journey was easy to track. The new one happens out of sight:

In many cases, the sale is made before a buyer ever hits your site.

Budget Pressure Pushes Teams Toward the Wrong Channels

At the same time, investing is getting harder. Marketing budgets are near all-time lows, CMO tenure is shrinking, and many leaders respond by chasing short-term wins.

That pushes money toward paid channels that are easy to attribute. Paxton warned that teams are “at risk of chasing attribution more than actually chasing the audience and going where they are.”

Your Website Is the Lever Most Teams Ignore

Jodi pointed out that brands have more control than they think. AI has to pull context from the web, and “the number one source of authoritative content about your brand and your business is your own website.”

Yet in WordPress VIP’s own survey, only 17% of leaders ranked their website as a top investment priority for the next 12 months. That gap needs to close.

AI Discoverability Assessment

Request a complimentary joint AI discoverability assessment from 97th Floor and WordPress VIP to see how AI currently views your brand and where to improve.

AI AI Discoverability Isn’t a Single Line Item

Paxton doesn’t treat AI search like media spend. It runs on investments you already make.

SEO is the foundation. Ahrefs data shows seven out of 10 pages cited in AI Overviews already rank in Google’s top 100. Google has also confirmed it uses query fan-out, so ranking for related subtopics feeds directly into AI answers.

Brand and PR carry real weight. An Ahrefs study of 75,000 brands found PR and brand building matter even more than backlinks for AI citations and mentions.

Technical work matters more than ever. SEO meant pleasing one bot. Now you’re dealing with many, and each one interacts with your site differently.

His takeaway: “there should be a coordinated investment in each of these existing line items.”

Decide What You’re Measuring For

Jodi made a confession on behalf of marketers. We’re good at measuring things. We’re less good at deciding what we’re measuring for.

Most leaders call AI discoverability a priority, yet haven’t defined what their metrics should tell them. Before you buy a tool or build a template, decide what business question you need answered.

A scorecard to start with

Lagging indicators are the north star metrics to share with stakeholders:

Leading indicators guide your own strategy: AI crawler activity, citations, brand mentions, share of voice, accuracy and sentiment, and branded search.

Set expectations on AI traffic

AI referral traffic is growing fast, but it’s still a small share for most brands. Jodi’s advice is to frame it as “small but mighty.” Research from Microsoft found AI-referred visitors stay longer and are three times more likely to convert than organic visitors.

Check AI crawler access weekly and track progress monthly. Save brand mentions, citations, conversions, and revenue for quarterly stakeholder reports.

The Results Can Show Up Fast

Paxton shared the story of an event and venue management software client. They were the big player in their space, but their pipeline was thinning and their close rate was dropping. AI search showed why. They sat at the bottom for citations and brand mentions.

They used those leading indicators to get leadership buy-in, then invested across brand, architecture, content, and third-party validation. Within three months:

Their presence nearly doubled, and their pipeline impact nearly tripled.

The work pays off in other ways too. AI-ready content doubles as sales enablement. A clearer brand helps everywhere, not just in AI. And it prepares you for a future where AI agents handle more of the buying.

Give Your CFO a Floor and a Ceiling

For Paxton, “It all comes down to ROI.” Jodi shared a two-part model built to pass a CFO’s sniff test.

Sourced ROI is the floor. Count only revenue from AI referral traffic against your AI discoverability investment. In her illustrative example, $250,000 invested returned $400,000 in attributed revenue, a 1.6x return. It’s intentionally conservative.

Influenced ROI is the ceiling. Count revenue from AI referral, organic, and direct traffic. It’s more generous, since much of AI’s impact shows up outside direct referrals.

The real return lives somewhere in between. That framing speaks in the dollars and cents a CFO wants, instead of leading indicators they don’t.

Where to Start if You Can Only Fund One Thing

Tools like Ahrefs, Semrush, and Profound can track mentions and citations, but they rely on sample prompts. Paxton trusts them for movement, not exact volume.

If he could fund just one thing this quarter, he’d start with brand consistency across the web, making sure your messaging is clear and compelling for your audience. Jodi would add an infrastructure audit. The best content in the world won’t help if AI crawlers can’t reach it.

Enterprise AI Discoverability Series

Benchmark how AI sees your brand, build a strategy to strengthen it, and measure the impact.

Last month, we attended BrightonSEO in San Diego. One of the highlights of the conference was hearing from Fili Wiese, an ex-Google Engineer. With his background, he could have talked about pretty much anything related to SEO, and the subject he chose to talk about was how Google crawls your site, giving tips on how to make it more efficient. Here is a full write up on that, where we cover why crawl efficiency matters. This article covers what bots do once they reach your site and what they look for.

You could have the best content ever written. It could be perfectly crafted for a specific audience or persona, answering every question they have in exactly the voice they want to hear. But if bots cannot access and read that content, none of it matters.

Search engines and large language models (LLMs) rely on bots to discover, read and understand what is on your site. If those bots can't get in, there is no mechanism for your content to show up in search results or AI-generated answers. And if it never shows up there, it will never be seen by the users you wrote it for.

So before you worry about anything else, it's worth understanding how bots read your site and what they are looking for when they get there.

How do you make your content accessible to bots?

The first step is simply letting bots in the door. Two things matter most here.

Your robots.txt isn’t blocking content you want found

Your robots.txt file is one of the first things a bot checks when it arrives at your site. It is a type of instruction manual for bots. It tells crawlers which parts of the site they are and aren't allowed to visit. A single disallow rule pointed at the wrong directory can quietly block bots from entire sections of your site, or even the whole thing. Make sure your robots.txt isn't disallowing the content you want search engines and LLMs to find. A technical SEO audit will catch rules like this before they cost you traffic.

Your content is not gated behind form fills, paywalls or other restrictions

If your best content sits behind a form fill, a paywall or any other restriction, a bot hits that wall and stops. From its point of view, the content doesn't exist. If you want content to be discovered, it needs to be openly available on the page.

How can you get your content noticed and picked up by bots?

Once bots can access your content, the next step is making sure they actually find it. There are a few reliable ways to point them in the right direction.

Sitemaps

A sitemap is essentially a map of your site built for bots. It lists the pages you want crawled so search engines and other crawlers don't have to discover every URL on their own. Keeping your sitemap up to date helps bots find new and updated content faster.

Webmaster tools like Google Search Console

Webmaster tools such as Google Search Console give you a direct line to search engines. You can submit your sitemap, request that specific pages be crawled and see which pages have been indexed. They also show you when bots run into problems on your site, so you can fix them.

PR efforts

PR efforts get your brand and content mentioned on other websites, in the news and across the web. Every one of those mentions is another place where bots can come across your name and follow the trail back to your site.

Backlinks

Backlinks are links from other websites to yours. Bots follow links from page to page, so every quality backlink is another path leading them to your content. Backlinks from trusted sites also signal that your content is worth paying attention to.

What are bots doing when they get there?

When a bot lands on your page, it isn't just skimming. It's trying to understand what your content is about and whether it has unique value worth showing to users. Here's what it's looking for.

Semantically relevant terms to a subject

Bots look for terms that are semantically related to the topic you're covering. A page about running shoes, for example, would naturally mention things like cushioning, stride and trail versus road. These related terms help bots understand the full context of your content and confirm that it actually covers the subject in depth.

Unique insights on a topic

Search engines and LLMs already have access to countless pages saying the same thing. What stands out is content that offers a unique insight, a perspective or angle that isn't found everywhere else. Bots are looking for content that adds something new to the conversation.

Proprietary data

Proprietary data is information only you have, such as original research, survey results or internal findings. Because it can't be found anywhere else, it gives bots a strong reason to treat your site as a source worth referencing.

How do we get the bot to spend its time on our site efficiently?

Bots don't have unlimited time to spend on your site. The easier you make it for them to move around and understand how your content fits together, the more of it they'll be able to find and read.

Internal linking strategy

Internal links connect the pages of your site to one another, creating a topic cluster built with supporting content. A thoughtful internal linking strategy guides bots from one relevant page to the next, helping them discover deeper content and understand how your topics relate. It also signals which pages are most important, since those tend to receive the most links. It is impossible to create topical authority on a subject without a strong internal linking strategy.

URL structures

A clean, logical URL structure helps bots understand where a page sits within your site. URLs that follow a clear hierarchy, like /blog/seo/how-bots-read-your-site, tell a bot what the page is about and how it relates to the rest of your content before it even reads the page.

Great content deserves to be found. By making sure bots can access your site, find your pages, understand your content and move through your site efficiently, you give that content its best chance of reaching the people it was written for.

Getting crawled is only the first step. Hear Kevin Indig explain what it takes to get retrieved, cited and trusted in AI search.

There seems to be a pattern in the SEO industry of jumping on new tactics before there's enough evidence to prove those tactics have a clear impact. It's understandable. We need to move fast to generate results quickly. But if you don't take a step back and question tactics, you'll end up wasting time without the results to show for it.

With SEOs being the early owners of AI Search, similar bandwagon patterns have started to show up. Schema has been the biggest hot button of these. Some SEOs call it the foundation of how AI understands your brand. Others say LLMs strip it out and ignore it. And files like llms.txt and markdown versions of pages have only added to the noise.

So at BrightonSEO in San Diego, we asked four experts the same question: how effective are schema, llms.txt, markdown files and other machine-readable formats for AI search, really?

We got four different answers, ranging from "zero impact" to "this is your knowledge graph AI uses to understand your brand." The disagreement is actually surprisingly useful when we break it down. Each expert is looking at a different part of the problem, and together they point to a clear strategy, rather than the confusion it may seem on the surface. 

The four positions at a glance

Thomas Peham, CEO & Co-Founder, OtterlyAIDoesn't work for AI citationsIn OtterlyAI's tests, schema had zero impact on citations in ChatGPT and Gemini.
Chris Long, Co-Founder, NectivNot game-breaking, but do itSchema's value for LLMs is unproven, but it's cheap, and early adopters win if agents adopt llms.txt.
Chima Mmeje, Senior Content Marketing Manager, MozSchema yes, llms.txt noSchema is important; llms.txt won't make the difference people claim.
Martha van Berkel, CEO & Co-Founder, Schema AppEssential, if you rethink itSchema isn't a rich-result checklist. It's the knowledge graph AI uses to understand your brand.

Thomas Peham: "It doesn't work right now for search"

Thomas Peham, CEO of AI visibility platform OtterlyAI, gave the most stark answer (watch Thomas's clip). He separates Google from AI chatbots:

"Schema markup's super important for Google, right? … Not so much on ChatGPT."

OtterlyAI tested it. In Thomas's words, schema markup had "zero, literally zero impact" on citation visibility in ChatGPT, Gemini and other AI search engines in their experiments. 

 Thomas also predicts that within a year, AI Mode will be the default interface on Google.com and SEO and GEO will merge. If that's right, schema's Google value and its AI value may stop being separate questions.

Chris Long: "Not game-breaking" but worth doing

Chris Long, co-founder of Nectiv, lands in the middle (watch Chris's clip):

"I don't think any of them are game-breaking, honestly."

On schema, he notes the research is split. Some studies show it helps and some don't. His lean is that it probably doesn't help much with LLMs, because they tend to strip markup out when they read a page.

But his advice is still to implement it with how much of a low lift it is. Schema, llms.txt and markdown files "take no time to implement," and schema is good technical practice anyway. If AI agents ever adopt llms.txt, early adopters will have one less thing to build. In other words, it's low cost with real upside.

Chima Mmeje: Schema yes, llms.txt no

Chima Mmeje, senior content marketing manager at Moz, splits the question in two (watch Chima's clip). She cited a talk from Moz's event in New York, where a speaker from Meta, quoting someone at Google, said Google isn't using llms.txt when it evaluates website content.

Her verdict:

"Schema is important, don't get me wrong, but llms.txt is not."

She isn't against llms.txt. "Do it, cool," she says. Just don't expect it to deliver the difference people have been promising. There's a lot more you can do to optimize content for AI search that has nothing to do with an llms.txt file.

This matches what our team found in Why llms.txt isn't a magic bullet for AI (but you should probably have one anyway).

Martha van Berkel: Schema is your knowledge graph

Martha van Berkel, CEO of Schema App, does schema markup "all day every day," so her answer isn't a surprise. Her reasoning is the most useful part (watch Martha's clip).

She argues most SEOs still think of schema as a checklist item  to win rich results. That misses the point. Schema is how you are

"connecting the dots across every single thing on your website."

Call it entities, business concepts, a semantic data layer or a context layer: it's the data that helps AI make the right inferences about your brand. Schema App works with more than 100 enterprises, including Wells Fargo, and Martha says they're seeing results in both visibility and conversion rates.

Her challenge to SEOs is to "think like a data architect." Adding markup to a page is easy. Building a real ontology or enterprise knowledge graph is harder and takes more time and effort, but it's where the value is. She points to research from data.world finding LLMs are far more accurate when they work from a knowledge graph. Schema App's free "Schema Markup 101" training is a good place to start getting familiarized with how a knowledge graph might work best for your site! 

She also looks further ahead, predicting AI agents will start acting on sites, booking appointments and filling out forms, and agents will need data they can trust (watch). 

We went deeper on this in Alyssa Felix's Entity mapping and knowledge graphs: the context layer we all need, which features Martha's thinking.

What we recommend

The four answers disagree less than they seem to. Thomas and Chris are measuring whether schema directly raises citations in chatbots today. Martha is describing how structured data helps AI understand who you are. Both can be true. Here's how we advise clients at 97th Floor:

  1. Implement schema, but don't expect a lift in individual LLM citations. It's low effort, it still matters for Google, and Google's AI features are becoming the default. Just don't expect a ChatGPT boom from markup alone.
  2. Build for entities, not rich results. Use schema to connect your organization, products, people and locations: sameAs, parent and sub-organizations, breadcrumbs. Consistent, connected data is what keeps AI from confusing your brand with someone else's.
  3. Add llms.txt if it's easy, then move on. It's another low-cost readiness step, especially for sites with docs or APIs, but it's not a priority for most brands.
  4. Treat markup as infrastructure. As our team put it in 97F Reacts: what markup, markdown and schema are really doing for AI search, clean HTML and structured data give content meaning for machines. Whether markdown beats schema is still an open question, so test it, and never hide content from users that you show to bots.
  5. Spend most of your effort where citations come from. Machine-readable files help AI read your site. Getting cited still depends on content worth citing, a site structure AI can follow, and off-site mentions in the sources LLMs trust. Those are the three pillars of our GEO/AEO approach.

Add the schema, then invest in what actually gets you cited

Ask four AI search experts whether schema matters and you'll get four answers. That isn't confusion, though. It shows that schema does a different job than many SEOs expect. It probably won't push you into a ChatGPT answer on its own. It will help machines understand your brand clearly and consistently, and that matters more as AI starts acting on websites, not just reading them.

So implement schema, skip the silver-bullet thinking, and put your real energy into the content, structure and off-site signals AI actually cites.

Want more takeaways from the conference? Read Mike Witham's 45/40/15: What BrightonSEO San Diego taught me about where SEO is headed.

Not sure how AI sees your brand? Let's talk.

If you were given the opportunity to witness a Ferrari being made, given an inside look into making one of the most amazing machines on earth. And then after you got this inside look at how this incredible work of craftsmanship was made, and you then were able to speak at a conference for car mechanics and Ferrari enthusiasts, what would you talk about? You could talk about the hand stitched italian leather, the near perfectly tuned v-12 engine that produces 800+ horsepower, or the aerodynamic design that beautifully mixes engineering and style. There are a lot of factors that make a Ferrari, well a Ferrari. And when you could talk about any of them, would you choose to talk about the sourcing of materials? 

I recently attended BrightonSEO, where we heard from an ex-Sr. Google Engineer named Fili Wiese, who, while at Google, worked on the documentation for manual action penalties. Which means, Fili could have talked about anything related to SEO, but he chose to talk about a subject that, in my opinion, is carried over and will be carried over as a top ranking factor for all of Google's existence: The ability for Google’s bots to crawl, fetch and index content from a site efficiently. 

At first glance, that sounds like the least exciting topic he could have picked. It's like passing over the V-12 to talk about where the aluminum comes from. But that's exactly the point. Ferrari can stitch the finest leather and tune the engine to perfection, and none of it matters if the materials going into the car are flawed. In SEO, your content, your links and your on-page work are the leather and the engine. Crawling and indexing are the materials. If Google can't reliably reach your pages, read them and store them, nothing else you've done gets a chance to count.

Googlebot believes what your server tells it

Fili's session was centered on HTTP responses, the status of a page your server sends back every time Googlebot requests a URL. Most of us only look at these when something breaks: a 404 here, a redirect there. His point was that Googlebot reads every one of them and adjusts its behavior based on what status is returned. Your server is constantly telling Google which pages exist, which have moved, which haven't changed and whether the site can handle more requests. When those messages are wrong, Google acts on the wrong information.

A few common examples:

Crawl efficiency can give you the competitive edge

Google will not spend unlimited time and resources crawling your site and making sure it catches ever page. So it is important to take advantage of every crawl request received from Google and make sure that the Googlebot is seeing only the most important content on the site. Every ranking depends on Google having a current, accurate copy of your page. A product page recrawled two weeks after a price change, a new article that sits undiscovered for days, or a page that keeps dropping out because of intermittent server errors will all lose to competitors whose sites are easier to crawl. That's why I'd argue crawl efficiency has been a top ranking factor from the beginning and will stay one. The algorithms on top keep changing. The need to fetch and index the page first does not, and the same is true for the AI crawlers now feeding AEO strategies.

Where to look on your own site

Fili's advice for finding these problems came down to two sources: Google Search Console and your server logs.

In Search Console, the Crawl Stats report (under Settings) breaks Googlebot's requests down by response code, file type and purpose, and flags host problems like robots.txt fetch failures and server connectivity errors. A growing share of 5xx or redirect responses, or a sudden drop in total crawl requests, is worth digging into. The Page Indexing report then shows which URLs Google crawled but chose not to index, and why.

Server logs show what Googlebot actually requested, not what you assume it requested. Running them through a tool like Screaming Frog's Log File Analyser shows which sections get crawled most, how often key pages are revisited, and where Googlebot is spending requests on parameter URLs, redirects or error pages. Comparing that log data against a regular site crawl is usually where the surprises turn up: important pages Googlebot rarely visits, and low-value URLs it hits constantly.

Back to the Ferrari

Nobody buys a Ferrari for the quality of its raw materials, but every part of the car depends on them. I often say that you could have the best content in the world, but if Google can’t see it,that content will never be shown in search results, and the chances of the intended user finding it are very low. Your site works the same way. Content, links and brand get the attention, and they should. Underneath all of it is the question Fili spent his whole session on: when Googlebot asks your server for a page, does it get a clear, honest and fast answer? Get that right, and everything else you build has a chance to perform. Get it wrong, and you're fitting hand-stitched leather onto a car with a cracked frame.

97th Floor CEO, Paxton Gray, and WordPress VIP CMO, Jodi Cerretani, teamed up on a three-part webinar series about AI discoverability. The second session covers how to turn your baseline into a strategy AI can understand and recommend.

Knowing how AI sees your brand is step one. Changing what it sees is step two. That's where most brands stall.

The goal is to show up in front of the right people, saying the right things, in a way that grows your bottom line. Paxton called that "the real trick these days." He breaks the work into four parts: brand, architecture, content, and third-party validation.

Enterprise AI Discoverability Series

Benchmark how AI sees your brand, build a strategy to strengthen it, and measure the impact.

Your Technical Foundation Works Like a Library

Jodi explains the technical basics with a library. Information architecture is the floor plan. It shows where everything lives. Structured content is the label on each book. It says what each page is. Governance is the librarian who keeps it all accurate over time.

Most content systems were built before AI search. Her three fixes:

If your website software can't do this automatically, "you might be in trouble."

Being Readable Isn't Being Recommended

As Jodi put it, "being readable isn't the same thing as being recommended." That's the job of content.

SEO meant getting one pillar page to rank. Paxton explained that "AI is much more holistic and it should be focused on topics and establishing authority around certain topics."

The biggest mistake he sees is treating prompts like keywords. Prompts are longer and more detailed, and AI runs several searches off each one. He calls this the query fan-out. "The key is answering these secondary questions through multiple pieces of content on our site, not simply on the single page."

That starts with audience research. Scour Reddit for the words and problems your audience shares. Map your personas, their pain points, and the questions they ask. Then build a pillar page with focused supporting pages around it.

Your Old Content Is Teaching AI the Wrong Things

Enterprise brands carry decades of content. Much of it mentions products they no longer sell or positioning they've moved past.

Paxton's team runs clients through a content analysis matrix: Is the page pulling traffic? How does it perform? Does it align with the business? A page can perform well and still point the wrong way. "That page should not exist and it is steering AI in the wrong direction."

He shared the example of a cruise line client that wanted to own Alaska. His team cut irrelevant content, then built focused pages on ports, wildlife, excursions, and timing, all mapped to the customer journey. Now, "They're the most cited cruise line in the world for anything related to Alaskan cruises."

Off-Site Validation Shapes What AI Believes

Paxton was clear that "Your site is one of the biggest factors that AI uses to determine who you are and when they should bring you up." But AI learns from the rest of the web too.

A brand sentiment analysis shows what AI thinks about you and where it learned it. Key sources vary by industry. For one client, Jazz HR was the third most cited site in its space. Enterprise brands also have partner marketplaces, which AI search currently treats as third-party validation.

Reflecting on his point about pruning, Jodi added, "more is not more when it comes to content. It's about clarity."

Quick Wins Come Before the Rebuild

Jodi splits the work into two buckets.

Quick wins: fix wrong tags on key pages, unblock AI crawlers locked out by accident, patch sitemap holes, and clean up conflicting messaging.

Foundational work: rebuild your tagging system, make sure your site technology automates what it should, build authority on your core topic, and write real governance rules.

AI Discoverability Needs One Captain

Marketing, content, SEO, web, engineering, security, and governance all play a part. What's usually missing, Jodi said, is one directly responsible individual to quarterback the work. A live poll showed just 5% of attendees had a clear owner in place.

The pressure is coming from the top. Boards see organic traffic falling, and their message to CMOs is simple: "You got to win your traffic back."

Paxton sees measurement as the industry's biggest bottleneck. "It's one thing to know that you need to do something and it's another thing to know whether or not what you're doing is having an impact."

Infrastructure and Content Only Work Together

Jodi closed with the core idea. Strong infrastructure without content makes you readable but not recommended. Strong content without infrastructure is "a great story that no machine can read."

You need both to get cited and recommended. Watch the full recording now.

At BrightonSEO San Diego, our team asked some of the sharpest people in AI Search a simple question: what's YOUR biggest unanswered question about AI search right now?

We expected a spread of answers: agents, Google's next move, the future of the website. What we got was a pattern that was so stark it even felt repetitive by the day's end. Five of the eight leaders who answered said some version of the same thing: we still can't measure this well.

Some framed it as a prompt tracking problem. Nobody knows what people actually type into ChatGPT, Gemini or AI Mode, so nobody fully trusts the visibility scores built on top of those prompts. Others framed it as an attribution problem. Even when you show up in an AI answer, connecting that to pipeline and revenue is murky. 

That matters, because measurement is what earns a budget and internal support. If you can't show what AI search is doing for the business, you'll struggle to keep investing in it. Here's what we heard, who said it, and what we think you should measure instead.

Enterprise AI Discoverability Series

Benchmark how AI sees your brand, build a strategy to strengthen it, and measure the impact.

Who we asked

I specifically interviewed the folks that I felt have been leading the AI Search conversation online, as well as the companies that have been diving in and experimenting the most. We do this often with our guests on our podcast, The Campaign. Kevin Indig came on with us and expressed a lot of the same concerns we heard at Brighton. Here was the lay of the land of folks responses to what is still unanswered for them in AI Search:

ExpertRoleTheir Unanswered Question
Brooke WellerAI Search Expert, LinkedInCan anyone accurately track prompts?
Chris LongCo-Founder, NectivWhat are people actually typing into AI search?
Ken MarshallCo-Founder, Meet SonaHow do prompt-tracking tools build their data?
Lucy HoyleFirst Content Engineer, CartaHow do we tie AI visibility to revenue?
Nick LaffertyFounding Marketing Engineer, ProfoundHow do we attribute AI-driven demand, and how does the organic listing interplay with ChatGPT ads?
Martha van BerkelCEO & Co-Founder, Schema AppWhat happens to the role of the website in the future?
Skyler RudolfskyGTM, SemrushHow will AI platforms monetize in the future?
Thomas PehamCEO & Co-Founder, OtterlyAIWhy do citation sources suddenly shift?

Chris Donnelly, CEO and Co-Founder of Searchable, mentioned the Prompt Universe, Searchable’s new product offering, as a way to answer the questions of, “What prompts am I tracking? Are they the right prompts? Am I tracking enough prompts?” While Searchable, Profound, and Scrunch are growing tools, trust across the board of their validity doesn’t seem to be strong in the SEO community.

Problem 1: People aren’t trusting prompt tracking just yet, so stick to knowing your audience

In traditional search, Google Search Console tells you which queries you showed up for. AI Search has no equivalent. That gap is what most of our experts are stuck on.

Chris Long, co-founder of Nectiv, put it plainly (watch Chris's clip):

"No one knows the prompts people are putting into AI search systems."

He isn't optimistic it gets solved soon. Every tool has its own method for building prompt sets, and none of the major AI platforms share real query data. Unless a data provider opens it up, he doubts there will ever be a great answer.

Brooke Weller, who leads AI search at LinkedIn, sees the same thing from inside a major brand (watch Brooke's clip):

"I don't think anyone can accurately track prompts."

Even at LinkedIn, she reports an AI search visibility score she isn't sure is accurate. She also points out that AI conversations are personal and long-running. She has threads that go on for months, so no single "first prompt" captures how people actually reach a brand.

Ken Marshall, co-founder of Meet Sona, wants more transparency from the tools themselves (watch Ken's clip). Many visibility platforms don't publish how they choose prompts or model personas.

This is a human element of search that still has to be in place from human marketers in order for a brand to achieve visibility and recognition. Alyssa Felix, one of our senior search marketers, put it plainly, “The best prompt research tool is still talking to your customers, reading their questions, and paying attention to what they actually say.”

Problem 2: Proving AI search drives revenue

The second half of the problem starts after a brand shows up in an AI answer. What did that appearance actually do for the business?

Lucy Hoyle, founding content engineer at Carta, named the pressure many marketers feel (watch Lucy and Nick's clip):

"Attribution is still really difficult. Visibility is one piece and it's very top of funnel, but how does that then track to engagement, revenue?"

Marketing teams are being asked to generate pipeline and prove ROI, and AI visibility on its own doesn't do that.

Nick Lafferty, founding marketing engineer at Profound, shared one low-tech fix that works. Profound's contact form forces every lead to write, in an open text field, how they heard about the company before being able to navigate further. Self-reported attribution catches the AI answers and conversations that tracking can’t uncover. Nick pointed out that this  kind of first-party signal "will become really foundational" for brands if they do want to have somewhat of an answer for attribution as it gets messier and messier. 

This matched what our own Alyssa Felix heard at the BrightonSEO AEO/GEO roundtable. For in-house teams, she said, "the big question right now is reporting and attribution": proving organic still works, and proving the team behind it still matters. The buzz phase of AI Search is over. Executives know it’s important, but HOW important is starting to be questioned and the level of investment will be determined by how much attribution can be shown. 

Rosy Callejas, who works on enterprise SEO and AI search at Microsoft, offered the practical takeaway in approaching executives about AI Search now. (watch Rosy's clip). To win trust from leadership, bring "ANY data to show them," because "we're all data driven at this point." Something is better than nothing! 

The other three answers

The three experts who didn't name measurement still raised questions worth tracking:

Even these answers come back to measurement. If citation sources can swing overnight, you need reporting that catches the swing and shows what it cost.

What to measure while prompt data stays murky

Perfect prompt data may never arrive. That's no reason to fly blind. Here's the approach we use with clients at 97th Floor:

  1. Start with first-party data. Before you buy another tracker, use what the platforms already give you. Google Search Console now has a generative AI performance report that separates AI Overviews and AI Mode impressions from regular search. Pair it with GA4 referral traffic from AI platforms and Bing Webmaster Tools.
  2. Track citations, not just mentions. A mention tells you AI named your brand. A citation tells you which of your pages informed the answer, and you can actually optimize that page. Track citation sources, share of voice and your top cited pages. Our guide on how to track brand mentions in AI search walks through it.
  3. Treat prompt tracking as directional. Use visibility scores to spot trends and gaps, not as a KPI you promise leadership. As Brooke put it, keep your eye on "the bottom line."
  4. Add self-reported attribution. Take Nick's advice and put an open "How did you hear about us?" field on your high-intent forms. It's the cheapest AI attribution tool there is.
  5. Connect visibility to pipeline. Report AI search alongside sessions, leads and revenue, not separately. That's how measurement goes from "woo-woo" to budget.
  6. Watch what AI says, not just whether it says it. Accuracy and brand sentiment are next. A citation that describes your product wrong can do more harm than no citation at all.
  7. Don’t settle thinking you know your customer base. Our GEO/AEO team tracks the KPIs that matter to the business, and centers on getting to know your audience in finite detail, as it’s the best source for prompt research. Talk to us to learn more about how we map your personas in order to capture the right volume in AI Search. 

Build your AI search scoreboard before leadership asks for one

AI search isn't missing strategies. It's missing a scoreboard. The experts we talked to at BrightonSEO know how to earn visibility in ChatGPT, Gemini and AI Mode. What they can't do yet is say, with confidence, which prompts drove it and what it was worth.

Until the platforms open up their data, the teams that win will be the ones that combine imperfect signals: first-party platform data, citation tracking, self-reported attribution and pipeline. Then they bring those numbers to leadership before anyone asks.

For more from the conference, read Mike Witham's 45/40/15: What BrightonSEO San Diego taught me about where SEO is headed.

Ready to prove what AI search is doing for your business? Let's talk.

97th Floor CEO, Paxton Gray, teamed up with WordPress VIP CMO, Jodi Cerretani, on a three-part webinar series about AI discoverability. The first session covers how AI sees your brand, where visibility gaps show up, and five asks to help your team set a baseline.

Search has changed more in the last two years than in the previous twenty. Paxton put it plainly: "there's not been a bigger update than what's happened with AI search." The goal used to be ranking in a list of links. Now the goal is to get recommended.

Buyers have already moved. According to G2, half of all B2B buyers start their research in AI chat. Two-thirds say AI changed their buying decision, and 80% say it made that decision faster.

If AI is shaping the shortlist, you need to know what it says about you.

Enterprise AI Discoverability Series

Benchmark how AI sees your brand, build a strategy to strengthen it, and measure the impact.

You Can't Fix a Gap You Haven't Measured

Paxton starts every AI search strategy with a baseline. "It's really hard to know what to do unless you know where you are."

He shared the example of a client in venue management software. They were a major player that wanted to move down market and win smaller venues. AI saw them differently. It called them the IBM of venue management software: well established, but clunky, hard to use, and not built for small venues.

That was the opposite of where they wanted to go. It was also a clear starting point. Find the gap first. Then build the strategy.

AI Reads Everything You've Ever Published

Jodi calls it the total information environment. AI doesn't just read your current site. It reads the 2019 microsite that's still up, legacy subdomains nobody sunset, outdated product PDFs, press coverage, and Reddit threads.

Her team evaluated one brand across six of its sites. AI comprehension ranged from 88% down to 59%. AI averages everything it finds, and as she put it, "an averaged brand is a diluted brand."

Ask #1: Inventory every property. Make sure every domain, subdomain, microsite, and PDF tells the same current story.

AI Reads Structure, Not Design

AI crawlers are fast but literal readers. "It does not care about your beautiful design. It reads structure," Jodi said. That means clear headings, semantic HTML, and schema.

In one WordPress VIP assessment, a brand had 16 product pages with no markup at all. They looked great to humans and were hard for models to decipher. Your website now has two audiences, and both need to understand it.

Ask #2: Get a structural audit. One H1 per page, a clean hierarchy, and schema on every page. It's back-end work, but it doubles as an accessibility win.

Live Content Can Still Be Invisible

For one pharmaceutical customer, WordPress VIP analyzed 28 million requests of server traffic. AI crawlers saw just 2% of what humans saw, and only 0.7% on product pages. One set of pages drew 233 human visits and six from AI crawlers. "That's not 6%, that's six crawlers."

The fix was simple. The pages weren't on the sitemap, and their content was locked in JavaScript, which shows up as empty boxes to most crawlers. Content can be live, on brand, and well written, and still be invisible to AI.

Ask #3: Audit your sitemap and add text fallbacks behind every JavaScript component.

Winning AI Search Starts With Your Audience

Once AI can read your site, the question becomes what your content says.

Traditional SEO starts with keywords. AI search starts with prompts, which create a long tail of specific questions. AI often runs several searches off one prompt. Paxton explained what that means: "the more you can answer all of those secondary query fan-outs, the more likely you are to show up in AI search."

That puts customer research at the center of content strategy. Be clear on who you're for, when your solution fits, and what questions your audience asks.

AI Search Needs New Metrics

Paxton tracks four areas: AI referral traffic, brand mentions, citations, and competitive comparison. He stressed that last one: "It's not enough to just look at these metrics in a silo."

Mentions, citations, and share of voice are leading indicators. The lagging indicators follow: more branded search, more direct traffic, and faster conversions. Buyers now do their research inside AI, so "they're coming ready to buy."

Your Dashboards Can't See AI Crawlers

Most analytics run on JavaScript, and AI crawlers don't execute it. That means one of the fastest-growing readers of your site is invisible to your dashboards. Jodi was direct: "There's really one source of truth, and that's in your server logs."

One global enterprise had a compliance plugin rejecting about 15% of all ChatGPT requests, roughly 68,000 in the period measured. WordPress VIP even found its own server layer was blocking one of Anthropic's crawlers.

Ask #4: Ask your team one question. "Show me AI crawler traffic in our server logs."

AI Visibility Needs an Owner

Echoing a point Paxton has made before, Jodi warned that AI models are "very impatient and has huge recency bias." Stale content falls out of answers fast. Baseline, fix, retest, and report on a regular cycle. WordPress VIP does it about every 90 days.

The work is spread across developers, security, WebOps, and brand. No one owns it by default.

Ask #5: Name an owner with the authority to pull those teams together.

The Brands That Move First Will Win

Paxton sees a familiar pattern. SEO demand used to start when the CEO Googled the company and it didn't show up. Now the CEO asks a chatbot, and the brand isn't there. Jodi added that it's become a board-level conversation.

Proving impact is still hard. "There's a lot of correlation and not causation," Paxton admitted. But waiting is the bigger risk. "The ones that move first are going to be the ones that win." Watch the full recording now.

I spent the first morning of brightonSEO San Diego in sessions on keyword and prompt research in the AI era. Good talks, smart speakers, full room. And about twenty minutes into the first one, I caught myself thinking: wait… isn't this just what we've been doing? For a moment, I was worried I wasn't going to learn anything new. But after sitting with what was talked about, I realized that's kind of the whole point. It's really not anything new if you've been doing marketing, especially search, already.

I did learn plenty in the other sessions, a lot of it technical. But the reassurance that this part isn't new if you're doing it right? That was its own kind of lightbulb moment. 

Back to the basics

The pressure to understand AI search and how it's changing, well, everything, is omnipresent. It feels like everyone is talking about it anytime you open LinkedIn or hop on a client call. And it's led to us all looking for the shiny silver bullet that fixes everything. Which is completely understandable. No one really wants to hear that there isn't an easy answer to AI search. It's just doing the work of knowing your audience.

And the work, at its core, is the same stuff it's always been: knowing who you're talking to, what they're trying to solve, and where they're talking about it. That shiny new AEO silver bullet everyone keeps talking about on LinkedIn is a distraction. It pulls our attention away from the one thing that has always held up, which is understanding the people we're trying to reach.

What the talks actually said

Over and over, speakers landed on some version of the same idea: start with the customer, not the tool. One session put it plainly that volume is not a strategy. To be fair, a lot of old-school keyword research was volume-first,and/or built around competitor gaps. And in my experience, every time I've leaned on that over real audience research, it hasn't worked as well. The version of keyword research that holds up has always been the one rooted in people.

Another session walked through how a team approached a brewery brand by going to the client's location and learning the customer through data and conversation and drinking lots of beer, thinking in terms of moments rather than keywords. A moment is the thing happening in someone's life that leads to the search. For the brewery, that meant birthdays, graduations, retirements: the occasions their beer shows up at. For my client, it's the problem someone is stuck on and can't solve alone, plus whatever led up to it.

They were describing audience research, and doing it well. That's the audience-first keyword research we've always done. At the end of the day, it's knowing your customer and what matters to them.

What knowing your audience looks like in practice

I've seen this a lot lately with a SaaS client of mine. They have had a lot of new features drop recently and to be honest pinpointing exact search volume is not easy. Search the feature name or the branded term and you'll find close to no demand, which makes sense: the feature is new, so nobody has the vocabulary for it yet.

If I were taking a volume-first approach, I would stop there and call it a dead end. But the demand isn't missing, I know it's there. It's just not shaped like a keyword yet. People are out there on the world wide web shouting about issues and describing the problems these features solve in their own words. The pain has existed long before the product language does. My client has just finally given it a name and a solution. So in doing research, the keywords didn't exist, but the problems definitely did.

So the work isn't finding the keyword. It's finding where your audience is having these conversations, learning how they describe the problem, and meeting them there. For me, a lot of the time that happens on Reddit or other online communities. I'll read whatever I can to put myself in the audience's shoes, because it's the closest thing to hearing them describe the problem unprompted.

For this particular client, it led me toward adjacent, jobs-to-be-done-style research around a new feature. Instead of stopping at no volume around head terms, I dug into the job someone is trying to get done with this feature or the problem they’re trying to solve or the outcome they want. That led to finding other signals that indicate demand, like how many times something gets downloaded that was meant to solve this problem. Or how often a particular workflow or template get used or shared as a solution. All of these are audience insights that showcase true demand.

What is actually different when it comes to keyword research

We do know that some things in search and audience behavior have changed. Queries are longer and more conversational, prompts carry more context from the searcher, visibility looks different (citations, mentions, sentiment… instead of just rankings), and measurement is messier and still being figured out.

Those are all real challenges and issues in today’s search landscape, and they change how we track and deliver the work, which makes the work we do feel different. But at the end of the day, that doesn't really change how we decide what matters. That still starts with empathy: knowing your audience inside and out.

The takeaway

If you already do audience-first research, you're not behind. You're probably closer than you think, and the move is to point the work you already do at new surfaces.

And if you haven't been doing that, the best prompt research tool is still talking to your customers, reading their questions, and paying attention to what they actually say.

It's not groundbreaking. It still works.