Home / The AI Visibility Gap: A Diagnostic Framework and Open Research Protocol for AI-Mediated Commerce Discovery

Cesar Beltran, Co-Founder & CTO, and Victoria Beltran, Co-Founder & CEO, Blackbelt Commerce.

Executive Summary

AI-mediated shopping replaces a single ranking event with a sequence of machine decisions: a system discovers a merchant, retrieves candidate evidence, interprets what that evidence represents, attributes it when attribution is due, weighs it against a buyer’s constraints, and either recommends it or does not. Each step can fail independently, and each failure calls for a different fix.

This paper does two things. First, it lays out a diagnostic framework for locating where that sequence breaks, grounded specifically in how Shopify’s agentic-commerce infrastructure works today — not in generic AI-search language. Second, and more importantly, it proposes an open, replicable protocol for testing the framework against real AI systems, because a framework without a way to be tested or falsified is an opinion, not evidence.

We want to be direct about scope. The diagnostic model draws on established information-retrieval vocabulary (discovery, retrieval, citation) and on current Shopify, OpenAI and Google documentation — it organizes existing mechanics rather than discovering new ones. The protocol in Section 7 is the part we believe is genuinely new: a concrete, disclosed method for measuring AI visibility that other practitioners can run, critique and improve. We are running it against our own client base and will publish dated results as a follow-up, including any results that don’t flatter the framework.

1. Why Visibility Is a Sequence, Not a Rank

A merchant no longer competes for a single keyword position. It competes to supply reliable evidence at several distinct points in a machine-mediated decision: whether the system can reach the information at all, whether that information gets pulled into the candidate set for a given question, whether the system represents it correctly, whether it credits the source, whether the product qualifies as a real option, and whether it survives the buyer’s actual constraints well enough to be suggested.

We call the sum of this Decision Coverage: the degree to which a company supplies credible, accessible evidence for the decisions its customers are actually trying to make. It is broader than keyword coverage because it includes product facts, comparisons, proof, policy, availability and commercial fit — and more content is not automatically better. The information that reduces uncertainty for a real buyer is what matters.

2. The Diagnostic Framework

The framework below is a diagnostic lens, not a claim about the internal architecture of any AI platform. Its value is in separating failure types that are often lumped together and treated with the same generic “GEO” fix.

Six points of failure

Discovery — can the system access the relevant information at all? Usually a crawl, index, or feed-access problem.

Retrieval — does the information enter the candidate set for the question? A relevance and evidence problem.

Understanding — can the system interpret the entity, product and claims correctly? A data-model and clarity problem.

Citation — is the evidence visibly attributed when attribution is relevant? An authority and structure problem.

Consideration — does the brand or product become a viable option? A fit-and-proof problem.

Recommendation — does the option survive the buyer’s constraints well enough to be suggested? Often a product, price, availability or offer problem — not a content problem at all.

This decomposition is not proprietary; it follows discovery/retrieval/citation vocabulary already used in information-retrieval and RAG literature, applied here specifically to commerce decisions. We’re naming that lineage rather than claiming to have invented the terms — the useful contribution is applying it consistently to Shopify’s actual infrastructure, not the vocabulary itself.

3. Ranking Is Not Retrieval, Citation or Recommendation

Ranking orders candidates within a particular system or surface. Retrieval selects evidence or candidates for a task. Citation assigns visible attribution. Recommendation selects an option for a decision. These events overlap but are not interchangeable, and conflating them produces bad measurement.

Shopify’s own tooling makes this concrete: merchants can inspect raw Shopify Catalog search behavior through Catalog previews, but individual agentic storefronts — ChatGPT, Copilot, Perplexity, Google AI Mode — may re-rank or filter those candidates using their own logic once retrieved. A Catalog position is not a universal ChatGPT rank, and reporting it as one is a measurement error, not a rounding error.

The working rule: always name the surface and the specific event being measured — Catalog retrieval, a specific model’s citation, a specific model’s recommendation — rather than a single undifferentiated “AI visibility” number.

4. What Actually Changed Under the Hood: Shopify’s Agentic Infrastructure

This is the section most existing “AI visibility” commentary gets wrong or leaves stale, because the infrastructure moved fast in 2026. As of this writing, the following is verified against current Shopify documentation:

Shopify auto-generates agent-discovery endpoints on eligible storefronts without merchant action: /agents.md, /llms.txt, /llms-full.txt, /.well-known/ucp, and an agentic sitemap. As of the May 2026 rollout, /agents.md is the canonical discovery document; /llms.txt and /llms-full.txt redirect to it by default unless a merchant supplies its own agents.md.liquid template.
Beyond static discovery files, Shopify now exposes a transactional layer through the Universal Commerce Protocol (UCP) and UCP-compliant MCP servers — letting a compliant agent search the Catalog, build a cart, and walk a buyer to checkout programmatically, not just read a text file about the store.
Shopify Catalog remains the primary distribution layer for product data — title, description, options, images, price and availability — to participating AI channels, refreshed continuously so pricing and inventory stay current across every channel it feeds.
Catalog Mapping matters specifically for stores whose real product data lives in metafields, metaobjects, or non-standard title/description patterns — by default, custom metafields are not passed to AI agents, and Shopify’s own guidance is to leave the product-category source at its default and only remap title and description deliberately, since remapping category risks breaking the underlying category linkage.
Channel behavior differs by platform and is still evolving: ChatGPT currently functions primarily as a discovery channel with checkout completed on the merchant’s Shopify store; Microsoft Copilot supports a Shopify-powered direct-checkout path for eligible merchants; Google AI Mode and Gemini distribution runs through the Google & YouTube sales channel and was still in early access as of mid-2026.

Source snapshot

Platform facts above were checked against Shopify Help Center and Shopify Dev documentation in August 2026. This is a fast-moving layer — treat any AI-visibility content, including this paper, as a dated working map rather than a permanent architecture, and re-verify before republishing.

The Commercial Truth Principle follows directly from this: every machine-readable representation of a product — storefront page, structured data, Shopify Catalog record, merchant feed — should describe the same underlying commercial reality. Inconsistency across these surfaces is now a measurable interpretation risk, not just a data-hygiene nice-to-have.

5. Structured Data: Model What’s True, Not What’s Impressive

Structured data should accurately model what the customer can already see: Product, Offer, variants or groups, Organization, and supported shipping or return terms where relevant. The goal is accurate machine interpretation, not the largest possible schema graph — markup should agree with visible content, Shopify data and merchant feeds.

This is worth stating plainly because it cuts against a lot of current GEO advice: Google’s own May 2026 guide to optimizing for generative AI features on Search states directly that structured data is not required for generative AI search, that there is no special schema.org markup to add, and that merchants don’t need llms.txt, content chunking, or AI-specific rewriting to be eligible for AI Overviews or AI Mode. Google’s generative features run on the same core ranking and quality systems as traditional Search — a site that ranks well there is already better positioned for AI surfaces. A special AI-only schema layer invented to satisfy a GEO checklist is not just wasted effort; it runs against what the platform has explicitly said.

On the OpenAI side, the mechanism is structurally different from crawling: ChatGPT shopping results are not indexed from the open web at all. Merchants push a structured product feed — JSONL, CSV or Parquet — directly to an OpenAI endpoint under the Product Feed Spec, with required fields for identifiers, pricing, availability and media, and optional fields covering reviews and performance signals that improve ranking and relevance. This means a Shopify store’s ChatGPT visibility depends on Shopify Catalog syndication (or a direct feed) being complete and current — not on structured data or schema markup on the storefront page at all. Conflating the Shopify Catalog / OpenAI feed mechanism with schema.org markup is a common and consequential error we see in GEO proposals.

6. The Decision Evidence Rule

If a fact could materially change whether a qualified customer buys, that fact should exist in clear, accessible, reliable information — not be implied by a lifestyle description. For luggage that means dimensions, weight, wheel type, warranty and airline compatibility; for electronics, ports, power requirements, compatibility and battery life; for a service business, expertise, scope, process and verifiable outcomes. A concise product record that answers the buyer’s actual decision beats a long page that doesn’t.

7. A Testable Protocol for AI Visibility

Frameworks in this space are cheap to publish and hard to falsify — our review of current published “AI visibility” material found many structurally similar funnels, almost none of them accompanied by a disclosed method for testing the claims against live systems. That gap is the actual opening. Below is a minimum protocol we’re committing to run and publish results from, and that we’re publishing openly so others can replicate or contest it.

7.1 Build a fixed, disclosed query set

Draw queries from real buyer decisions, not brand-name prompts — “best carry-on under 20 lbs with a lifetime warranty,” not “tell me about [Brand].”
Fix the set before testing (e.g., 30–50 queries across 5–10 product categories) and publish the exact list, so results are reproducible rather than cherry-picked after the fact.

7.2 Operationalize every stage of the framework

Discovery = does a direct fetch of the product URL, Catalog record, or feed entry succeed and return complete data, checked independently of any AI platform.
Retrieval = does the product or brand appear anywhere in the model’s raw candidate set or tool calls for that query, where inspectable (e.g., Shopify Catalog search previews).
Citation = is the source visibly attributed in the model’s answer, with the specific model, date and query logged.
Consideration = is the product named as an option, even without being the top pick.
Recommendation = is the product the system’s suggested choice for that specific query and stated constraints.

7.3 Log and disclose honestly

Record platform, exact query text, date, and a reference to the raw response — not a paraphrase — for every test.
Report null and negative results alongside wins. A protocol that only publishes favorable outcomes isn’t evidence, it’s a highlight reel.
Re-run the same query set periodically and disclose drift — these systems change weekly, and a one-time test is a snapshot, not a claim about ongoing performance.

We do not yet have applied results to publish. What we’re publishing here is the method, openly, ahead of the data — on the view that a disclosed, testable protocol is worth more to the field right now than one more framework diagram. A dated results addendum will follow.

8. A 90-Day Operating Sequence

Days 1–30 — baseline: select priority products and decision queries, verify crawl/index access and Shopify Catalog eligibility, confirm product data, schema, feeds, policies, pricing and availability agree, and record first-party platform data where available.

Days 31–60 — close evidence gaps: improve decision-critical product facts, Catalog Mapping, structured interpretation and internal linking; fix commercial inconsistencies before producing large volumes of new content, since new content on top of bad data compounds the problem.

Days 61–90 — measure outcomes: track retrieval diagnostics, run the query protocol above under controlled conditions, and measure AI referrals, qualified leads, orders and revenue through first-party systems — using controlled observation to investigate why, not just to report what happened.

9. Conclusion

A product can now exist simultaneously as a storefront page, a structured-data entity, a merchant-feed record, a Shopify Catalog record, an AI shopping candidate, and a transaction destination reachable through UCP. The companies best prepared for this won’t build a different version of reality for every machine — they’ll maintain one accurate commercial record that can be represented correctly across all of them.

Visibility in AI-mediated commerce is no longer just the ability to appear in a search result. It’s the ability to stay accurate, evidenced and competitive across every point where a machine makes a decision on a buyer’s behalf — and, we’d argue, the ability to show your work about how you know that’s true.

Related Resources

Methodology and Scope

This paper is a synthesis of current platform documentation and applied Shopify implementation work, organized into a diagnostic framework — it is not an empirical study and should not be cited as one. Where an internal ranking, retrieval or recommendation mechanism is not publicly documented by a platform, we have not presented inference as platform fact. Platform-specific claims should be re-verified against first-party documentation immediately before republication, since this layer changes on the order of weeks, not years. Section 7 proposes a method for producing the empirical evidence this version of the paper does not yet contain.

Sources

Shopify

“Shopify Catalog and product discovery for agentic storefronts.” Shopify Help Center. help.shopify.com/…/agentic-storefronts/products

“Mapping your product data sources for Shopify Catalog.” Shopify Help Center. help.shopify.com/…/shopify-catalog/default-listing

“Setting up agentic storefronts for stores on the Agentic plan.” Shopify Help Center. help.shopify.com/…/agentic-plan-setup

“agents.md.liquid.” Shopify Dev. shopify.dev/…/agents-md-liquid

“Agentic commerce” (UCP, MCP). Shopify Dev. shopify.dev/docs/agents

“Agentic Commerce: Benefits & How To Get Started.” Shopify. shopify.com/blog/agentic-commerce

“Agentic Commerce on Shopify: How It Works.” Shopify. shopify.com/blog/how-agentic-commerce-works

OpenAI

“Key concepts — Agentic Commerce.” OpenAI Developers. developers.openai.com/commerce/guides/key-concepts

“Product feeds — Agentic Commerce.” OpenAI Developers. developers.openai.com/commerce/specs

“Product Feed Spec — Agentic Commerce.” OpenAI Developers (stable spec version 2026-01-30). developers.openai.com/commerce/specs/spec

“Shopping with ChatGPT Search.” OpenAI Help Center. help.openai.com/…/shopping-with-chatgpt-search

Google

“Optimizing your website for generative AI features on Google Search.” Google Search Central, published May 15, 2026. developers.google.com/…/ai-optimization-guide

“Google Search’s guidance on generative AI content on your website.” Google Search Central. developers.google.com/…/using-gen-ai-content

“A new resource for optimizing for generative AI in Google Search.” Google Search Central Blog, May 2026. developers.google.com/…/a-new-resource-for-optimizing

All sources checked August 2026. This is a fast-moving documentation layer — re-verify every claim above against the live page immediately before republication.

Recommended Citation

Blackbelt Commerce. (2026). The AI Visibility Gap: A Diagnostic Framework and Open Research Protocol for AI-Mediated Commerce Discovery. Blackbelt Commerce Industry Papers, No. BBC-IP-2026-001 (Rev. 3).