---
type: Leaf
title: AI search visibility
description: How AI assistants retrieve and quote content, which machine-readable protocols matter, and what to fix first so your site stays findable.
resource: https://nationaldigital.com.au/headless-cms/omnichannel-content-delivery/machine-readable-delivery/
tags:
  - headless-cms
  - Machine-Readable Delivery
  - AI Search Visibility
  - Structured Data
  - Agent Interoperability
  - generative engine optimisation
  - answer engine optimisation
  - WebMCP
  - llms.txt
  - structured data for AI search
  - Schema.org JSON-LD
  - AI crawlers
  - digital strategy
  - AI search optimisation Australia
timestamp: '2026-09-01T00:00:00.000Z'
language: en-AU
publisher: National Digital
author: Cameron Young
authorRole: CEO
readingTimeMinutes: 6
wordCount: 1252
datePublished: '2026-09-01'
dateModified: '2026-09-01'
pillar: headless_cms
canonical: https://nationaldigital.com.au/headless-cms/omnichannel-content-delivery/machine-readable-delivery/
---

# AI search visibility

How AI assistants retrieve and quote content, which machine-readable protocols matter, and what to fix first so your site stays findable.

**Quick answer:** AI assistants fan a prompt into sub-queries, retrieve against real indexes, filter to roughly fifteen URLs and quote passages - so structured data and passage-level content decide visibility, and WebMCP decides operability.

## About this page

- Written by Cameron Young, CEO
- 6 min read
- Updated 1 September 2026

## How do you make a website visible to AI search and AI agents?

Structure the site so machines can both read and use it: complete Schema.org JSON-LD, content written for passage-level extraction, AI bot traffic actually measured in your logs, and - increasingly - named tools an agent can call directly rather than screen-scrape.

**Additional Context**

Traditional SEO makes a page findable by a crawler. Agent-readable structure decides whether an assistant can quote it, and whether an agent can complete a task on it without guessing which button submits the form.

_Medium confidence · Verified 1 Sept 2026_

*The Retrieval Pipeline*

## First: How AI Actually Searches

An AI assistant doesn't run your query. It takes your question apart.

| Stage | What happens |
| --- | --- |
| **1. Fan-out** | One prompt becomes 5-12 parallel sub-queries. Complex prompts exceed 20, and deep-research modes have been observed running hundreds. |
| **2. Retrieval** | Each sub-query hits a real search index - Google, Bing, or the vendor's own crawl (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot). Roughly 80-100 candidate URLs surface. |
| **3. Filtering** | The pool narrows to around 15 URLs on title relevance, structural clarity and freshness. |
| **4. Grounding** | Narrow verification queries check individual facts before the model commits them to writing. |
| **5. Synthesis** | The model quotes **passages**, not pages. |

One practical note: freshness rules differ by platform. ChatGPT skews hard to the current year, while Google gives last year's content a grace period. Every emerging protocol below is an attempt to make this pipeline cheaper, cleaner or unnecessary.

The filtering stage is where most sites lose. A page that ranks perfectly well for a human search can still be dropped before synthesis because its structure gives the model nothing clean to quote - which is a different problem from the one [preserving search rankings through a replatform](/headless-cms/wordpress-migration-to-headless/seo-preservation) solves, and it needs a different fix.

*The Machine-Readable Stack*

## From SEO to AXO, or Maybe GEO

All of this is so new that there isn't actually an agreed term. GEO (generative engine optimisation), AEO (answer engine, or agent experience, depending who's talking) and AXO (agent experience optimisation) are all in circulation, and Search Engine Land found fewer than a third of practitioners kept their terminology consistent across the year. Google's own 2026 documentation takes the opposite line and says optimising for generative AI features is still just SEO.

Whatever you call it, this is the stack underneath - three decades of telling machines what a site is, and one recent addition that tells them what it can do.

| Layer | What it is | Who actually reads it |
| --- | --- | --- |
| **robots.txt** (1994) | Access rules | Every crawler |
| **RSS / Atom** | Change feed | Aggregators, some AI pipelines |
| **Schema.org / JSON-LD** (2011) | What your things *are* - product, price, author, event, FAQ | Google, Bing, and every LLM retrieval pipeline downstream of them |
| **llms.txt / llms-full.txt** (2024) | A curated Markdown index of your site, plus a full-text version | Coding agents, docs tooling |
| **OKF** (2026) | Portable knowledge bundles: Markdown plus YAML frontmatter | Enterprise agents, internal knowledge systems |
| **WebMCP** (2026) | Structured *tools* your site exposes to agents | ChatGPT's browser today; Gemini in Chrome next |

Jeremy Howard of Answer.AI proposed llms.txt on 3 September 2024 to solve a token problem. Two years on, Google has said no on the record, and no major provider - OpenAI, Google, Anthropic, Perplexity - has committed to consuming it. It remains genuinely useful if you publish documentation or an API, and largely ceremonial if you don't.

*OKF and WebMCP*

## OKF: The Interesting One That Isn't About Your Website

Google Cloud published the **Open Knowledge Format** in June 2026 - v0.1 fits on a page. It formalises the "LLM wiki" pattern: knowledge as a directory of Markdown files with YAML frontmatter, cross-linked into a graph an agent can navigate without a proprietary SDK.

It's deliberately not a runtime, a search index or a model. Think OpenAPI, but for knowledge.

Pairing an OKF bundle to each public page is possible but experimental, and nobody's consuming it that way at scale yet. Treat it as an internal knowledge play with a public option, not a publishing standard - the same way [omnichannel content delivery](/headless-cms/omnichannel-content-delivery) treats a new channel before the audience arrives.

## WebMCP: From Readable to Operable

This is the one that matters, and it isn't SEO at all.

Every protocol above helps an agent *understand* your site. WebMCP lets an agent *use* it. Your site registers named tools - `search_inventory`, `book_appointment`, `check_delivery` - each with a description and a typed input schema. The agent calls them like functions. Your existing frontend code does the work.

The alternative, which is what's happening on your site right now, is an agent taking screenshots and guessing which button submits the form.

### Where It Stands

WebMCP is a W3C Web Machine Learning Community Group draft, authored by Google and Microsoft engineers, announced 10 February 2026. Chrome moved it from a Canary flag to a public origin trial at Chrome 149 in May, running through Chrome 156. Edge has preview support. On 26 August 2026 OpenAI shipped WebMCP support in the ChatGPT desktop browser - the first mainstream agent to consume it - with permissioning built in and confirmation still required for purchases and messages.

### The Mildly Infuriating Part

The API is mid-rename from `navigator.modelContext` to `document.modelContext`. Tools are only discoverable once an agent is already on your page - there's no equivalent of a sitemap yet. And the security model has open questions, including what the spec itself calls the lethal trifecta: private data, untrusted content and an outbound channel in the same session.

Build behind a feature check. Don't bet a quarter on it. But don't wait for it to settle either - Gemini in Chrome is expected to consume these tools, and Chrome plus Edge accounts for the large majority of the browser market. Exposing a tool surface is closer to [API development and management](/platform-engineering/api-development-and-management) than to anything a marketing team has run before, which is worth knowing before it lands on the wrong desk.

*Priorities*

## What To Do, In Order

Sequencing matters more than coverage here. The first three items pay off against retrieval as it works today; the last three are positions taken against where it is going.

1. **Audit and extend your JSON-LD.** Products, articles, authors, FAQs, organisation. This is what actually feeds retrieval today.
2. **Restructure content for passage extraction.** Descriptive titles, one clear answer per section, dated original data.
3. **Check your logs for AI bot traffic.** You cannot manage what you haven't measured, and most teams have never looked.
4. **Ship llms.txt if you publish docs or an API.** Skip the per-page Markdown mirrors - indexable duplicates hurt you.
5. **Pick one high-value action** - search, quote request, booking - and prototype it as a WebMCP tool behind a feature check.
6. **Pilot OKF internally** on the knowledge your own agents keep re-learning.

Items one to three are ordinary technical work that a team already doing [digital strategy](/digital-strategy) can sequence into an existing roadmap. Items four to six are option-taking: cheap now, and awkward to retrofit later if the standards hold.

*Key Takeaways*

## Findable Was the Old Job. Operable Is the Next One

For thirty years the job was making a website findable. The next job is making it operable - and the things worth watching shift from clicks toward whether you are cited at all.

- **Structured data is the layer that actually feeds AI retrieval today** Schema.org JSON-LD is read by Google and Bing, and by every LLM pipeline downstream of them. It is the one item on the list with an immediate, measurable payoff rather than an option on a future standard. _Critical_
- **Models quote passages, not pages** A page can rank well for a human and still be dropped before synthesis because no section of it reads as a self-contained answer. Descriptive headings and one clear answer per section are what survive the filtering stage. _Critical_
- **Most teams have never looked at their AI bot traffic** GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot appear in ordinary server logs. Checking which of them fetch what is a short piece of work that tends to change how seriously the rest of the list gets taken. _Important_
- **llms.txt is useful for documentation, and largely ceremonial otherwise** No major provider has committed to consuming it and Google has declined on the record. Ship it if you publish docs or an API; skip the per-page Markdown mirrors, which create indexable duplicates. _Helpful_
- **WebMCP is worth a prototype, not a quarter** It is an origin-trial-stage W3C community draft with an API rename in progress, no tool-discovery mechanism, and unresolved security questions. One high-value action behind a feature check is the proportionate bet. _Important_

## AI Search Visibility: Common Questions

### Is optimising for AI search different from SEO?

Partly. The retrieval stage still runs against real search indexes, so conventional SEO work continues to matter and Google's own 2026 documentation argues it is simply still SEO. What differs is the filtering and synthesis stages: models quote passages rather than pages, so content structured as self-contained, clearly-headed answers survives where an otherwise well-ranked page does not. The genuinely new territory is agent operability, which is not SEO at all.

### What is WebMCP, and should we build for it yet?

WebMCP lets a site register named tools - a search, a booking, a delivery check - each with a description and typed input schema, which an agent calls like a function instead of screenshotting the page and guessing at the interface. It is a W3C community group draft in Chrome origin trial with Edge preview support, and the API is mid-rename. A proportionate response is prototyping one high-value action behind a feature check, not committing a quarter of engineering time.

### Do we need an llms.txt file?

Only if you publish documentation or an API, where coding agents and docs tooling genuinely read it. Jeremy Howard proposed it in September 2024, but Google has declined to consume it on the record and no major provider has committed to it. Publishing the file is cheap; the mistake is shipping per-page Markdown mirrors alongside it, which create indexable duplicate content.

### How do we tell whether AI assistants are reading our site?

Check your server logs for the named crawlers - GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot among them. Most teams have never looked, so the first pass usually reveals both which assistants fetch the site and which sections they concentrate on. That measurement should precede any protocol work, because it establishes whether the traffic exists to justify it.

### What is the Open Knowledge Format, and is it a publishing standard?

OKF is a Google Cloud specification published in June 2026 that formalises knowledge as a directory of Markdown files with YAML frontmatter, cross-linked into a graph an agent can navigate without a proprietary SDK. Pairing bundles to public pages is possible but experimental and not consumed at scale, so it is better treated as an internal knowledge play with a public option than as a way to publish a website.

### If agents complete tasks for users, what replaces clicks as the metric?

Citation share and task completion - whether the brand appeared in the answer, and whether the agent could finish the job without handing back to a human. Sites that remain screen-scrapable will keep working badly, in the way non-responsive sites kept working through 2012, while sites publishing a tool surface become the ones agents reach for by default.

## Talk to an engineer about AI search visibility

- [Book a call](https://calendly.com/cameron-at-national-digital/discovery)

## Related reading

- [Web delivery best practices for Australian device and platform usage](/headless-cms/omnichannel-content-delivery/web-delivery/)
- [Professional mobile apps solutions for Australian businesses](/headless-cms/omnichannel-content-delivery/mobile-apps/)
- [Content adaptation best practices for Australian device and platform usage](/headless-cms/omnichannel-content-delivery/content-adaptation/)
- [JSON-LD for AI search](/headless-cms/omnichannel-content-delivery/json-ld-for-ai-search/)
- [Auditing AI bot traffic](/headless-cms/omnichannel-content-delivery/auditing-ai-bot-traffic/)

## Related

**Parent:**
- [Omnichannel content delivery](/okf/headless-cms/omnichannel-content-delivery.md)

**Related:**
- [Web delivery best practices for Australian device and platform usage](/okf/headless-cms/omnichannel-content-delivery/web-delivery.md)
- [Professional mobile apps solutions for Australian businesses](/okf/headless-cms/omnichannel-content-delivery/mobile-apps.md)
- [Content adaptation best practices for Australian device and platform usage](/okf/headless-cms/omnichannel-content-delivery/content-adaptation.md)
- [JSON-LD for AI search](/okf/headless-cms/omnichannel-content-delivery/json-ld-for-ai-search.md)
- [Auditing AI bot traffic](/okf/headless-cms/omnichannel-content-delivery/auditing-ai-bot-traffic.md)

# Citations

- [Schema.org - shared vocabulary for structured data](https://schema.org/) — The structured-data vocabulary consumed by search engines and the retrieval pipelines downstream of them.
- [W3C Web Machine Learning Community Group](https://www.w3.org/community/webmachinelearning/) — The community group in which the WebMCP proposal for agent-callable page tools is being developed.