Case study · AI search visibility measurement
A measurement engine for AI search, built and run on our own site
AI Visibility Management is a National Digital product, built and run by us. This study describes what its engine measures and how it works.
National Digital offers AI visibility management built on this engine.
First published , last updated
At a glance
- Industry
- AI search visibility measurement
- Engagement type
- In-house product
- First release
- 2026
- Team
- National Digital engineering team
- Technology
- Claude, Cloudflare R2, Python, GitHub Actions, DataForSEO, Google Search Console, Bing Webmaster Tools, Google Analytics, Model Context Protocol, Cloudflare Workers
The challenge
Customers ask AI assistants which supplier to choose, and a business can see the answer only if it asks the same questions on a schedule and keeps the replies. Search tools report clicks and rankings but not what an assistant says or which page it cited. The engine had to read many sources on different cadences, keep every row, and run unattended at a cost the product could carry.
What we built
National Digital built an engine that pulls each source on its own schedule. It commits the normalised rows to an instance repository so the record accumulates and every figure traces to a row. Each instance has its own repository and its own secrets. A signals file for each brand joins search data with assistant answers and crawl readings on the page they concern. Rules turn that file into findings with their evidence.
- Scheduled pulls from search, analytics, assistant, crawler and site sources
- Standing questions put to AI assistants, with each full answer, its citations and the pages the assistant read
- A git record of every normalised row, with raw responses in object storage
- A signals file that joins pages, queries and questions
- Findings from written rules, each with its evidence and the action it proposes
- A dashboard that is rebuilt from the record
- A remote MCP server with 39 tools that lets an assistant read the record
- An agent canary that drives Claude through each site's task set
What we built in each area
Search & AI visibility
The engine puts standing questions to AI assistants and keeps each full answer with its citations and the pages the assistant read. An agent canary drives Claude through each site's task set.
- Claude
Data & reporting
Normalised rows are committed to an instance repository, so the record accumulates and every figure traces to a row. Raw responses are kept as compressed files in object storage, and a signals file joins pages, queries and questions.
- Cloudflare R2
Integration layer
The engine pulls each source on its own schedule: DataForSEO, Google Search Console, Bing Webmaster Tools, Google Analytics, Cloudflare logs and Screaming Frog. A remote MCP server lets an assistant read the record.
- Python
- GitHub Actions
- DataForSEO
- Google Search Console
- Bing Webmaster Tools
- Google Analytics
- Model Context Protocol
Cloud, security & support
The engine runs on a daily schedule with extra Sunday and Monday runs. A static dashboard and the MCP server run on Cloudflare Workers.
- Cloudflare Workers
- GitHub Actions
The outcome
National Digital has measured nationaldigital.com.au each week since 19 August 2026. Its instance now puts 42 standing questions to AI assistants every week. The record shows what each assistant said and which pages it cited. The study states no effect on visibility. The figures describe what is measured, and the person who owns an action reads any movement from the record.
Where it went next
The engine carries no tenant, so each new instance brings its own questions, target terms and competitors. Planned work includes an outbound alert when a high-severity finding has no action against it, and review text alongside review counts.
This case study describes a live product built and run by National Digital. It reports what the engine measures. It reports no change in National Digital's own search or AI visibility.
Key Takeaways
What the engine's design shows
- Asking the same questions every week and keeping each full answer makes AI visibility something a team can audit.Critical
Each probe stores the answer text, the citations, the queries the assistant searched and the pages it read. A reader can open the answer behind any figure instead of trusting a score.
- Committing every normalised row to git gives the record a history that a person can diff.Critical
A pull writes dated tables to the instance repository. Raw responses go to object storage, so a figure can be rebuilt from the raw file that produced it.
- A single signals file per brand lets a content system plan from one source.Important
The file is keyed by page, query and question. It holds search figures, assistant citations, crawl and extractability readings and the findings, so a reader does not join the tables by hand.
- An MCP server lets the people who use the record ask their own assistant about it.Important
A reader adds the server as a connector in Claude and signs in with a work email. The tools answer from a copy of the record, and the only change a tool makes is to record a decision in the reader's name.
The engine pulls each source on a schedule, keeps every row in git and puts the record in front of people through a dashboard and an MCP server. It reports what is measured and states no effect on visibility.
Questions decision-makers ask about this product
What does AI visibility management measure?
The engine asks AI assistants a standing set of questions about the brand's market and stores each answer in full, with its citations and the pages the assistant read. It also reads Google Search Console, Bing Webmaster Tools, Google Analytics, results pages from DataForSEO, the site's own files and pages, and crawler traffic in Cloudflare logs.
What does AI Visibility Management cost?
The fee is $7,950 (AI Visibility Management (first brand, per month)) a month for the first brand, ex GST, with setup included. The fee covers the weekly record of search and assistant data, a fortnightly senior review and two committed senior engineering days a month. The service page lists the full scope.
How does the engine ask AI assistants questions?
Each week it puts the instance's standing questions to the assistants that the instance has switched on and stores the replies. An agent canary also drives Claude through a task set on the site, once through a Markdown fetch and once through a browser, and records where the task fails.
How does an assistant read the record?
The instance runs a remote MCP server that an assistant adds as a connector by its address. A reader signs in with a work email, and the server answers from a copy of the record. It serves 39 tools, and every answer carries the commit its copy was taken from.
How long has National Digital been measuring its own site?
National Digital has measured nationaldigital.com.au each week since 19 August 2026. The weekly probe first ran in the site's own repository, and the engine instance took it over on 17 October 2026.
Does this study show that measuring improved National Digital's own visibility?
The study does not claim that. It describes what the engine measures and how it keeps the record. Whether a change in visibility followed a piece of work is a reading taken from the record for that piece of work.
Who holds the record?
Each instance keeps its record in its own repository, with its own secrets, questions and target terms. A brand's history is a set of dated files in that repository.
More from our own products
Public interest / open government data and AI agents
publicdata.au: Versioned Government Open Data, Ready for AI Agents
Civic technology
how2vote.au: A Provably Neutral Voting Tool
What's next
Want your own site measured?
The service page lists what each month covers and what the first call looks like. The AI visibility guide explains the readings in more detail, and AI Visibility Management sets out the scope and price.
Want to see what AI assistants say about your business?
Book a 30-minute call and bring one website. We will run the standing questions against it and walk you through the record.