Recording what AI assistants say about your business
A screenshot of one answer is an anecdote. The same questions, asked every week with the answers and sources kept, is a measurement you can act on.
Quick answer: Fixed customer questions put to each AI assistant weekly, with full answers and sources kept, give a series that shows what assistants state about a business and which page mechanisms explain a wrong claim.
- AI Search Visibility
- Measurement
- Machine-Readable Delivery
Quick answer
How do you find out what AI assistants say about your business?
Write down the questions your customers ask, put the same questions to each assistant on the same day every week, and keep the full answer with the sources it cites. The record shows what each assistant states, where it got it, and what changes from week to week.
Additional Context
One answer read once tells you what one assistant said to one person on one day. Assistants rewrite their answers as the pages they read change, so the useful unit is the series.
Sources
- Google Search Central - AI features and your website
Google's guidance that AI Overviews and AI Mode use the same eligibility rules as Search and need no special markup.
- OpenAI - Overview of OpenAI crawlers
OpenAI's published user agents, separating the training crawler from the search crawler and the user-triggered fetcher.
The record
Why a screenshot is not a measurement
Someone in the business asks an assistant about the company, reads the answer, and forwards a screenshot. The answer is wrong about a fee or flattering about a competitor, and for a week it is the most discussed document in the office. Then it is forgotten, because nobody can say whether the answer is the same today.
An assistant composes each answer at the moment it is asked. It draws on what the model already holds and, for anything current, on pages it fetches while it answers. Change the pages and the answer changes. Ask on a different day, in a different phrasing, and the answer changes again. A single answer therefore cannot be compared with anything. A measurement needs the same questions, the same assistants, the same day each week, the full text kept, and the sources recorded beside it. Once those five things are fixed, each week's answers become a row in a series, and the series is what shows whether anything moved.
Writing the question set
The questions are the ones customers ask, in the words customers use. They ask what the service costs, whether the business is licensed, accredited or registered, which provider is closest to a suburb, whether a particular thing can be done at all, and how long it takes. A set in the low twenties covers one business line without the weekly read turning into noise. Include the questions where a competitor is the honest answer, because those are the ones that show what the assistant knows about the market and where the business sits in it.
The set stays fixed for months. A question changes only with a dated note, so that a shift in the record can be traced to the question and not mistaken for a shift in the answer. New questions join as a new row, with their own start date.
What to keep from each answer
Keep the full text, the list of sources the assistant cited, whether the business was named at all, and each claim it made about the business. Then check every claim against the business's own published pages and mark it as matching, differing or unverifiable. A claim that differs is the start of a finding. A claim that is unverifiable is often the more interesting case, because it usually means the business has never published the fact anywhere an assistant can read it.
What the record changes about the engineering
A wrong fee in an answer traces back through the cited sources. Often the business's own site is absent from that list and a directory or a review site took its place. Often the site is present and the fee is still wrong, because the fee schedule sits in a PDF, or in a tab that only appears after a script runs, or on a page that disagrees with another page on the same site. Each of those is a page problem, and page problems are engineering work: publish the fact as HTML text, mark it up with structured data, serve it to the agents that fetch it, and keep every copy of it in agreement. The pages on the sources assistants cite and machine-readable delivery cover the mechanisms in detail.
The fix is never a rewrite of what the business says about itself. It is the delivery of what the business already says, in a form an assistant can read and trust. That line is what keeps the weekly record an engineering instrument. The managed form of this work is AI Visibility Management, and the process for turning each row into a ticket is described in turning findings into engineering tickets.
Step-by-step
How to run a weekly AI answer record
About half a day to set up, then about an hour a week to readIntermediate
The minimum that makes the answers comparable from one week to the next.
Tools Required
- Enquiry emails and call notes
- Access to each assistant's API or interface
- A spreadsheet or a CSV repository
Steps
- Write the question set from customer language(Two hours)
Pull the questions from enquiry emails, call notes and search terms. Keep the customer's wording. Aim for a set in the low twenties for one business line and freeze it.
- Pick the assistants and the day(Half an hour)
Choose the assistants your customers use and ask each one through its API or interface on the same weekday every week, with web access on where the product offers it.
- Keep the whole answer and its sources(Automated after the first week)
Store the full text, the cited URLs, the date and the assistant version in a file you own. A spreadsheet is enough to start; a repository of plain CSV is better once the record is a few months long.
- Check each claim against your own pages(An hour a week)
For every statement about the business, record whether it matches a published page, differs from one, or cannot be verified because the fact is not published.
- Trace each differing claim to a mechanism(As findings arise)
Follow the cited sources. Note whether your page was read, whether the fact was readable on it, and whether another surface disagrees. Write the mechanism down as a finding.
Summary: Freeze a customer-language question set, ask each assistant weekly, keep full answers and sources, check each claim against your own pages, and trace differences to a page mechanism.
Key Takeaways
What a usable AI answer record requires
- One answer is an anecdote and a weekly series is a measurementCritical
Assistants compose answers at the moment of asking from the pages they can read, so only a fixed question set asked on a fixed cadence can show whether anything changed.
- The questions come from customers, including the ones you would rather not askImportant
Questions where a competitor is the honest answer show how the assistant understands the market and where the business sits in it.
- Every claim gets checked against your own published pagesCritical
Matching, differing and unverifiable are the three states. Unverifiable usually means the fact has never been published where an assistant can read it.
- A differing claim is a page problemImportant
The fix is delivery of facts the business already publishes: HTML text, structured data, agents allowed to fetch, every copy in agreement.
A weekly record of fixed questions, full answers and cited sources turns what assistants say about a business into a series, and each claim that differs from the business's own pages becomes an engineering finding with a mechanism behind it.
Questions about recording AI assistant answers
How many questions and how many assistants are enough?
A set in the low twenties covers one business line, and four assistants cover the ones customers commonly reach for. More questions make the weekly read slower without changing what it shows. A second business line gets its own set.
Should the questions be asked with web access on or off?
On, wherever the assistant offers it, because that is how a customer asking about a current fee or an opening hour is answered. The answer with web access shows which pages were fetched, and the cited list is the column that turns a claim into a finding.
Why keep the full answer when a score would do?
A score shows that something moved. The text shows what moved and the sources show why. When a claim changes from matching to differing, the sentence and the cited page are what turn it into a ticket.
Does the record show whether a change on the site worked?
It shows what the assistants said before and after, on the dates recorded. It does not prove that the change caused the difference, because the assistants and their sources change too. The honest reading is the series, with the change marked on it.
