How to Run an AI Search Audit for Your Website

Run Your AI Search Audit Step by Step

Create 1 working sheet before you start.

Begin with:

Use Pass, Fail, or Unknown. Choose Unknown when the evidence is not enough to decide.

Step 1. Set the Platforms and Questions You Will Audit

Start with AI search products your customers use or your business needs to measure.

For most sites, that means choosing from:

You do not need to test every AI product.

For each platform, note:

Next, list questions customers genuinely need answered.

Useful sources include:

Focus on tasks rather than isolated keywords.

Questions may cover:

Make each question specific enough that its result tells you what to investigate.

For high-value questions, create a few natural wording variations while keeping the intent unchanged.

“Which accounting software works for a small agency?” and “What accounting software should a small agency use?” can belong to the same question family.

Keep the question set unchanged during the audit. If the scope changes, save a new version rather than mixing new questions into the old baseline.

Step 2. Verify Each Platform Can Reach and Read Your Pages

You do not need to inspect every URL first.

Start with pages connected to your priority questions. Expand into a broader technical crawl only when evidence suggests the problem affects more of the site.

Take each candidate page and check whether the search system can retrieve the content needed to answer the question.

For each page, check:

For Google Search, minimum technical requirements include allowing Googlebot, returning HTTP 200, and providing indexable content. Meeting those requirements still does not guarantee indexing or serving.

Check the server and firewall as well as robots.txt. A crawler can be permitted in robots.txt and still fail at the WAF, CDN, authentication, or rate-limit layer.

If logs are available, save:

Where a platform provides bot verification or IP information, use it rather than trusting a User-Agent string alone. Bing specifically warns that User-Agent strings can be spoofed and provides Bingbot verification guidance.

Finish each page with:

A useful finding is:

This step tells you if page retrieval is blocked. It does not tell you if the page will be cited.

Step 3. Match Each Audit Question to the Page That Should Answer It

Take every question from Step 1 and ask:

Which page should give the best complete answer to this question?

Start with your CMS, sitemap, or site crawl. Use a site: search only as a secondary discovery check, not as your complete site inventory.

Then inspect what each candidate page is actually meant to do.

Assign 1 state:

Suppose someone asks, “How much does project-management software cost for a 50-person team?”

A site may have both a product page and a pricing page. Keyword overlap does not decide ownership. The page responsible for the pricing decision should own the answer.

The answer may belong:

For each question, save:

Actions may include keep, expand, merge, reassign, review a new page, or exclude.

Finish when each important question has 1 owner, a deliberate content gap, or an explicit exclusion.

Step 4. Check Each Page Gives a Clear, Current, Supported Answer

Now audit the content on each owner page.

Use 4 checks.

1. Can you find the answer?

Ask:

Can I point to 1 sentence or short block that answers this question?

If not, mark the answer as missing.

2. Does evidence support the answer?

Break important statements into individual claims.

“Product X is fastest and reduces reporting time by 40%” contains at least 3 claims:

Each claim needs support.

For important claims, save:

Use:

When several websites repeat the same statistic or claim, trace them back to the original source.

3. Is the information still current?

Recheck facts that can change quickly, including:

Old evidence is not automatically wrong. Time-sensitive claims simply need current verification.

4. Do names, facts, and qualifiers agree?

Check company names, products, services, locations, current roles, versions, and acronyms across page copy and structured data.

Keep important conditions next to the claim they change.

If evidence applies only to a particular country, version, sample, or date, state that beside the claim instead of leaving the main sentence sounding universal.

This is also where you can reject supposed AI-optimization shortcuts.

Use structured data for its normal purpose and keep it consistent with visible content.

Finish each page with:

Save the problem before trying to fix it.

Step 5. Collect the AI Search Visibility Data You Already Have

Before manual testing, collect data already available from logs, search platforms, analytics, and conversion tracking.

Keep each signal separate.

This tells you a request happened. It does not prove a citation.

Use Bing data to answer 2 practical questions:

A grounding query is a grouped phrase, not the user’s exact prompt.

Save:

Use measurement states consistently:

Keep these 4 datasets separate:

Choose a baseline period that gives enough data for your site. A low-volume B2B site may need a longer observation window than a high-traffic publisher.

Use this baseline to decide where manual testing should begin. If native data already shows that a page receives citations but referrals are weak, you do not need to start by assuming a crawl problem. Move to the part of the audit that can explain the gap between citation and visit.

Step 6. Test Your Questions Across AI Search Platforms

Now run controlled tests using the question set from Step 1.

For each run, save:

If a state cannot be observed, mark it Unknown.

Use a fresh conversation for independent tests where possible. Previous conversation context can affect later responses.

Run the exact stored question and keep every valid result.

Save:

Classify the brand result simply:

Citation is a separate field. A brand can be mentioned without its site being cited.

Referral is separate too. Check analytics rather than inferring a visit from the answer.

Repeat high-value questions when 1 result is not enough to support the decision.

A 2026 preprint that repeatedly sampled Perplexity Search, OpenAI SearchGPT, and Google Gemini found substantial variation in citation distributions across repeated runs and argued that single-run visibility estimates can look more precise than the underlying results justify.

Use a decision-based stopping test instead:

After each batch, ask whether another comparable run could realistically change the conclusion you would report.

If the result keeps switching between not mentioned, mentioned, and recommended, report the finding as unstable instead of forcing a precise percentage.

Test natural wording variants separately:

When reporting a rate, show the denominator.

That 60% describes your test panel, not market share.

Automation is fine when it preserves:

Step 7. Trace the Sources Behind Important AI Answers

A citation count does not tell you whether the cited source supports the answer.

Trace sources for high-value claims, recommendations, prices, comparisons, current facts, negative statements, and factual errors.

Use 5 actions:

If an answer says a company specializes in enterprise cybersecurity, has offices in 3 countries, and costs less than a competitor, those are 3 separate claims. Check each one.

Classify support as:

For important claims, use human review rather than relying only on another AI model to judge citation support.

Then classify the source as:

Check freshness too. A source can accurately describe an old state and still be wrong for a current answer.

For brand information, use:

If an important AI claim has no visible citation, mark the source as Unknown.

Compare cited sources with your own page.

Ask:

Compare the information first.

Step 8. Find the Earliest Problem Blocking Visibility

Now connect findings from Steps 1–7.

Find the earliest problem you can prove.

Record:

Then decide whether you have a content/evidence gap or an unresolved retrieval-selection question.

A 2026 SAGEO Arena preprint treats retrieval, reranking, and generation as separate stages and reports that optimization methods can behave differently when retrieval and reranking are included rather than assuming the candidate document is already present.

If the evidence is not strong enough to choose 1 diagnosis, collect more evidence instead of guessing.

Not seeing a citation in your test panel does not prove the platform cannot retrieve your site.

Step 9. Fix the Problem and Run the Audit Again

Fix the diagnosed problem, not everything around it.

Save:

Match the fix to the problem:

Re-run only the steps affected by the change.

Check again when the relevant system has processed the change.

Useful triggers include:

Keep comparison conditions as similar as possible: same question version, platform, market, language, and classification rules.

If conditions changed enough to invalidate the comparison, mark the result Not comparable.

Report movement without claiming causation you cannot prove.

That is an observed change in the test panel.

It does not by itself prove that the page edit caused the difference.

Bing makes the same distinction in AI Performance: citation trends can reflect changes in user questions, your content, or the underlying AI systems, and the trends cannot be attributed to a single cause from the dashboard alone.

Close each finding as:

Publishing a fix does not close a finding.

Evidence does.

Know What Each Audit Signal Proves and What It Does Not

Each signal answers a different question.

No citation in 1 run means only that no citation appeared in that observation. Missing dashboard data means the metric is unavailable, not 0.

Build the Final AI Search Audit Report

Keep the report practical. Someone who did not run the audit should still know what failed and what to do next.

Include 6 parts:

Keep 1 change log so the next audit retains the baseline.

The executive summary should answer 5 questions:

The audit is complete when every important finding shows what was tested, the evidence behind it, the problem found, the required action, the owner, and the next check.