AI Search Prompt Testing: Our Baseline and a Repeatable Method

Choose a real customer decision

A prompt test is useful when it tells you something about a customer decision. “Say good things about my business” tells you little about discovery or suitability. Start with the task a customer needs to complete and the constraints that actually affect the service.

Profound's persona research examined how context affected answers across selected categories and models. The small-business method below is an editorial application, not a replication of that study. It uses service needs, locations, timing, and budget constraints rather than demographic stereotypes.

Ask the person handling enquiries for common decisions. Examples might include choosing a suitable class, finding a provider that serves an area, or understanding whether a job needs an assessment. Remove personal details and avoid entering sensitive customer records into a testing tool.

Build a small prompt matrix

Choose three meaningful service decisions and four prompt types for each. That creates twelve prompts: enough for a manageable manual review, but not a statistically representative sample of the market. Keep the prompt text in a versioned worksheet.

For a fictional language school, a real constraint might be an adult beginner seeking evening classes. Before using that prompt to assess a particular business, confirm whether the school actually offers those classes. The test should reveal fit, not pressure the website into claiming every attractive service.

Keep branded and unbranded results separate. Including your name in a prompt makes a mention much easier to obtain and changes what the observation means.

Keep test conditions comparable

Record the platform, visible model or mode, date, language, and relevant location context. Use a fresh conversation for each test unless you are intentionally testing a multi-step journey. If personalization cannot be controlled, note that limitation.

For a first round, save every answer in the agreed sample. For a later round, repeat the same prompt set and conditions as closely as practical. Record website edits between rounds and any platform changes you know about.

If you explore additional wording, keep those observations in a separate discovery sheet. Do not replace the original difficult prompts with easier ones and describe the resulting percentage as improvement. A stable denominator is essential to an honest comparison.

Score separate dimensions

Use a short rubric instead of one opaque total. For each answer, record whether the business was mentioned, whether an owned page was cited, and whether each important factual claim was correct, incorrect, or not assessable.

Evaluate usefulness separately. Did the answer identify a relevant provider, explain an important limitation, and suggest a workable next step? A non-mention can still be an appropriate answer if the business does not meet the request.

Do not infer factual accuracy from a positive tone. An enthusiastic answer can invent services. Do not infer poor accuracy from a neutral comparison. Keep the approved business fact record next to the rubric so reviewers use the same reference.

When two reviewers disagree, discuss the claim rather than averaging incompatible judgments. An ambiguous answer may deserve an “unclear” note. Preserving ambiguity is more useful than forcing every observation into a favorable binary score.

Where the 70-question library came from

We maintain a 70-question editorial evaluation library: 37 unbranded task questions, 17 competitor-seeded questions and 16 brand-fact checks. These questions were written for page review and testing; they are not records of searches by real customers. Five library questions match earlier API tests; 65 have not been run. We assess unbranded discovery, competitor comparisons and brand accuracy separately.

Keyword research informed the themes. A keyword’s estimated search volume is not the volume of a complete authored question. Before expanding the test, use real enquiries and search-query evidence when available to replace weak hypotheses with customer language. The complete library is an internal planning record; the exact questions actually tested are published below.

An observed FoundUp baseline: September 28, 2026

We ran eight authored questions through DataForSEO's ChatGPT LLM Responses API using gpt-4.1-mini-2025-04-14, temperature 0.2, a 1,200-token output limit and forced web search. Six unbranded observations used US context, one used Singapore context, and a separate branded diagnostic used US context. “Unbranded” here means the question did not name FoundUp; several questions did name competitors. The seven observations contain six distinct prompts because one question was repeated. This is an exploratory API sample, not consumer ChatGPT usage data or a representative market survey.

FoundUp was mentioned in 0 of the 7 unbranded observations and had no structured citation to its domain in those responses. The branded answer linked to getfoundup.ai in its text but returned no source annotations. It also incorrectly expanded GEO as “Geographic” rather than Generative Engine Optimization. We recorded the link, annotation status and factual error separately. A prompted mention is not unprompted discovery.

The repeated service-versus-software question cited different domains across its two runs. That variation argues for keeping multiple observations. It does not prove one source is more influential. We responded to the factual error by clarifying the acronym and service definition across FoundUp's pages; we have not established what caused the error or measured an improvement after publication.

This example shows why a useful baseline can contain disappointing results. Its value is a specific correction to investigate and a repeatable question set, not a favorable percentage. The collection method is documented in the DataForSEO LLM Responses reference. The broader buying questions also informed our software and service comparisons.

What the same-day repeat did and did not show

The repeat used the same model, temperature, response limit, market contexts and web-search setting. It followed a reported production update; the local articles and comparison pages had not been published. It therefore cannot measure the effect of those local edits.

Do not combine the two rounds into “14 customer queries”. They are repeated, authored API observations with a duplicated question and some named competitors. The sample supplies a baseline and an error to investigate, not evidence of market-wide absence, demand or a particular consumer ChatGPT ranking.

Turn observations into proportionate actions

1. Repeated wrong fact: trace the public sources and use the correction workflow.
2. Repeated missing explanation: improve the relevant service or pricing page if the question matters to customers.
3. Unavailable source page: investigate technical delivery before writing more content.
4. Different but accurate recommendations: record the variation without assuming the website is broken.
5. No clear pattern: retain the baseline and gather more observations before making a large change.

Choose one or two useful changes, publish them carefully, and verify the pages. Then run a comparable observation round. The result can suggest whether an issue persists, but a small uncontrolled test cannot isolate a website edit as the sole cause of a change.

Connect the findings to customer evidence. If the same question also appears repeatedly in enquiries, improving that explanation has a clear human benefit. Use the measurement guide to keep answer observations, referrals, and qualified enquiries distinct.