Skip to main content

​
Prompt Runs

​
What Is a Prompt Run?

A prompt run happens when Gauge sends one prompt to one AI model and saves the response. Throughout the Gauge platform, a prompt run is often called an answer. Each answer includes the full response, the brands it mentions, its sentiment, and the sources it cites.

​
How Gauge Runs a Prompt

Gauge is designed to measure the answer a prospective customer would receive from an AI product, not an artificial benchmark response.
  • A fresh session for every question. Each prompt run starts without chat history, memory, or context from another prompt. On answer engines with a public web interface, Gauge uses a clean, logged-out browser session comparable to an incognito window. API-only engines receive an independent, stateless request.
  • The default consumer experience. For front-end runs, Gauge uses the default model and reasoning behavior the answer engine serves to logged-out users. Gauge does not pin a hidden model version or set a separate reasoning-effort level.
  • Web search is enabled. Gauge uses the engine’s search-enabled experience for every prompt run. The answer can still combine retrieved sources with pretrained knowledge, but the run is not limited to pretraining data.
  • The answer engine controls retrieval. Gauge does not replace an engine’s search system with its own search provider. The engine chooses its indexes, partners, query rewrites, and sources.
AI providers can change their default model, reasoning behavior, training cutoff, or search partners without changing the product name shown to users. Gauge intentionally follows that live default experience. This makes the data representative of what users see, but it also means there is not one permanent model version, effort setting, or training cutoff for front-end runs.

​
Which Search Provider Does Each Engine Use?

Gauge forces the search-enabled path, while the answer engine controls how that search is performed. Because providers can change their search infrastructure, treat this table as a description of the product surface Gauge measures—not a guarantee that a particular third-party backend will remain fixed.

​
How Often Does Gauge Run Each Prompt?

Gauge runs each prompt once per AI model per day. If you track one prompt across six models, Gauge collects six new answers for that prompt each day.

​
When Does Gauge Run Each Prompt?

Gauge starts its scheduled prompt runs first thing each morning. Gauge also runs a prompt when you add it to the platform for the first time.

​
What Happens When I Add a New Prompt?

Gauge tries to run a new prompt as soon as you add it. Results often appear within 30 seconds and should appear within 30 minutes. This applies whether you add one prompt, upload prompts in bulk, or use Gauge to generate prompts.

​
Is One Run Per Day Statistically Significant?

One answer is not enough to show a reliable pattern. AI models can give different answers to the same question, even when you ask it in the same way. Running each prompt every day builds a more reliable sample over time. In a study Gauge published with Graphite, we looked at answers collected through APIs and front-end scraping, including answers that used web search. We found that the results reached statistical significance within 20 responses. By default, Gauge shows results from a rolling 30-day window. This gives each prompt up to 30 recent answers per model. It provides a strong sample while still showing how your results change over time. The first week or two can give you a helpful sense of direction. After one month, you have a much stronger set of data. The window then keeps moving forward each day as new answers replace old ones.

​
How Do I Connect a Change to an Action We Took?

Gauge saves every daily answer, citation, and brand mention for every tracked prompt and enabled engine. To evaluate an action:
  1. Mark the action Complete in the Action Center.
  2. Gauge automatically evaluates the completed action using the ongoing daily measurements connected to it.
  3. Review visibility, citation rate, mention rate, and sentiment before and after the action.
  4. Open the targeted prompts, engines, and URLs to see which sources or descriptions changed.
Each Action Center item is connected to the specific visibility gap, prompts, and content it targets. Its automatic evaluation closes the loop by checking whether the expected citations and answers moved after completion. You can then inspect the daily history for the underlying evidence.
A before-and-after change shows correlation, not proof that one edit caused the result. AI providers, competitors, and the web change at the same time. The strongest evidence is a change that begins after your action and is concentrated in the prompts and pages that action targeted.

​
Does Mention Order Affect the Score?

No. Brand visibility records whether the brand appears in an answer. A mention counts the same whether the brand appears first or last, and Gauge does not currently use mention order as a separate metric or weighting factor. Sentiment is measured separately, so a positive, neutral, or negative description can be analyzed independently from visibility. Gauge also saves the full answer for qualitative review.
Look at each prompt across many days instead of focusing on one answer. You should also review groups of related prompts instead of relying on one exact phrase or wording.

Related →

AI Engines Gauge Tracks