> ## Documentation Index
> Fetch the complete documentation index at: https://docs.withgauge.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Experience

> Measure whether coding agents can complete tasks with your product and identify why they pass or fail.

Agent Experience measures execution. It tells you whether a coding agent can complete a task with your product and whether the final result meets the requirements you defined.

The central question is:

> When an agent uses you, can it succeed?

This is different from [Agent Preference](/agents/concepts/agent-preference), which measures what an agent chooses when several products could solve the task.

## What Agent Experience reveals

An Agent Experience eval can show whether an agent:

* Finds the correct setup path
* Uses current APIs and configuration
* Produces working code instead of a plausible stub
* Handles required security and error cases
* Recovers from documentation or tool friction
* Reaches the same outcome across agents, models, and repositories

Agent Experience separates a coding session's execution status from its quality verdict. A run can finish normally and still fail its pass criteria. A run without completed judging does not count as passed or failed.

## The eval model

Each eval combines three things:

| Part          | Purpose                                         |
| ------------- | ----------------------------------------------- |
| Task prompt   | Tells the coding agent what to build or change  |
| Pass criteria | Defines the observable requirements for success |
| Run presets   | Defines where and how Gauge runs the task       |

### Task prompt

Write a task with a concrete outcome. Give the agent the context a developer would provide, but do not prescribe every implementation step unless following that step is what you want to test.

```text theme={null}
Add password-reset emails to this application. Use Resend for delivery and keep credentials out of source control.
```

### Pass criteria

A pass criterion is a rubric, not a general aspiration. It should describe evidence that Gauge can verify in the final diff or session trace.

```text theme={null}
The final code sends password-reset email through Resend, reads the API key from RESEND_API_KEY, and does not hardcode credentials.
```

You can define more than one criterion. Gauge judges each criterion independently. The run passes only when every criterion passes.

Good criteria are:

* **Observable:** Point to code, configuration, or trace evidence.
* **Specific:** State the required behavior or final state.
* **Current:** Reflect the API and integration path you support now.
* **Independent:** Avoid combining unrelated requirements when separate verdicts would be more useful.

Avoid criteria such as “the integration is good” or “follow best practices.” They do not define a testable boundary.

### Run presets

Run presets let you reuse the same eval across different environments. A preset can define:

* Repository and Git ref
* Coding agent and model
* Judge persona
* Skills available to the agent
* MCP servers available to the agent
* Runs per cycle

Changing one dimension at a time helps you explain why pass rates moved.

## How an eval works

<Steps>
  <Step title="Define the task">
    Create an eval set with the coding task you want agents to complete.
  </Step>

  <Step title="Write the pass criteria">
    Describe exactly what the final result must contain or do. Gauge never judges an eval without a criterion.
  </Step>

  <Step title="Attach run presets">
    Choose the repositories and agent configurations you want to test. Attach multiple presets when you want a controlled comparison.
  </Step>

  <Step title="Run the coding sessions">
    Launch the eval on demand or let its cycle create runs. Gauge executes each run in an isolated workspace and captures the trace and final diff.
  </Step>

  <Step title="Judge each criterion">
    The eval judge inspects the final diff, relevant trace events, web activity, and your implementation reference. It returns a strict pass or fail with concrete reasoning for every criterion.
  </Step>

  <Step title="Aggregate and act">
    Gauge calculates pass rates, compares agents and models, and groups recurring failures into actions you can investigate.
  </Step>
</Steps>

## How judging works

The judge evaluates the final state of the run. A difficult path can still pass if the final result satisfies the rubric. Friction does not change a verdict unless the criterion makes that friction relevant.

For each criterion, the judge returns:

* A strict `passed` value
* Reasoning tied to a diff path or trace event
* The specific missing evidence when the criterion fails

The judge can also record observations outside the rubric. Observations capture issues such as deprecated APIs, documentation gaps, near misses, or unnecessary friction without changing a correct criterion verdict.

<Info>
  When the available evidence cannot establish that a criterion is met, the criterion fails. Gauge does not pass a run on the benefit of the doubt.
</Info>

## Read the results

| Result                   | What it means                                        |
| ------------------------ | ---------------------------------------------------- |
| Criterion verdict        | Pass or fail for one requirement in one judged run   |
| Run verdict              | Passed only when every criterion passed              |
| Pass rate                | Passed runs divided by judged runs                   |
| Agent or model pass rate | The same outcome grouped by execution configuration  |
| Judge reasoning          | Evidence explaining why a criterion passed or failed |
| Observation              | A useful finding outside the explicit rubric         |

Pending or failed judging does not enter the pass-rate denominator. Review the judged-run count with the percentage, especially when the sample is small.

## Choose Agent Experience when

Use an eval when you want to measure:

* Installation and onboarding success
* API integration correctness
* Migration or upgrade instructions
* Framework-specific setup
* Troubleshooting and recovery
* The effect of a skill, MCP server, or documentation change

Run the same eval across several presets to compare agents or models. Keep the task and criteria stable so the comparison remains meaningful.

## Use both measurements together

Preference and experience can move independently:

| Agent behavior                               | What it suggests                                                                      |
| -------------------------------------------- | ------------------------------------------------------------------------------------- |
| Chooses you and passes                       | The agent selects your product and can implement it successfully.                     |
| Chooses you and fails                        | Preference is strong, but setup, APIs, or documentation create friction.              |
| Does not choose you but passes when directed | The product works for agents, but it is not entering or winning the initial decision. |
| Does not choose you and fails                | Investigate both selection and implementation barriers.                               |

<Card title="Agent Preference" icon="ranking-star" href="/agents/concepts/agent-preference">
  Measure which products agents choose before you evaluate whether the implementation succeeds.
</Card>