Skip to main content
Gauge recreates realistic coding tasks, runs agents in isolated workspaces, and records enough evidence to explain the result. You can compare agents, models, repositories, and tool configurations without giving a local agent access to Gauge infrastructure.

​
What Gauge measures

Agent Preference

Measure which products an agent recommends, mentions, installs, or uses for an open-ended task. Track results by market, brand, agent, model, and repository context.

Agent Experience

Define a task and explicit criteria. Gauge judges each completed run against those criteria and records pass rates and evidence.
Both measurement types use the same execution pipeline. The prompt defines what to measure. The run preset defines the environment in which Gauge measures it.

​
From measurement to result

1

Define the measurement

Create an Agent Preference prompt or an Agent Experience eval set. Add the question, task, criteria, tags, and optional market context that Gauge should preserve across runs.
2

Attach run presets

A run preset combines a repository, optional Git ref, agent and model choices, a judge persona, skills, and MCP servers. Attaching a preset determines which configurations run during each cycle.
3

Expand a cycle

On the configured cadence, Gauge expands each measurement across its attached presets and samples. You can also launch a measurement immediately from the web app, CLI, or API.
4

Run the agent

Gauge restores or creates a workspace, starts an isolated sandbox, and runs the selected coding agent headlessly. Limits on turns, wall-clock time, and spend bound the session.
5

Capture evidence

Gauge stores the session trace, the working-tree diff, cost, timing, exit reason, and extracted tool-use signals. A run does not finish successfully until its durable evidence is available.
6

Judge and aggregate

Gauge evaluates the run, attaches verdicts and evidence, and rolls results into Agent Preference rankings, Agent Experience pass rates, dashboards, and recommended actions.

​
Configuration model

This separation lets you reuse one run preset across many measurements. It also lets you change a model, skill, or MCP server once and compare the new configuration consistently.

​
Experiments

Experiments test a change against a run that exposed a problem. Gauge can fork the original context, create candidate variants, rerun the task, and compare each result with the original baseline. Use this workflow to answer questions such as:
  • Did a documentation rewrite help the agent choose the right API?
  • Did a new skill reduce failed tool calls?
  • Did a different setup path improve task completion?
Gauge preserves the source run and variant evidence so you can inspect the recommendation before you ship a change.

​
Control plane and execution plane

The web app, CLI, and API are control-plane clients. They create configuration and enqueue work. They do not run coding agents locally. Gauge executes agent-generated code in isolated sandboxes. The control plane keeps organization data, provider credentials, scheduling, and billing separate from those sandboxes. This boundary lets Gauge record realistic behavior without exposing control-plane credentials to the agent.