Skip to main content
Use an optimization to test whether a change to your docs or skills helps agents do better. Gauge runs the task with the change and compares the results. Each test is called a trial.

​
Plan a useful comparison

Start with completed results and a specific problem. For example, an agent repeatedly misuses an API because the setup page omits a required initialization step. Open an eval or preference prompt that has judged runs and click New optimization (or Optimize if it already has one). All your optimizations are also listed on the Optimizations page. Choose the page or skill you want to change. Make a change that helps developers use your product. Keep the task and pass criteria the same so you can tell whether the change helped. Choose one agent and model. Set how many times to run each trial. More runs give you more evidence, but cost more. Review the estimate before starting.

​
How an optimization runs

Gauge measures. You write the changes.
  1. Start runs the baseline: the task with your current docs or skill.
  2. You write a change and launch it as a trial. Each trial is compared with the same baseline.
  3. Gauge judges the trial sessions and shows whether the change helped.
  4. Repeat with new changes until you have a clear result.
  5. Complete the optimization.
Gauge does not end an optimization on its own. Credits are charged per session as trials run. Gauge does not set credits aside when you start.

​
Run the loop from the CLI

Ask Gauge where the sessions fail, then estimate the cost:
Use a target from the model catalog in place of <harness:model>. To optimize preference, replace --eval <eval-id> with --preference <prompt-id>. Start the optimization and wait for the baseline:
--effort low|medium|high|xhigh sets how many sessions and rounds to plan. Use --samples and --rounds instead for exact values. --credit-cap only records an estimate; it is not a spending limit. watch exits with code 4 when it is your turn. Get the page as the trial will see it, edit it, and launch it as a trial:
To change a skill instead, read its files with gauge optimizations skill-files <version-id>. Then pass --edit-skill <version-id> --skill-dir <path>, where the directory holds the complete skill, SKILL.md included. The idempotency key makes a retry safe: the same key returns the first trial instead of paying for a second one. Not sure what to do next? gauge optimizations get <optimization-id> -o json includes a nextAction field that names the next step.

​
Review trials

Read the change and compare the results. Open the sessions to see why the score changed. Repeat important tests: one better run does not prove that the change will help every time. When you have a clear result, complete the optimization:
Completing an optimization does not change your eval settings or your published site. To add more rounds to a completed optimization, use gauge optimizations continue <optimization-id> --rounds 2. To stop an optimization and its running sessions, use gauge optimizations cancel <optimization-id>.

​
Ship the change

Review the final change and publish it through your usual content or code workflow. For a docs rewrite, gauge optimizations change <optimization-id> <trial-id> --body prints the page body. A shipping marker in Gauge records your action; it is not a deployment. After publishing, run the task again against your live docs or skill. Try other relevant tasks and models too, to check that the change helps beyond this one eval.