Skip to main content
Use a private repository when an eval should start from your team’s application or an internal example project. Connect the Gauge GitHub App, grant access to the repository, then select it in the eval’s run configuration. Gauge prepares that code for the coding agent to work on. For automatic checks on product changes before merging, follow Pull request evaluations. That workflow uses the same App connection, with committed cases and results reported on each eligible PR.

​
Connect your private repository

1

Select the Gauge organization

Sign in to Gauge Agents and select the organization that will own the evals. The GitHub connection and repository access must belong to that organization.
2

Install the GitHub App

Open Settings → GitHub and select Install GitHub App. On GitHub, choose the account that owns the private repository and grant the App access to that repository.If you need a GitHub account owner’s approval, Gauge shows Install requested until the owner approves it. When the installation is complete, return to Gauge and confirm GitHub App connected.For an existing installation, select Add or remove repositories to grant access to another repository.
3

Add the repository in Gauge

Open Settings → Repositories and add a repository. Use Choose from your connected repositories to select the private repository the App can access, then save it. Connected private repositories are marked Private.A repository URL by itself does not grant access. Your personal GitHub login or a local Git checkout does not give Gauge permission to prepare the private code.

​
Configure and run an eval

Create an eval in Agent Experience, or edit an existing eval. Choose the private repository as its starting workspace, select a Git ref, and configure the task, pass criteria, agent/model, and sample count. Start with one sample and manual cadence. Use a branch when you want each launch to follow that branch. Pin a commit when you need a repeatable starting point for a comparison. The repository ref belongs to the eval’s run configuration. You can also update an existing eval from the CLI after granting App access and adding the repository:
Replace the organization, eval ID, repository, and ref with yours. get lets you review the saved settings; run asks for confirmation before launching. The eval keeps its existing agent/model and criteria unless you change them. See Configure runs for those options. When the session finishes, open it from the eval to review the agent’s work, code changes, and judged criteria. Model-provider keys, service credentials, and billing eligibility are separate from repository access. If the task needs an external service, configure its credentials on the eval as well.

​
Generate evals from a private repository

The same App connection lets you use Generate evals from repo in Agent Experience. Select the connected repository and the product surface to inspect, then review the proposed tasks and criteria before creating the suite. The repository you scan supplies context for generating evals. Review each resulting eval’s starting repository and run settings before launching. See Generate evals from a repository.

​
Manage access and automatic checks

Use Settings → GitHub → Add or remove repositories to change which repositories the App can access. Keep the installation authorized while using the repository in future runs. The separate Pull request checks setting controls automatic PR evaluations. You can leave those checks disabled while running manual or scheduled evals on private code. Enabling PR checks also requires the committed configuration described in Pull request evaluations.

​
Troubleshoot repository access