> ## Documentation Index
> Fetch the complete documentation index at: https://docs.withgauge.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run evals on private repositories

> Give Gauge Agents access to a private GitHub repository and use it as the starting workspace for an eval.

Use a private repository when an eval should start from your team's application or an internal example project. Connect the Gauge GitHub App, grant access to the repository, then select it in the eval's run configuration. Gauge prepares that code for the coding agent to work on.

For automatic checks on product changes before merging, follow [Pull request evaluations](/agents/guides/github-pr-checks). That workflow uses the same App connection, with committed cases and results reported on each eligible PR.

## Connect your private repository

<Steps>
  <Step title="Select the Gauge organization">
    Sign in to [Gauge Agents](https://agents.withgauge.com) and select the organization that will own the evals. The GitHub connection and repository access must belong to that organization.
  </Step>

  <Step title="Install the GitHub App">
    Open **Settings → GitHub** and select **Install GitHub App**. On GitHub, choose the account that owns the private repository and grant the App access to that repository.

    If you need a GitHub account owner's approval, Gauge shows **Install requested** until the owner approves it. When the installation is complete, return to Gauge and confirm **GitHub App connected**.

    For an existing installation, select **Add or remove repositories** to grant access to another repository.
  </Step>

  <Step title="Add the repository in Gauge">
    Open **Settings → Repositories** and add a repository. Use **Choose from your connected repositories** to select the private repository the App can access, then save it. Connected private repositories are marked **Private**.

    A repository URL by itself does not grant access. Your personal GitHub login or a local Git checkout does not give Gauge permission to prepare the private code.
  </Step>
</Steps>

## Configure and run an eval

Create an eval in **Agent Experience**, or edit an existing eval. Choose the private repository as its starting workspace, select a Git ref, and configure the task, pass criteria, agent/model, and sample count. Start with one sample and manual cadence.

Use a branch when you want each launch to follow that branch. Pin a commit when you need a repeatable starting point for a comparison. The repository ref belongs to the eval's run configuration.

You can also update an existing eval from the CLI after granting App access and adding the repository:

```bash theme={null}
gauge auth login
gauge orgs use your-org-slug
gauge evals edit <eval-id> \
  --repo https://github.com/your-org/private-app \
  --ref main \
  --cadence none --samples 1
gauge evals get <eval-id>
gauge evals run <eval-id>
```

Replace the organization, eval ID, repository, and ref with yours. `get` lets you review the saved settings; `run` asks for confirmation before launching. The eval keeps its existing agent/model and criteria unless you change them. See [Configure runs](/agents/guides/configure-a-scenario) for those options.

When the session finishes, open it from the eval to review the agent's work, code changes, and judged criteria. Model-provider keys, service credentials, and [billing eligibility](/agents/guides/billing) are separate from repository access. If the task needs an external service, configure its [credentials](/agents/concepts/credentials) on the eval as well.

## Generate evals from a private repository

The same App connection lets you use **Generate evals from repo** in Agent Experience. Select the connected repository and the product surface to inspect, then review the proposed tasks and criteria before creating the suite.

The repository you scan supplies context for generating evals. Review each resulting eval's starting repository and run settings before launching. See [Generate evals from a repository](/agents/guides/create-an-eval-and-run-an-experiment#generate-evals-from-a-repository).

## Manage access and automatic checks

Use **Settings → GitHub → Add or remove repositories** to change which repositories the App can access. Keep the installation authorized while using the repository in future runs.

The separate **Pull request checks** setting controls automatic PR evaluations. You can leave those checks disabled while running manual or scheduled evals on private code. Enabling PR checks also requires the committed configuration described in [Pull request evaluations](/agents/guides/github-pr-checks).

## Troubleshoot repository access

| Symptom | What to check |
| - | - |
| **Install requested** | A GitHub account owner must approve the installation. Return to Gauge after approval. |
| Repository missing from the picker | Use **Add or remove repositories** to share it with the App, and confirm you are in the Gauge organization that owns the connection. |
| Access denied or repository not found | Confirm the installation is active, the repository is still shared, and its URL and requested Git ref are correct. A private repository can appear as not found when access is missing. |
| Preparation fails before the task begins | Open the failed session and inspect its setup error. Restore repository access or correct the ref before retrying. If those are correct, use [Run troubleshooting](/agents/guides/troubleshooting-runs). |
| Repository access works, but the task cannot use another service | Configure that service's credentials on the eval. The GitHub App connection supplies repository access. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.