Skip to main content
Credentials let agents sign in to APIs and MCP servers during a run. Add a credential to your organization, then select it in an eval or preference prompt. You can also connect it to an MCP server used by those runs. Agents see placeholders, not the raw secret. Gauge supplies the real credential only to the approved service hosts. The agent can do whatever the service allows that credential to do.

​
Choose the right kind of credential

​
Add a service credential

1

Open Credentials

In Settings, open Credentials and select Add credential. Choose the service or configure a custom destination, then enter the key or token. Saved values are not displayed again in full.
2

Name it and review its destination

Use a recognizable name such as acme-test. Names can contain English letters, digits, dots, dashes, and underscores, must start with a letter or digit, and cannot contain spaces. Review the approved domains and placeholder environment variable before saving.
3

Select it for the measurement

Add the credential to the eval’s or preference prompt’s run settings. For an authenticated MCP server, associate the credential with the server and select that server version for the measurement.
4

Verify with a small run

Launch a run and inspect the tool activity. Confirm that the agent can reach the intended service before increasing samples or enabling recurring runs.
Use a service credential with only the permissions the task requires. For tasks that write data, choose a service account or test environment appropriate for those writes.

​
Use a username and password pair

Some services sign in with HTTP Basic authentication and a pair of values, such as a token ID and a token secret. Store both values in one credential:
1

Choose HTTP Basic

When you add the credential, set the service URL, such as https://api.acme.dev. For authentication, choose basic — HTTP Basic.
2

Name both environment variables

Enter the Username environment variable and the Password environment variable. Use the names your SDK expects, such as ACME_TOKEN_ID and ACME_TOKEN_SECRET.
3

Enter each value in its own field

Put the ID in Username or token ID and the secret in Password or token secret. Do not paste ID:SECRET into one field. Gauge would send it as the wrong value.
The agent gets two placeholders, one in each environment variable. SDKs, curl -u, and other Basic authentication helpers use them as normal. Gauge replaces both placeholders with the real values only for the approved hosts. A run can use only one credential for each host. Use one paired credential for a Basic service, not separate credentials for the username and password. An existing credential that stores a single value cannot become a pair. To switch, create a new credential with both fields and select it instead of the old one. You can change the two environment variable names on a paired credential later.

​
Names, IDs, and placeholders

These values serve different purposes:
These CLI commands use hosted Gauge. The CLI uses the local backend by default, so set hosted once after you sign in, or add --hosted to each command. See Choose a backend.
Copy the ID from the Credentials table or list credentials from the CLI:
For a username and password pair, gauge connections list shows both placeholder names. It never shows the saved values. --connection replaces the credentials selected for that eval. To use more than one, include each ID with its own --connection flag. Review the resulting selection before running. The same flag is available on gauge preference create|edit. The read-only API endpoint is GET /api/v1/orgs/{org}/connections. It lists each credential’s ID, name, service, allowed domains, and type. It also shows the placeholder name and the last few characters of a saved key, when available. It does not return secrets or OAuth tokens. Create and manage credentials in the web app; the CLI’s connections group lists them.

​
Add an authenticated MCP server

In MCP setup, enter the server URL and wait for Gauge to check its authentication requirements. Gauge shows the sign-in or API-key options for that server. You can also use a saved credential approved for its domain. For OAuth, sign in to the service and review its requested access. Some services require an OAuth application first; Gauge provides the callback URL and requests the client ID and, when needed, client secret. Complete authorization before using the server in a run. If you change the server’s host while entering a key, review authentication again. Credentials are bound to the destination they were approved for. Find the saved server and its versions with gauge mcp list and gauge mcp get <server-handle>. Select a version in the eval’s run settings or reference it in a Markdown case:
The agent can use that server with the connected account. Keep secrets out of the case; the task can simply ask the agent to use your connected account.

​
Which domains can use a credential

Gauge only sends a credential to the exact domains you approve. For example, approving api.example.com does not approve example.com or other.example.com. If you add more than one domain, check each one. Once signed in, the agent can do whatever the credential permits. For example, a key with permission to delete data still allows deletion, even though the agent cannot see the key.

​
Rotate, reauthorize, or delete

Manage existing credentials under Settings → Credentials.
  • Rotate a key or token: save a replacement key. For a username and password pair, save both new values together. Evals and prompts that use this credential keep using it. Check a new run to confirm it works.
  • OAuth authorization: Gauge refreshes tokens automatically and updates access for active sessions. Use the available sign-in or reauthorization flow when access needs attention. Do not replace an OAuth grant with a pasted API key. Follow the displayed recovery instructions if refresh fails.
  • Delete: review the affected run configurations and type DELETE to confirm. Gauge removes the credential from those configurations, and runs already in flight lose access. Its name becomes available again.
Gauge blocks deletion if an MCP server still uses the credential. Remove the server first, or keep the credential and sign in again. To invalidate the key everywhere, also revoke it at the service that issued it.