In this guide
  1. Separate investigation from edits
  2. Distinguish editing from external actions
  3. Write acceptance criteria first
  4. Budget by access route
  5. Example: repairing a search defect
  6. Who should pilot it

Separate investigation from edits

Ask first for the failure path, relevant files and missing evidence. Review that explanation before authorizing changes. A reproducible login defect is a better evaluation task than a request to redesign the entire application: it gives you a known starting state and an observable stopping condition.

Distinguish editing from external actions

Permission rules can control tools and their targets. In your evaluation, treat a file edit, a test, an external request and a production action as different decisions. Use a development copy with the credentials necessary for that task, then check whether the resulting permission prompts are understandable to the person supervising it.

Write acceptance criteria first

For a CSV importer, include empty fields, invalid dates, duplicate rows and encoding behavior in the acceptance criteria. An agent can add tests that merely repeat its own assumptions. Review the expected outputs as carefully as the code, and retain one failing example that independently demonstrates why the change was needed.

Budget by access route

Separate subscription allowance from metered API spending. Record a routine edit and a larger investigation as different task classes, including rejected attempts before an accepted change. Check current limits and overage settings for your own access route; a headline subscription price is not a guaranteed cost for every development workload.

Separate model choice, context size, parallel work and retries when consumption rises. Reconcile the contract and API route separately. Consider additional capacity for a demonstrated interruption and identify the settings owner and billing record. A purchase is not evidence that the quality or maintainability of a change improved.

Example: repairing a search defect

Consider a hypothetical search defect where records with a missing optional field disappear. Supply a minimal dataset and the expected result, then review the identified cause before accepting an edit. Require checks of both the missing-field case and ordinary records. Keep deployment outside this pilot. The useful result is a narrowly explained patch and reproducible verification, not an assertion that the agent repaired the whole search system.

Suppose a hypothetical maintenance fix takes three attempts. Include investigation, failures and clarification, not just the successful final attempt. Measure the route to a patch that satisfies independent expectations. This distinguishes a genuine need for more allowance from an unclear task that would remain inefficient under a larger subscription.

Who should pilot it

A team with someone able to review diffs and validate behavior can run a meaningful pilot. If nobody can describe the existing behavior or start a test environment, document those basics first. Repeated unexplained changes or unnecessary dependencies are reasons to narrow the task, rather than a reason to grant the agent broader authority.

Keep accepted diffs, requirements, tests and startup procedures in the repository. Do not leave essential assumptions only in a conversation. Verify the next maintainer can continue, then separately review app, API and integration arrangements if access changes. Preserve only the authority the continuing workflow needs.

Official sources

Pricing, availability and feature limits can change. Check the linked official pages before subscribing.

How we compareAdvertising disclosure