Agentic Usage

An AI coding agent that can edit your code but can't verify it is working half-blind. Point it at the Operio CLI and it closes the loop: write a feature, write a test for it, run that test against a real environment, read the failure, fix the code, run again.

Why the CLI suits agents

JSON on stdout — an agent parses a result instead of scraping a table.
Exit codes carry the verdict — 0 passed, 1 failed, 3 needs re-auth, 7 retry later.
Nothing blocks — no prompts, no confirmations, no pagers.
Errors are one line — no raw HTTP dumps eating the context window.
Secrets are stripped — an agent can read a full run result without pulling your staging password into a transcript.

Setting an agent up

Do the interactive part yourself, once:

npm install -g @operio/cli
operio auth login
operio teams use <team-id>
operio projects use <project-id>

Every command the agent runs from here is scoped to that project, and none of them need a browser.

Then drop this into your agent's instructions file (CLAUDE.md, AGENTS.md, or a system prompt):

## Testing with Operio

This project uses Operio for end-to-end tests, driven by the `operio` CLI.
The CLI is authenticated and already scoped to the right team and project.

- List tests: `operio tests list`
- Read one: `operio tests get <id>`
- Create: `operio tests create --input -` (JSON on stdin; creates a draft), or
  build it from flags with repeated `--step "action:…"` / `--step "verify:…"`
- Publish a draft: `operio tests activate <id>`
- Run and wait: `operio tests run <id> --environment-id $OPERIO_ENV_ID --wait`
- Read a failure: `operio runs get <run-id> --full`

A run that exits 1 means test failures — read the run before changing anything.
Never run `operio auth login`; it needs a browser. If a command exits 3, stop and
tell me the session needs refreshing.

A worked example

A feature was just implemented. Here's the loop the agent runs to cover it.

1. Look around

operio whoami
operio environments list
operio tests list --status active

The agent learns which project it's in, which environments exist, and what's already covered — so it doesn't duplicate a test.

2. Write the test case

Piping through stdin avoids every shell-quoting problem:

cat <<'JSON' | operio tests create --input -
{
  "name": "Guest checkout with a discount code",
  "description": "A signed-out visitor applies a valid discount code and completes checkout",
  "severity": "high",
  "steps": [
    { "order": 1, "type": "action", "instruction": "Open the product page for the Standard plan" },
    { "order": 2, "type": "action", "instruction": "Click Buy now" },
    { "order": 3, "type": "action", "instruction": "Enter the discount code SAVE20 and apply it" },
    { "order": 4, "type": "verify", "assertion": "The order total drops by 20 percent" },
    { "order": 5, "type": "screenshot", "label": "Discounted order summary" },
    { "order": 6, "type": "action", "instruction": "Complete the purchase with the test card" },
    { "order": 7, "type": "verify", "assertion": "The confirmation page shows an order number" }
  ]
}
JSON

The response contains the new id. It's a draft — nothing runs until it's published.

3. Publish and run

operio tests activate <test-case-id>
operio tests run <test-case-id> --environment-id <env-id> --wait

--wait blocks until the run finishes. Exit 0 if every case passed, 1 if not.

4. Read the failure

operio runs get <run-id> --full

The full run includes each step's status plus what the agent observed — expected behaviour, actual behaviour, and where they diverged. That's what a coding agent needs to tell whether the app is broken or the test was written wrong.

5. Fix and re-run

operio tests update <test-case-id> --input ./checkout-test.json
operio tests run <test-case-id> --environment-id <env-id> --wait

If the app was wrong instead, the agent fixes the code and re-runs the same command. Either way it's the same three commands.

Agentic Usage

On this page