Workflow

User testing in CI: AI agents on every pull request

User testing used to be a study you ran a few times a year. With agents, it can be a check that runs on every build. Here is how to wire it up and what to expect from it.

5 agents try to .

First test freeRun it from your coding agent

Can you run user testing in CI?

Not with real people, but with AI agents acting as users, yes. Mimiq runs from GitHub Actions or any CI: the agents use the deployed build in real browsers, and the job reports what broke, which requests failed or were slow, and where agents got stuck, with a link to the full report. You choose whether it comments, warns or fails the build.

Why user testing never kept up

Recruiting, scheduling and watching sessions takes days, so user testing happens before big launches. Every small change after that ships untested by anyone who is not on the team. Most usability bugs arrive in exactly those small changes.

A GitHub Action for preview deploys, with one pull request comment.
Plain JSON-RPC over HTTP for any other CI.
Describe the agents in plain words, like your customers.
Test accounts and protected previews supported.
Fail the build on gave-up or stuck, or just comment.

How it works

01
Give it a link and a task
Paste a public address, such as a preview deployment or staging, and say in plain words what a user would try, like "Sign up and start using it". No test scripts and no selectors.
02
Pick how many agents
1, 3, 5, 10 or 20 agents, 5 by default. Each acts as a different one of your users and gets its own real browser for up to 40 steps. About 5 minutes for up to 3 agents, about 5 to 15 minutes for 5, and about 15 to 30 minutes for 20. Agents take turns on shared browsers, so it can take longer when other runs are going, and you can close the tab and come back.
03
Read what they hit
The report shows what broke, what the browser caught on your site (JavaScript errors, requests that failed or answered 4xx or 5xx, and requests slower than 3 seconds), where each agent got stuck or gave up and why in its own words, with the step and a screenshot, and the problems ranked by how many agents hit them, each with a suggested fix.
04
Fix it and run the same agents again
Rerun on the fixed build and see whether the problem is gone for everyone who hit it. Or let your coding agent do the loop through the MCP server, or let the GitHub Action do it on every preview deploy.

GitHub Actions

Runs when a preview deploy succeeds:

on: deployment_status
permissions:
  pull-requests: write
jobs:
  mimiq:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: victorgulchenko/mimiq-preview-check@v1
        with:
          api-key: ${{ secrets.MIMIQ_API_KEY }}
          goal: Sign up and reach the dashboard
          people: 3

Any other CI

GitLab, CircleCI, Buildkite or a cron job: call the endpoint with your key and the URL to test.

curl -s https://mcp.mimiqai.com/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Authorization: Bearer $MIMIQ_API_KEY" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"mimiq.test_flow","arguments":{"url":"'"$PREVIEW_URL"'","goal":"Sign up and start using it","count":3}}}'

What to gate on

Start by commenting only. After a week, you will know which tasks are stable enough to gate on. Agents are not deterministic, so gate on "gave up" (an agent could not continue) rather than on exact step counts.

Agents are simulated users. They find problems fast and on every build; they do not forecast how many real people will convert, they are not a deterministic regression suite, and they do not install native mobile apps.

40steps at most for each agent, each in its own real browser. Every step comes with a screenshot and what the agent was thinking at the time.

20agents at once, each acting as a different one of your users. The browser also records every error, failed request and request slower than 3 seconds on your site, on the step it happened. Agents find problems; they do not forecast how many people will convert.

$0for your first test, with a free account (Google sign-in, no card). It includes 50 credits, enough for 10 agents. After that an agent uses 5 credits, and packs start at $9 for 120 credits.

See a sample report

Questions

Can the agents be our actual users?

In the app, yes: import customers from a CSV and Mimiq builds agents from them (names, emails and phone numbers are stripped first), saved as an audience you rerun on each release. From CI and coding agents, describe the audience in plain words with the audience input.

Is the result flaky?

A run is several independent users, so it varies like users do. Problems most agents hit show up run after run; a problem one agent hit once may not.

What does it cost?

5 credits an agent per run. a $9 Starter (120 credits that never expire) or monthly plans from $19.

Keep reading

Let the agents find what breaks first.

Your first test is free: sign in with Google and it starts right away. Last checked 2026-10-22.