Skip to content

Concepts

Test suites

Test how your system reacts when a partner's data changes over time. A test case sets a starting state, plays a timeline of changes through one or more simulations as named people, and checks what your system did: the calls it made, the data it wrote back and how quickly it noticed. Test suites come with the Team and Scale plans.

How a test case works

PartWhat it holds
GivenThe starting state: a suite baseline (a snapshot of every simulation, restored first), setup changes made at the start, and history written with timestamps in the past for anything that needs time to have passed.
WhenA timeline of steps at offsets such as +0m, +10m, +2h. Each step is an action by a person on chosen records of one simulation. A wait step holds the timeline until your system reads the changed records or makes a call.
ThenExpectations with a deadline. call and state are judged by Slurry; observed is something only your product can show, passed or failed by a person or your own harness.

A suite groups cases over a set of simulations. Simulations that share your organisation's world can be tested together: the same people slip work in Jira, post in Teams and update the plan.

a case, as YAML
name: Invoice goes past due, then is paid
given:
  baseline: clean                # suite snapshot restored first
  history:                       # written with timestamps in the past
    - action: update
      target: { collection: invoices, ids: [in_1042] }
      params: { set: { status: open, due_date: "{{date-3d}}" } }
      ago: 10d
when:
  - at: +0m
    action: transition
    target: { collection: invoices, ids: [in_1042] }
    params: { to: past_due }
    as: overdue
  - at: +1m
    action: wait                 # hold until your system reads it
    target: { ref: overdue }
    params: { for: read, timeout: +15m }
  - at: +5m
    actor: finance@northwind.example
    action: update
    target: { ref: overdue }
    params: { set: { status: paid, amount_paid: 1200 } }
then:
  - kind: call                   # your system called the API
    by: +10m
    method: GET
    path: /v1/invoices/in_1042
  - kind: call
    by: +20m
    method: POST
    path: /v1/invoices/*/send
    none: true                   # ...and never sent a reminder
  - kind: state                  # data your system wrote back
    by: +20m
    target: { collection: notes, where: [{ field: invoice, op: eq, value: in_1042 }] }
    count: { min: 1 }
  - kind: observed               # only your product can show this
    by: +15m
    expect: The customer's account shows as up to date
options:
  speed: 60                      # an hour of timeline a minute

Describe a test in your own words

The fastest way in. Dump everything you have into one prompt: the story, who is involved, what changes and when, and what your system should do. Slurry drafts complete cases against the suite's simulations and their real records and people, plays back what it understood, and asks about anything it could not map. Answer in the same conversation to refine. Drafts are never saved by themselves; saving a case freezes every generated value, so a run never calls a model. Drafting uses your AI allowance.

draft from a prompt
curl -X POST https://api.slurry.io/v1/suites/billing/draft -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY" -H "Content-Type: application/json" -d '{
  "prompt": "An open invoice goes past due, the customer pays half a day later and disputes the rest. Our system should fetch each change within five minutes and never send a second reminder after the payment."
}'

# 200: draft cases (not saved), what Slurry understood, and its questions
{ "cases": [ { "name": "...", "definition": { "given": {}, "when": [], "then": [] }, "validation": { "ok": true, "errors": [], "warnings": [] } } ],
  "playback": [ "1. Invoice in_1042 is open and due in three days", "2. At +0m it moves to past_due", "..." ],
  "questions": [] }

# refine: send the draft back with your answer
curl -X POST https://api.slurry.io/v1/suites/billing/draft -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY" -H "Content-Type: application/json" -d '{ "prompt": "...", "previous": { ... }, "answer": "The reminder is POST /v1/invoices/{id}/send" }'

Actions

Every simulation has an action catalogue (GET /v1/simulations/{id}/actions), built from its own model, so it works on any API:

ActionWhat it does
createNew records, generated like the rest of the data, with any fields you set.
updateSet fields (params.set, dotted paths) and stamp the record's updated time in the shape it already has.
transitionMove records along their status workflow (force: true to move outside it).
deleteRemove records, as if someone deleted them in the vendor app.
touchStamp records as changed now without changing anything else.
respondAn attendee accepts, declines or tentatively accepts calendar events.
faultAn outage, a rate-limit storm or a slow network for a while; scope it to a path or a collection, or let the first requests through (passFirst), for a partial failure.
jira.*transition, comment, slip, assign, priority, reopen: with the changelog entries Jira writes.
teams.*post, reply, chat: as a member, in the thread structure Teams keeps.
calendar.leaveSomeone goes on leave: an all-day out-of-office event (showAs oof) from a day to a day, as the mailbox owner sees it.
teams.meetingA Teams meeting that has happened, organised by the actor (Microsoft Graph simulations): the diary entry with responses, the online meeting, attendance, recording and a transcript of exactly the lines you write (Name: words, one turn per line), spread across the meeting. declined and noShow invitees miss it.
a meeting step
{
  "at": "+0m", "action": "teams.meeting", "actor": "priya.shah@acme.example", "as": "steerco",
  "params": {
    "subject": "Claims Modernisation steering committee",
    "attendees": "tom.evans@acme.example, Sam Taylor, dana.whitlock@vendorco.example",
    "declined": "Sam Taylor",
    "start": "-2h", "minutes": 60,
    "lines": "Priya Shah: Apologies from Sam again, so we cannot approve the change request today.\nTom Evans: I am on leave from the 3rd for four weeks. Dana covers the cutover plan.\nPriya Shah: Decision recorded: the dress rehearsal moves to 14 November."
  }
}

Targets choose records by ids, where conditions on dotted paths (eq, ne, in, contains, gt, lt, exists and more), a ref to records an earlier step named with as, or pick and count. Values can be relative: {{now+3d}}, {{date-2w}}, {{epoch}}, {{tag}} (unique per run).

The joystick

Fire one action now, from a simulation's Joystick tab or the API. Each action lands on a session timeline, spaced as you fired it, and a session can be saved as a test case to add expectations to.

fire an action
curl -X POST https://api.slurry.io/v1/simulations/acme-jira/actions -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY" -H "Content-Type: application/json" -d '{
  "action": "jira.slip",
  "actor": "ivan.kirkbride@acme.example",
  "target": { "collection": "issue", "where": [{ "field": "fields.project.key", "op": "eq", "value": "UWP" }, { "field": "fields.status.name", "op": "ne", "value": "Done" }], "count": 3 },
  "params": { "days": 14, "text": "Moving the date: the vendor drop is late again." }
}'

# keep going in the same session, then keep it as a test case
curl -X POST https://api.slurry.io/v1/runs/$SESSION/save-as-case -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY" -H "Content-Type: application/json" -d '{ "suite": "delivery", "name": "UWP slips" }'

Runs

  • A run holds its suite's simulations: one run per set at a time, others queue. Their URLs and keys stay the same, so your system is configured once.
  • Quiet mode (on by default) stops background behaviour during the run; the API keeps serving.
  • speed plays the timeline faster: 60 plays an hour a minute. Deadlines scale with it.
  • Each step records when it was applied and when your system first read a record it changed: the first response after the change that contained it.
  • Pause, resume, bring the next step forward, or abort, from the run page or the API.
  • Results: passed, failed, errored (a step could not apply), aborted, or needs_review when an observed expectation is still undecided after its deadline.
run and wait
# run one case (or leave "case" out to run every case in order)
curl -X POST https://api.slurry.io/v1/suites/billing/runs -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY" -H "Content-Type: application/json" -d '{ "case": "invoice-goes-past-due-then-is-paid", "trigger": "ci" }'

# wait up to 25 seconds at a time for it to finish
curl https://api.slurry.io/v1/runs/$RUN_ID/wait -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY"

{ "id": "...", "state": "finished", "result": "passed",
  "summary": { "steps": { "total": 3, "applied": 3, "read": 1 }, "readLagMs": { "median": 2100 }, "expectations": { "passed": 4, "failed": 0 } },
  "report": "https://slurry.io/app/tests/runs/..." }

In CI

Start a run with trigger: "ci", wait for it, and fail the job unless the result is passed. Suites can also run on a schedule (hourly, daily or weekly). The command line does all of this in one step: npx @slurry/cli test billing --speed 60 --junit slurry-results.xml.

.github/workflows/slurry.yml
name: Slurry tests
on: [pull_request]
jobs:
  slurry:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # start your system pointed at the suite's simulations here, then:
      - name: Run the Slurry suite
        env:
          SLURRY_ACCOUNT_KEY: ${{ secrets.SLURRY_ACCOUNT_KEY }}
        run: |
          RUN=$(curl -sf -X POST https://api.slurry.io/v1/suites/billing/runs \
            -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY" -H "Content-Type: application/json" \
            -d '{"trigger":"ci","speed":60}' | jq -r .id)
          for i in $(seq 1 60); do
            R=$(curl -sf https://api.slurry.io/v1/runs/$RUN/wait -H "Authorization: Bearer $SLURRY_ACCOUNT_KEY")
            [ "$(echo "$R" | jq -r .state)" = finished ] && break
          done
          echo "$R" | jq '{result, summary, report}'
          [ "$(echo "$R" | jq -r .result)" = passed ]

Reset hook and webhook

Your system remembers what it saw in the last run; Slurry cannot reset it. Give the suite a resetHookUrl and Slurry POSTs a signed run.reset before each run (a non-2xx answer stops the run), or write cases that create fresh records each run with {{tag}}. A webhookUrl gets a signed run.finished.

run.finished
Slurry-Signature: t=1792771200,v1=hex(hmac_sha256(signingSecret, t + "." + body))
Slurry-Event: run.finished

{ "type": "run.finished", "id": "...", "suite": "billing", "kind": "suite", "result": "failed", "trigger": "ci",
  "summary": { ... }, "report": "https://slurry.io/app/tests/runs/..." }
Observed expectations can be decided by your own harness: POST /v1/runs/{run}/expectations/{index} with { "result": "passed" }. The run's result is recomputed and the webhook sent again.

Plans and limits

  • Test suites and the joystick come with the Team and Scale plans.
  • Suite snapshots are ordinary snapshots, one per simulation, and count towards your plan's storage like any other.
  • Up to 50 suites, 100 cases per suite, 12 simulations per suite, 200 steps per case and 20 runs queued or running at once.
  • MCP tools: list_actions, fire_action, create_test_suite, draft_test_cases, save_test_case, run_tests, get_test_run and more.