Skip to content
v0.8.4stable

Integrate Run Governance

Run Governance attributes every Provider Attempt to a Run, enforces a lifecycle budget in addition to the Project daily budget, and lets an independent evaluator report a structured Outcome. Halro does not split work, orchestrate agents, or decide success: the application owns lifecycle, Halro owns cost/attribution evidence, and the evaluator owns Outcomes.

Divide the objects correctly
ObjectBusiness meaningExample
Work UnitWork that should produce exactly one final business resultTicket 8472
RunOne execution of that Work UnitFirst agent execution, then a second after failure
Outcome DefinitionVersioned allowed results and success criteriaticket_result v1
OutcomeOne evaluator assertion or revisionaccepted

A Run may contain multiple requests, retries, and fallbacks. Restarting the whole agent execution closes the old Run and creates a new one under the same Work Unit. The final result counts once; cost includes every Run.

Three common Work Unit boundaries
WorkflowWork UnitRunOutcome DefinitionDo not use
SupportOne ticket needing final resolutionOne agent handling; human-return rerun is newticket_result = accepted/rejectedOne Work Unit per model call
Document extractionFile ID + immutable versionOne complete extraction pipelineextraction_result = accepted/rejectedReusing it after file replacement
Code changeIssue + target source baselineOne agent repair executionreview_result = mergeable/changes_requestedNew Run for each Provider fallback

The test is “how many final business results should be counted?” One result belongs to one Work Unit; a restarted full execution is a new Run; Halro internal retry/fallback stays in the original Run.

In Admin Console → Projects, enable Run Governance and set default/maximum budget, default/maximum TTL, active Run limit, and open Work Unit limit. The Console uses USD and hours; the API uses micros USD and seconds (1 USD = 1,000,000 micros USD).

Prepare scopes and the Project
CallerScope
Agent orchestrationinference, work_unit:create, run:create, run:attach; optionally governance:read
Business evaluatoroutcome:write; optionally governance:read

Existing inference-only Keys receive no governance scope. A Run-creating Key does not automatically receive outcome:write. Project and Run budgets are distinct: Project is the business hard boundary; Run is one lifecycle inside it.

Create the Definition under Admin Console → Run Governance:

{
"name": "ticket_result",
"data_type": "CATEGORICAL",
"allowed_values": ["accepted", "rejected"],
"success_values": ["accepted"],
"description": "Final ticket acceptance result",
"enabled": true
}

The first release accepts BOOLEAN and CATEGORICAL with 2–16 values. Names and historical versions are immutable; edits create a new version. Work Unit creation freezes the enabled version, so later Definition upgrades do not rewrite existing work.

  1. Create a Work Unit with a stable idempotency key and save wku_...:

    Terminal window
    curl https://halro.example.com/halro/v1/work-units \
    -H "Authorization: Bearer $HALRO_ORCHESTRATOR_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: 8472-work-unit-v1" \
    -d '{"outcome_definition_ids":["odef_xxx"]}'
  2. Create a Run and save run_...:

    Terminal window
    curl https://halro.example.com/halro/v1/runs \
    -H "Authorization: Bearer $HALRO_ORCHESTRATOR_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: 8472-run-1-v1" \
    -d '{"work_unit_id":"wku_xxx","budget_micros_usd":500000,"ttl_seconds":3600}'

    Omitted budget or TTL uses the Project default.

  3. Attribute model calls with X-Halro-Run-ID:

    Terminal window
    curl https://halro.example.com/v1/chat/completions \
    -H "Authorization: Bearer $HALRO_ORCHESTRATOR_KEY" \
    -H "Content-Type: application/json" \
    -H "X-Halro-Run-ID: run_xxx" \
    -d '{"model":"chat","messages":[{"role":"user","content":"Handle ticket 8472"}]}'

    Do not send Work Unit ID. Before Provider I/O, Halro atomically checks Project daily budget and Run committed + reserved + pending. Insufficient balance returns 403 run_budget_exceeded. OpenAI, Anthropic, and deferred Responses use the same header.

  4. Close the Run, then close the Work Unit only after every Run is closed and the whole work will not restart:

    Terminal window
    curl https://halro.example.com/halro/v1/runs/run_xxx/close \
    -H "Authorization: Bearer $HALRO_ORCHESTRATOR_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: 8472-run-1-close" \
    -d '{"reason":"completed"}'
    curl https://halro.example.com/halro/v1/work-units/wku_xxx/close \
    -H "Authorization: Bearer $HALRO_ORCHESTRATOR_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: 8472-work-unit-close" \
    -d '{}'
  5. Let the independent evaluator report the Outcome:

    Terminal window
    curl https://halro.example.com/halro/v1/work-units/wku_xxx/outcomes \
    -H "Authorization: Bearer $HALRO_ACCEPTANCE_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: 8472-outcome-v1" \
    -d '{"definition_id":"odef_xxx","value":"accepted","observed_at":"2026-09-04T08:30:00Z","evidence_ref":"ticket_8472_acceptance_3","evidence_sha256":"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}'

Send only Definition ID, structured value, observation time, and optional redacted opaque evidence reference/digest. evidence_ref is at most 128 characters and must not look like a URL or credential; SHA-256 is 64 lowercase hex digits. Never send prompts, responses, full review text, URLs, tokens, or passwords.

POST a revision with a new idempotency key and the current Outcome ID:

{
"definition_id": "odef_xxx",
"value": "rejected",
"observed_at": "2026-09-04T09:00:00Z",
"supersedes_outcome_id": "out_previous"
}

Each result allows at most 20 revisions. Writes stop 30 days after Work Unit closure. An Outcome is provisional while its Work Unit is open or Attempts remain pending/inflight and must not be treated as mature final result.

Drill from Work Unit to Runs and Attempts. Review Outcome, coverage, success rate, known/estimated/in-progress/unknown cost, and cost per success.

  • Outcome coverage = Work Units with an Outcome / eligible Work Units.
  • Success rate = successful Outcomes / evaluated Outcomes.
  • Cost per success = known model cost of mature Work Units / successful Work Units.
  • partial and unknown are not zero; they mean missing results, active work, or incomplete cost.

Built-in cohorts cover at most 90 days or 100,000 Work Units. Use Governance Export for larger analysis; see FinOps handoff.

  • Create and verify a pre-upgrade backup: first opening a new version migrates metadata and enables a Ledger feature epoch; rollback requires restoring that backup, not opening upgraded data with the old binary.
  • Test wrong-Project, expired, closed, unknown, and over-budget Run IDs.
  • Prove abandoned Runs expire and reservations settle.
  • Separate orchestrator and evaluator Keys.
  • Include Governance in backup, restore, and export acceptance.
  • Pilot a real workflow and document Work Unit boundary, Definition, evaluator, observation window, decision use, owner, rollback, and maturity exit criteria.

See Best-practice scenarios for cross-feature examples.