Best-practice scenario library

Mixed maturity and release boundaries

These scenarios use both compatible and experimental capabilities in stable v0.7.0. They support design and acceptance; they do not replace the contract page and release notes for the version you deploy.

Capability coverage matrix

Capability coverage matrix

Capability or endpoint	Contract maturity	Scenarios
Project, Gateway Key, Route, environment isolation	Operational; verify deployed release	1, 2
Chat Completions, synchronous Responses, portable Messages	compatible	2, 3, 7
Project budget, Token Guard, Run budget, Attempt	Run portion is v0.7 experimental	3
Work Unit, Run, Outcome, Governance Export	v0.7 experimental	4, 5, 13
Deferred Responses	create is compatible; resource operations experimental	6
Streaming	Verify endpoint/Profile	7
Admin, Metrics, failure capture, backup, upgrade	Operational; verify deployed release	8
Embeddings and Rerank	Embeddings compatible; Rerank experimental but no creatable backend	9
Native Anthropic Messages and Token Count	compatible	10
Moderations, Images, Speech, Transcriptions	experimental	11
Files, Batches, Async Invocations	experimental; Async has no creatable backend	12

Scenario 1: isolate SaaS services with Projects and Keys

Separate environments first, then split workloads only where budgets, routing, or audit responsibility genuinely differ:

Scenario 1: isolate SaaS services with Projects and Keys

Object	Recommended split	Why
Project	support-prod, analytics-prod, support-staging	Independent Route authorization, daily budget, concurrency
Gateway Key	One per calling service; separate orchestration/evaluation	Independent revocation, rotation, least scope
Route alias	support-chat-prod vs support-chat-staging	Provider/Deployment changes stay out of business code

Test a staging Key against the production alias, revoke one Key without affecting others, confirm Project-level bills/Attempts, and prove the evaluator Key cannot infer.

For support governance, the orchestrator Key uses inference, work_unit:create, run:create, and run:attach; the independent acceptance service uses outcome:write. Do not share one all-powerful Key and do not create one Project per HTTP request.

Scenario 2: multi-Provider routing and failover

Put semantically equivalent, priced, healthy Deployments behind an ordered Route. Use separate failure domains. Never call a different Provider after streaming begins or after an ambiguous billed result. Verify which failures may fall back and that every Attempt retains its Profile and price evidence.

Use ordered for primary/backup; round_robin is distribution, not failover. Cross-target fallback applies only to compatible Chat, synchronous Responses, portable Messages, and Embeddings before any payload is exposed and only for unambiguous retryable failures. Authentication/permission errors, invalid fields, upstream 500/502/504, and connection loss after request transmission are ambiguous and do not silently switch. Native Anthropic fixes one target; deferred Responses fix a Deployment at submission; resource endpoints reject non-unique targets with 409.

When upstream supports enumeration, read its real model list; otherwise record why and explicitly declare the target. Establish capability evidence separately, then inject one controlled primary and backup failure. A model list proves who exists, never what it can do.

Scenario 3: bound Project, Run, and retry cost together

Use the Project for organizational budget, Token Guard for abnormal requests, and an experimental Run budget for one execution. Disable SDK retries, include every gateway retry/fallback in the same Run, and create a new Run when the complete business execution restarts.

Scenario 3: bound Project, Run, and retry cost together

Result	Handling
run_budget_exceeded	Halro does not auto-close; reduce the request within balance or close by business policy
token_guard_blocked	Adjust request/model/policy for cost threshold; wait for TTL or admin action for a temporary block
429 with Retry-After	Back off by the supplied value under an application retry cap
Ambiguous network result	Accept that it may be billed; a resend is a new cost event

Keep gateway.pricing_unknown_policy: reject in production. Run-attached requests require known price even if ordinary unknown-price traffic is globally allowed.

When price is unknown, Run admission returns 409 price_unavailable even if ordinary requests are otherwise allowed.

Create the Run with budget_micros_usd: 500000 for a 0.50 USD lifecycle. Halro checks Project daily budget and Run committed + reserved + pending atomically. Start Token Guard in observe or alert, establish absolute token/cost/concurrency/error/source thresholds, and enable temporary_block only after false-positive and unblock acceptance. EWMA alerts on relative anomalies; it is not a deterministic block.

Scenario 4: close the support-ticket outcome loop

Make the ticket plus policy snapshot the Work Unit. Freeze the resolution schema, create a Run for one agent attempt, attach model requests, then have an independent evaluator report the Outcome. Cost without an accepted outcome is not success.

For ticket 8472, the evaluator submits Definition ID, value, observed_at, and an opaque evidence_ref: ticket_8472_acceptance_3 or digest—never prompt, response, full review, URL, or credential. A still-open Work Unit or pending/inflight Attempt makes the Outcome provisional; only closed, settled work enters matured_units. Preserve cost complete/partial and outcome complete/partial/unknown; missing evidence is never zero or rejected by default. A rerun creates a second Run but the ticket enters the denominator once, and Outcome revision leaves history auditable.

The frozen definition is ticket_result v1; revising it later creates a new version rather than changing old work.

Scenario 5: immutable inputs for document extraction

Key the Work Unit by source digest and schema revision. Store the exact model Route, Run, result digest, validation evidence, and rejected-field details. A changed document or schema creates a new Work Unit version, not an overwrite.

Use distinct stable Idempotency-Key values for Work Unit create, Run create, close, and Outcome write. After timeout, replay the same key with the same body; a random new key is a new operation. Duplicate creation of one immutable version resolves to the same resource, while body change under the same key conflicts.

Scenario 6: deferred Responses for long work

Enable the feature on one Project, use background: true with a stable Idempotency-Key, respect the 256 KiB request, 1 MiB result, and 24-hour TTL limits, and poll using Retry-After. A restart fails in-flight work; a failed or oversized result may still be billed.

The same key, body, and X-Halro-Run-ID returns the same record; changing any attribution under the key returns 409. First successful retrieval starts a final 15-minute reread window, then GET returns 404. Budget admission occurs when work leaves the queue, so a queued item may later become failed. Deferred execution still obeys gateway.route_total_timeout (default two minutes); keep server.shutdown_timeout >= gateway.route_total_timeout and raise both for genuinely longer generation. Timeout ends as deferred_response_timeout and may already have cost. After restart, resubmit with a new idempotency key; create a new Run only when the full agent execution restarts.

Scenario 7: handle streaming interruption correctly

Record the request ID before consuming content, persist only application-safe partial output, and distinguish failure before and after the first event. Never concatenate a retry onto a partial answer. Reconcile the terminal Attempt even when the client disconnected.

For Responses, distinguish response.completed, response.incomplete, and error; incomplete may mean output limit rather than success. Test disconnect both before and after the first event and ensure each resend is a separate request and cost.

Scenario 8: production security, diagnostics, and recovery

Terminate TLS at a reviewed boundary, keep Admin and Metrics restricted, require MFA for remote Admin, trust only exact proxy CIDRs, back up the Master Key separately, monitor integrity and audit anchors, and perform isolated restore and rollback drills.

Use separate Gateway, Admin, and Metrics listeners. Non-loopback Metrics requires independent credential_file plus mTLS/client CA and must never reuse a Gateway Key. Keep Provider credentials, Master Key, and data under separate custody. gateway.failure_capture stays off by default; when enabled it stores Master-Key-encrypted, redacted upstream request/failure content that can still contain customer prompts and tool arguments. Require privacy/DLP review, bounded size/count/retention, audited reads, and expiry tests. A backup verify authenticates archive and manifest only; recovery requires an isolated restore drill with the original Master Key. Roll back by restoring the pre-upgrade backup, never by opening upgraded data with an old binary.

Set the exact admin.external_origin and admin.mfa_policy: required for remotely reachable Admin. Before upgrading governance data, stop Halro and run backup create plus backup verify. A local deployment with mfa_policy: optional is not automatically rejected, so the operator must enforce this production gate.

Scenario 9: Embedding index and Rerank planning

Version an index by source corpus digest, embedding target, dimensions, normalization, and Route. Never mix vectors from incompatible versions. Rerank availability requires a creatable served Profile; a contract entry alone is not an operational backend.

Use a versioned alias such as kb-embed-v2. Some Profiles accept only one string and dimension support differs. The only Rerank Profile, bedrock.agent-runtime.rerank.cohere-v3-5.v1, is withheld in this build, so /v1/rerank is not production-available; use accepted external ranking until a later build explicitly serves it.

Scenario 10: Anthropic native features and Token Count

Use the Anthropic SDK base URL without /v1. Choose portable mode for routable canonical behavior and Halro-Route-Mode: native only when one exact Anthropic-wire Profile is required; native mode disables cross-Provider fallback. Count Tokens is a real Provider call and can incur a fixed per-request fee.

POST /v1/messages/count_tokens supports only direct Anthropic, not portable mode or streaming. Token input/output rates are zero because no generation occurs, but USD / request still applies. It creates an Attempt and consumes request budget; it informs trimming but never replaces TPM, Token Guard, or cost admission. Count against the same alias/target later used for generation.

Scenario 11: content safety and media generation

Apply Project scope, capability filtering, input limits, redaction policy, output review, and retention before exposing media. Keep user identity out of prompts and URLs. Treat oversized or undeliverable output as potentially billed work.

Moderations, Images, Speech, and Transcriptions are separate experimental capabilities requiring uniquely resolved targets and least-privilege Keys; Chat availability proves none of them. Current creatable media support comes from openai.media-resources.v1; the implemented Bedrock Titan Image Profile is withheld. Preserve Halro request ID, business object ID, policy version, and user-visible state for each step, distinguishing safety rejection, unsupported capability, Provider failure, and cancellation.

Where the product risk model requires it, call /v1/moderations before media generation and decide separately whether generated output also needs review.

Scenario 12: Files, Batches, and Async resource lifecycle

Track Halro ownership, request and resource IDs, immutable input digests, idempotency, polling, terminal state, result retrieval, and deletion. Retrieval and deletion remain pinned to the creation target. Withheld-only Async or Rerank surfaces are not operational.

Files retain up to 30 days and Batch records 7 days. File creation selects with Halro-Route; Batch inherits the File target. The Async contract has a 7-day TTL but its sole bedrock.runtime.async.nova-reel-v1.v1 Profile is withheld, so this build cannot create/read/cancel it. Anthropic Batch input remains local and is validated line-by-line. Delete a File only after dependencies are terminal and results saved; deletion removes upstream object, metadata, and local content but never cost. Future Async cancellation fails closed because Bedrock exposes no cancel operation.

Scenario 13: hand Governance exports to FinOps

Generate verified Usage and Governance files, bind them with a signed manifest, keep their watermarks independent, propagate partial/unknown values, import idempotently, and reconcile aggregates plus unmatched identifiers.

Governance provides Work Unit, Run, Outcome, Definition, and manifest; Usage schema 6 provides Attempt, cost, work_unit_id, and run_id. Run rows contain budget, not actual cost. An administrator session creates the local Governance export; immediate Usage refresh requires stopped halro usage compact and halro usage verify. Validate both manifests, reconciliation, SHA-256, and counts before stable-ID join over a trusted or independently signed/MACed channel. Governance’s Accounting watermark covers the full Ledger, while Usage last_sequence covers exported Attempts only; they must not be compared for equality. Preserve cost_completeness, outcome_completeness, unknown Attempts, and partial cutoffs.

Production review checklist

• every Route target has current capability and price evidence;
• negative authorization, budget, and unsupported-field tests pass before Provider I/O;
• application retries are bounded and Request IDs are retained;
• backups, Master Key custody, restore, and rollback are proven;
• experimental features have a named pilot owner and exit criteria;
• Usage, Governance, and business outcomes reconcile without hiding unknowns.

Scenarios are not production evidence

These are design and acceptance methods. Mark a capability production-verified only after testing the real Provider, real data boundary, and recovery drill in your environment.