Skip to content

vouchfx public roadmap

This is the public, capability-level roadmap for vouchfx. Progress is measured by implemented capabilities and passing release checks, rather than a calendar target. Stable Core CLI and SDK packages include Compose support and bundled DCP tools. Companion repositories have independent release versions; validate the selected versions together using the release checks below.

For what changed when, see the changelog. For how project decisions are made, see governance.

Delivered

The following capabilities are implemented on main. The reference scenario combines REST, Kafka, PostgreSQL and webhooks; individual providers have their own tests. This catalogue describes the implementation, not a claim that every release requirement or open defect is resolved:

  • The compile-once memory model — .e2e.yaml compiles through Roslyn exactly once into a collectible AssemblyLoadContext; a memory-leak regression test over the full provider closure is a permanent CI gate.
  • Health-gated orchestration — headless .NET Aspire + Testcontainers topologies, started deterministically and torn down cleanly.
  • Twenty-five Core providers across eleven step families — http.rest, http.soap; db-assert for PostgreSQL, MySQL, SQL Server, MongoDB and DynamoDB; mq-publish/mq-expect for Kafka, RabbitMQ, NATS, Azure Service Bus and Redis Streams; cache-assert for Redis and Elasticsearch; mail-expect.smtp; webhook-listen.http; metrics-assert.prometheus; storage-assert.s3; trace-expect.otlp; script.csharp (inline code or an external file reference).
  • Engine-owned asynchronous verification — verifyMode: RETRY with bounded exponential backoff (Polly v8); authors never write Thread.Sleep.
  • Automatic state reset between sequential scenarios — PostgreSQL, SQL Server, MySQL, MongoDB, Redis and Elasticsearch dependencies are automatically reset (data cleared, structure preserved) after each scenario completes; broker, DynamoDB, MinIO and S3 (s3) dependencies are not reset; a failed reset surfaces as an environment error naming the dependency.
  • Transport security for authenticated infrastructure — a security: block declares TLS or mutual TLS per target: on any environment.services entry, and on a kafka dependency. The HTTP family (http.rest, http.soap, metrics-assert.prometheus) and the Kafka families (mq-publish.kafka, mq-expect.kafka) take their client transport from that declaration, presenting the declared client certificate under profile: mtls and pinning the peer to the declared caCert. Server-side keystore and certificate files are declared as serverArtifacts and streamed into the target's own container (never bind-mounted, so a remote daemon behaves like a local one). Before any step runs, a fail-closed confirmation probe connects with the same material a step will and reports a named level — TransportConfirmed, or AuthenticatedRoundTrip where the engine can complete a protocol round trip and, under mtls, show that the peer refused an anonymous client. A declaration the engine cannot confirm breaks CI with no gating flags set — one of four deliberate exceptions to "only Fail breaks CI by default" (the others are a parse failure, a suite refused before anything ran, and a scenario refused at a provider- or engine-surface guard; see CI integration). Which technologies this reaches, and which it does not, is set out per integration in the security compatibility matrix.
  • Frozen public contracts — golden files guard the language schema, provider SDK surface and event-wire contract. Environment extensions must preserve frozen provider APIs, helper bodies and event fields; evolution within v1.x is additive only.
  • The Provider SDK (Vouchfx.Sdk) with worked example providers, a conformance test harness, and the community provider hub.
  • Secrets as references — ${secret:env/…} and ${secret:vault/…}, resolved at execution time, redacted at the source; the redaction path has passed a penetration test.
  • Reporting off one event stream — terminal (WCAG-conscious), self-contained HTML, JUnit XML and a raw JSON Lines --events feed, all rendered from the same schema-versioned stream.
  • Live event streaming — vouchfx run --events-stream <path> writes an incremental, tailable JSON Lines stream to a file in real time as each step and attempt completes, useful for real-time tailing, CI progress streaming, and downstream consumers.
  • Compile-level suite tooling — validate, list, schema, and scaffold subcommands perform Docker-free YAML validation, step-type discovery, schema composition, and test-skeleton generation (machine-readable --json output where applicable), executed once at suite load time.
  • Coverage and gap analysis — vouchfx plan performs deterministic, read-only intersection of declared suites, run history, and available step catalogue, reporting coverage gaps (suite never run, step never exercised, dependency not asserted, vocabulary missing, service missing HTTP step), history-health signals (stale, flaky, fragile, inconclusive-prone), and identity ambiguity. Every gap finding carries structured hints the scaffold tool consumes. Docker-free; human-readable summary and JSON output; configurable health thresholds; taxonomy-aware exit codes including exit 5 for gaps (with --fail-on-gap).
  • A headless CLI with tag/owner/path/change-set selection, taxonomy-aware exit codes (only Fail breaks CI by default), parallel topology-per-scenario runs, and watch mode.
  • Editor tooling — a VSCode extension with schema-driven validation and autocomplete, C# highlighting in script.csharp blocks, and Test Explorer integration.
  • CI integration — a reusable GitHub Actions workflow and an include-able GitLab CI template.
  • A signed release pipeline — keyless cosign signatures, SLSA provenance attestations, CycloneDX SBOMs, and NuGet.org publication via Trusted Publishing (no long-lived keys).

Stable Core and ecosystem release checks

The first public releases shipped on 2026-07-08. The alpha and release-candidate series are historical; Core 1.0.0 was the first stable CLI and SDK milestone. Current stable distributions are available on GitHub Releases and NuGet.org, with cosign signatures, SLSA provenance and CycloneDX SBOMs. Install the latest stable CLI:

dotnet tool install --global vouchfx

Core and its companion repositories are released independently. The MCP server is available as the Vouchfx.Mcp .NET tool. Use the following checks when selecting release versions across repositories:

  • User-supplied environments — Core 1.0.0 introduced owned, fresh-per-scenario HTTP/TCP/PostgreSQL Docker Compose environments through a format-neutral registered-driver boundary. Pure binding supplies logical targets; acquisition supplies a ready lease and owns resource cleanup, without changing assertion providers. Existing inline environments remain supported. Other external formats require a registered driver implementation. Legacy 1.0.0-rc.6 does not accept environment.source. See the Compose guide for supported modes, prerequisites and recovery; further ecosystem acceptance remains separate.
  • Release-blocking defects and execution evidence — resolve failures in verdict reporting, secrets, lifecycle cleanup, supported security paths and consumer compatibility. Required runtime checks must prove that assertions executed; unavailable prerequisites or skipped probes do not establish release readiness.
  • Installed-package and fleet compatibility — validate the published Core CLI and SDK, independently released community providers, MCP tool, samples and telemetry backend together. Source-build success alone does not establish clean-install compatibility.

  • Current documentation — run the documented quickstarts against the selected release versions and validate all five built sites. Supported behaviour, prerequisites and known limits must match the implementation before release.

  • Validation of the end-to-end experience in real use — gathered in the open through the published sample applications, the migration guide (worked Postman, xUnit and SpecFlow ports), and the community provider hub, rather than a formal pilot cohort.

  • Core 1.0.6 is the current stable CLI release. The current Provider SDK (Vouchfx.Sdk, its Vouchfx.Sdk.Testing harness and their engine dependency closure) is published at 1.0.6 on NuGet.org. Its frozen provider contract remains the compatibility boundary for the v1.x series; rebuilt providers must declare their supported engine floor.
  • Some release artefacts gain further signatures (Windows Authenticode, macOS notarisation, GPG) only once the respective certificates are provisioned — cosign signatures and SLSA provenance are present on every artefact from day one, so verification is never blocked on those extras.

The v1.x series — additive only

The three public contracts carry an additive-only compatibility commitment for the v1.x series. Within that constraint, the near-term direction is:

  • Provider breadth through the community pathway — the Provider SDK is the mechanism by which new technologies arrive (gRPC, Oracle, SQS and the long tail) as Community providers — with the Vouched badge marking maintainer-reviewed ones (per reviewed version) — without engine changes. This is deliberately community-first; the provider hub is the front door. The launch catalogue (§5.7 of the DSL specification) now names the planned community catalogue and reserves one additional step family (realtime-expect) for future growth — trace-expect graduated out of that reserved state with trace-expect.otlp shipping as its Core provider, and http.soap joined http.rest as the http family's second Core provider. The first Community-tier provider (rpc.json-rpc, the tier's hub-hosted reference implementation) and a comprehensive implementation guide are live on the hub.
  • Provider distribution and consumption — the first Community provider package (Vouchfx.Community.JsonRpc) is published on NuGet.org and consumable today from a small custom runner built on the public engine API (the ledger-jsonrpc sample is the worked example); using an unpublished provider still means building from source with the provider referenced. The programme closing the remaining gap has additive steps. Community source submissions on the hub are already open: community-tier providers may be contributed as source into the hub repository's community/ directory (each building and testing in isolation, author-owned, explicitly unendorsed — hosting is not endorsement) so authors need no NuGet account to participate; see the hub's contributing guide. Still ahead: a provider directory loader — the CLI learns to load provider assemblies from a directory (a --providers flag, an environment variable, and a default probe location), guarded by the same reserved-namespace and version-conflict checks the engine already enforces — and then one-command acquisition: vouchfx providers install <id> resolves a step kind through the community registry, downloads the package, and places it in the providers directory, with a dependency-only Vouched metapackage installing the whole badge-holding set once Vouched providers exist. The frozen v1 SDK contract is what makes externally-built provider binaries loadable across the whole v1.x series.
  • Transport security for the remaining dependency kinds — 1.0 wires a client connection for exactly two target kinds: any declared service, and a kafka dependency. On every other dependency kind a security: block is rejected outright, and no profile or substitute declaration restores it. That narrowing is a release position, not a permanent boundary — the schema and the CHANGELOG both name it as a 1.1 capability, and widening it is additive, because both gates run at validation time and so decide only which suites validate at all. The per-integration breakdown, including which deferrals are 1.1 and which are out of scope, is the security compatibility matrix.
  • Additional secret sources — the ${secret:…} syntax is forward-compatible with cloud secret managers (Azure Key Vault, AWS Secrets Manager); adding them is configuration, not redesign.
  • Live GitLab validation — the GitLab CI template is static-validated and behaviourally cross-checked against the GitHub workflow; a live-instance run is the outstanding confirmation.
  • Editor depth — full in-block C# IntelliSense for script.csharp is a documented fast-follow.

v2 candidates

Two items are explicitly parked for v2, and the reasons are part of the trust story:

  • Per-file telemetry opt-out (metadata.telemetry: false) — deliberately not added in v1.x because it would mutate the frozen v1 schema. The v1 suppression surface (global consent, --no-telemetry, VOUCHFX_NO_TELEMETRY) already covers the privacy requirement; telemetry remains opt-in and off by default.
  • Richer control flow — the DSL models a single linear sequence by design; conditional and parallel steps are a deliberate post-v1 language decision, taken slowly because language mistakes are forever.

Deliberately not on the roadmap

Named here so nobody waits for them — build them via the Provider SDK or externally:

  • Automated migration tooling from Postman, k6, xUnit or SpecFlow. Worked porting examples are the supported path; an automated converter is out of scope.
  • Mid-suite checkpoint-and-resume — a crashed suite re-runs from the start.
  • Single-file executables — the engine discovers provider assemblies via Assembly.Location, which single-file publishing breaks; multi-file self-contained archives are the supported form.
  • UI/browser testing and unit testing — vouchfx tests the seams between systems; it is not a Playwright or xUnit replacement and will not become one.

Open source, permanently

vouchfx publishes its feature boundary explicitly, in advance, so nobody discovers it later:

Surface Tier
The engine, the YAML DSL, all twenty-five Core providers, the Provider SDK, the VSCode extension, the terminal renderer, the HTML report, the JUnit XML output, the structured event stream, the CLI runner, the secret-reference resolution mechanism, the reproducibility envelope. Apache-2.0, free permanently.
A hosted cross-run dashboard, managed Vault integration, cloud execution fabric (remote provisioning), agentic test planning/generation/healing, performance-testing tooling, run-history retention beyond the local cache, organisational analytics. Commercial (future), layered above the open-source engine — never carved out of it.
SSO/OIDC federation, on-premises Helm deployment, audit logging, SIEM export, data-residency commitments, dedicated support SLAs. Commercial enterprise (future).

Nothing in the first row will ever move down the table. The open-source engine is the product, not a demo.