Testing strategy · vanilla-test

Shared · Node Same core suite

Host-neutral API and lifecycle behavior.

Shared · Chrome Same core suite

Real browser, native module, no transform.

Node tooling Integration suite

CLI, configuration, server, timeout, and paths.

Packed artifact Install verified

Exact tarball, offline reinstall, poisoned root dependencies, bare package imports, and a scoped native-browser import map.

Suite topology

The shared inventory contains 46 unique, narrowly scoped cases: 15 Unit, 10 Functional, 11 Integration, 1 Behavioral, and 9 Regression. That inventory is executed unchanged in both Node and Chrome.

Gate	Runtime	Evidence	Failure means
Shared categorized inventory	Node 22.12, Node 24, Chrome Stable	46 unique Unit, Functional, Integration, Behavioral, and Regression cases from the same source modules	The public Web-standard API does not behave consistently.
Node tooling integration	Node 22.12 and Node 24	Argument parsing, exit codes, result validation, glob matching, configuration, local server safety, timeouts and leaked handles	The developer-facing CLI or harness boundary is broken.
Native coverage gates	Node 24 and Chrome Stable	Independent executable-range, block-range, function-range, and executable-line measurements of `index.js`	One runtime did not exercise the complete configured core scope.
Packed npm artifact	Node 22.12, Node 24, Chrome Stable	`npm pack`, offline reinstall, exact dependency integrity checks, poisoned incompatible root packages, static/dynamic ESM, packed CLI, and a real-Chrome Given/When/Then run through an explicit scoped import map	The repository may pass while the published file set or native no-bundler dependency boundary is unusable.

Count unique cases separately from runtime executions.

The five category totals are one 46-case inventory. Node and Chrome execute that same inventory; they are evidence from different hosts, not 92 different cases. Tooling, coverage gates, and package smoke retain their own scopes.

Run tests locally

npm ci
npm test
npm run test:package-conformance

npm test executes all five shared sets through the thin test/node.js adapter, then runs the Node-only CLI, output, server-security, status-builder, and benchmark-harness tests. Run one shared set directly when narrowing a failure:

npm run test:unit
npm run test:functional
npm run test:integration
npm run test:behavioral
npm run test:regression

npm run coverage
# or isolate one collector
npm run coverage:node
npm run coverage:chrome

Check the terminal.

Test output, assertion errors, and collector failures appear there. Use each command's exit status as its automation result.

Chrome coverage requires Google Chrome Stable. The CLI intentionally does not substitute Chromium, Edge, Firefox, or another browser.

What the five shared sets check

Unit · 15 cases

Small contracts in isolation

Exports, EventTarget inheritance, strong-type delegates, strict state, typed public arguments, and delay boundaries.

Functional · 10 cases

Public lifecycle outcomes

Numbered expectations, pass/fail recording, active-test guards, undecided failure, reporting preconditions, and the empty suite.

Integration · 11 cases

Contracts working together

Mixed reports, immutable snapshots, rendered output, subscription-free scheduling, late completion subscriptions, native event-type coercion, cleanup, and isolated instances.

Behavioral · 1 case

Given/When/Then consumer journeys

A multi-step sign-in journey proves that a completed user flow reports success with its numbered steps in exact order.

Regression · 9 cases

Earlier fixes stay fixed

Duplicate state integrity, lenient and strict repeat decisions, report identity, log/event deduplication, and post-report sealing.

Run these checks in your browser Read the shared verifier

Interpret the live browser page correctly

The deployed verification page executes the current deployed test/CI.js in your browser. It proves that browser run only. The published status above records the corresponding Node and Chrome CI executions and includes commit and timestamp provenance.

A green live page is useful interactive evidence, not a substitute for the CI matrix.
A stale deployed page can lag current main; use the CI badge and displayed commit together.
Coverage percentages prove execution of the configured scope, not correctness outside that scope.

Rust native + browser topology

Rust's seven focused #[test] functions live once in src/lib.rs, including one Behavioral Given/When/Then consumer journey. Native Cargo runs them with full names and diagnostics. The browser compiles those exact same seven functions to wasm32-unknown-unknown; a separate host adapter performs passive rendered-text and delivery checks before calling the safe exported harness main() once in real Chrome.

Gate	Runtime	Evidence	Failure means
Rust native tests	Rust 1.85 native	7 lifecycle/result/error/Unicode/thread-safety/Behavioral library tests + 3 browser-build CLI tests + 1 README doctest	The native Rust API, browser builder, or documented usage is broken.
Rust browser harness	Real Chrome WebAssembly	Passive rendered-text/delivery checks plus 1 aggregate check that executes the same 7 library tests with zero host imports; status `0` passes	The browser page, WASM delivery, safe export, or Rust implementation is broken.
Rust package	Cargo package verifier	Explicit crate allowlist, dependency-free build, formatting, and warnings denied	The crates.io artifact or minimum-toolchain contract is broken.

The seven focused Rust functions collectively cover the shared lifecycle/report behavior: an all-passing Given/When/Then user flow in exact step order, exact unique descriptions and numbering, one active case, active-case guards, pass/fail recording, repeated same and opposing decisions with first-decision wins, undecided failure, reporting preconditions, ordered results, empty and nonempty cached reports, finality, and suite isolation. Rust's &str and borrowed result types enforce several JavaScript runtime checks at compile time; typed errors, Unicode isolation, and Send/Sync are Rust-specific guarantees.

cargo fmt --check
RUSTFLAGS="-D warnings" cargo test
cargo test behavioral
cargo run --bin cargo-vanilla-test -- --browser
cargo package

Count the Rust cases once.

Native and WASM are two executions of the same seven library tests, not fourteen unique tests. The zero-import Rust module cannot call JavaScript or the DOM; JavaScript validators, events, console/ANSI behavior, helpers, functions, and direct DOM interactions remain in the JavaScript lane. Browser-side passive page and delivery checks execute once in the host adapter, and native Cargo remains the detailed diagnostic path.

Run the Rust suite in this browser Read the Rust documentation

C# native + browser topology

Seven focused MSTest methods exercise the dependency-free net8.0 core, including one Behavioral Given/When/Then consumer journey. Native dotnet test runs them with the standard adapter. The companion tool publishes a net10.0 browser host that discovers and executes those same public, parameterless methods inside the .NET WebAssembly runtime.

Gate	Runtime	Evidence	Failure means
C# native tests	.NET 10 test host	7 lifecycle, result, error, null, immutability, rendering, and Behavioral tests against the `net8.0` library	The native C# contract is broken.
C# tool tests	.NET 10	4 focused parser, template, path, test-host, and benchmark-host tests	The browser builder contract is broken.
C# browser host	Real Chrome + .NET WebAssembly	The same 7 MSTest methods plus 5 visible page and AppBundle checks	The page, .NET runtime bundle, reflection host, or shared test inventory is broken.
C# packages	Clean local-package consumers	Library pack/install/run plus tool pack/install/version/browser-build smoke	The repository may pass while a shipped `.nupkg` artifact is unusable.

dotnet test dotnet/VanillaTest.slnx
dotnet test dotnet/tests/VanillaTest.Tests/VanillaTest.Tests.csproj --filter TestCategory=Behavioral
dotnet pack dotnet/src/VanillaTest/VanillaTest.csproj
dotnet pack dotnet/tools/VanillaTest.Tool/VanillaTest.Tool.csproj
cd dotnet/tests/VanillaTest.Tests
dotnet vanilla-test --browser --project VanillaTest.Tests.csproj
npx --yes node-http-server@10.0.0 --root dist/vanilla-test --port 4173

One C# test inventory, two hosts.

The browser output is a complete multi-file .NET AppBundle, not a single standalone .wasm file. The focused host supports public parameterless [TestMethod] methods and the documented initialize, cleanup, ignore, async, and disposal behaviors; native MSTest remains the full diagnostic host.

The benchmark gate is separate from the seven-test correctness gate. It publishes the shared one-million-case C# pipeline as its own AppBundle, runs one warmup and five managed samples in real Chrome, and requires the live benchmark page to reach a passed state.

Run the C# suite in this browser Read the C# documentation

Benchmark the aligned lifecycle and complete pipeline

The benchmarks are separate, auditable performance workloads—not inflated correctness-test counts. The aligned recorder measures JavaScript, Rust, and C# native/browser lanes with one initialized-runtime boundary. The JavaScript framework pipeline separately retains cold startup, native V8 coverage, validation, JSON, LCOV, standalone HTML generation, and teardown.

npm ci --prefix benchmark
# six equivalent language/runtime lanes
npm run benchmark:languages

# JavaScript framework cold pipelines
npm run benchmark

# quick end-to-end harness verification
npm run benchmark:smoke

The aligned recorder discards one warmup and retains five fresh million-case lifecycle samples in each initialized runtime. Only the richer built-in node:test runner and pinned Mocha appear as competitors in the separate JavaScript framework pipeline; raw loops and less capable micro-runners do not enter that ranking.

Open benchmarks and machine specs Inspect the complete harness

Add the right kind of test

Change	Put the test here	Reason
Isolated export, type, delegate, or boundary contract	`test/shared/unit.js`	One small contract should fail for one focused reason.
Public lifecycle outcome	`test/shared/functional.js`	Exercise behavior through the consumer-facing sequence.
Multiple public contracts working together	`test/shared/integration.js`	Keep cross-contract behavior separate from isolated units.
A consumer-facing Given/When/Then journey	`test/shared/behavioral.js`	Describe the observable user flow and verify its ordered public result.
A previously fixed state or idempotence edge	`test/shared/regression.js`	Name the protected behavior without copying it into another set.
CLI, config, filesystem, process, or server behavior	`test/tooling.js`	These are intentionally Node-specific host concerns.
Published-file boundary	Package smoke step	Repository-relative imports can hide packaging defects.

Keep the shared verifier independent of vanilla-test itself as an oracle. Its small Web-standard assertion helpers prevent the framework from certifying its own behavior.

Different suites prove different claims.

Latest published result