Benchmarks

AxiomCore publishes no "typical" performance numbers. Instead, the CLI includes a benchmark you can run on your own contract and machine. A benchmark claim is useful only with its contract, command, machine class, CLI version, warm-up, sample count, and report. This page also lists the usage data the CLI sends.

Run the benchmark

Run it from a backend directory that contains axiom.acore:

axiom benchmark --iterations 20 --warmup 3 --output axiom-benchmark.json

The command:

• runs the warm-up builds, then measures each iteration;
• reports min, median, p95, and max (nearest-rank percentiles);
• writes a versioned JSON report;
• restores any existing axiom.axiom in the current directory when it finishes or fails, or removes the one it created; and
• uploads nothing.

Metrics

Report format

{
  "schema_version": 1,
  "cli_version": "0.149.1",
  "contract": "axiom.acore",
  "variant": "default",
  "iterations": 20,
  "warmup_iterations": 3,
  "build_ms": { "min": 11.2, "median": 12.1, "p95": 14.8, "max": 14.8 },
  "artifact_decode_ms": { "min": 0.1, "median": 0.1, "p95": 0.2, "max": 0.2 },
  "artifact_bytes": 8124,
  "endpoints": 9
}

These values only illustrate the schema; they are not published AxiomCore performance claims.

Attach the report to a pull request, or keep it in a dedicated results repository. Treat it as evidence, not a pass or fail gate: shared CI runners are too noisy for an honest regression threshold. Set thresholds only after you collect a stable baseline on dedicated hardware.

Publish a fair comparison

1. Pin the CLI version and the fixture commit. Do not compare contracts of different sizes without reporting endpoint count and artifact bytes.
2. Use a release build of the CLI on an otherwise idle machine.
3. Run at least 3 warm-up and 20 measured iterations.
4. Report median and p95, not only the fastest sample.
5. Track build time, decode time, artifact size, browser runtime start-up, and request overhead separately. Do not merge them into one "Axiom overhead" number.
6. Run browser benchmarks as a separate suite. Record the browser version, network conditions, cache state, contract size, and whether WebAssembly was cold or warm.

What is not measured yet

axiom benchmark covers only the compiler and artifact boundary. AxiomCore has no published benchmark yet for web runtime start-up, request overhead, cache behavior, or compatibility analysis. Measure those yourself with the protocol above before you rely on them.

The Experimental executable extensions have their own hardening record for server and headless hosts, Chromium Workers, the Android Emulator, and the iOS Simulator: cold load, warm invocation p99, cancellation, teardown, worker recovery, and bounded streams. It does not certify physical mobile devices or all browser engines. See the Extension reference.

CLI usage telemetry

The CLI sends usage telemetry only for commands that contact Axiom services, and only from an installation registered with a referral code. Local commands such as check, build, run, serve, test, ui, database, and inspect send nothing. A command pointed at a local control plane sends nothing either.

Each event contains exactly these fields:

The CLI never sends raw command arguments, repository or project names, paths, URLs, contract contents, headers, tokens, credentials, or raw error messages. The wire-compatible args field always contains only {"schema_version":1}.

Turn telemetry off for a process with AXIOM_TELEMETRY set to 0, false, or off:

AXIOM_TELEMETRY=0 axiom release backend.axiom

Runtime telemetry is separate. A signed contract's scoped OTLP settings configure it for a project's request evidence, and it is never reused for CLI analytics. See Observability. Analytics on the public websites is described in Docs site privacy.