---
title: "High performance sandbox benchmarks"
description: "Eleven sandbox environments, including Runloop and run.cloud, benchmarked on one 4 vCPU / 8 GB target across synthetic tests and three cold repo pipelines."
url: https://starsling.dev/hpc-sandbox-benchmarks
canonicalUrl: https://starsling.dev/hpc-sandbox-benchmarks
---

# High performance sandbox benchmarks

Compare 11 sandbox providers on one target hardware shape, across real developer, CI/CD, and reinforcement-learning environment workloads.

Run 31066359914 - commit cf002382d6be - run date 2026-08-06 - target 4 vCPU - 8 GB - 40 GB - n = 2 to 12 replicate sandboxes per case - 11 providers validated.

We measured the end-to-end time developers and agents actually experience when using 11 sandbox providers to complete software engineering tasks. Going from a ticket to a PR is a multi-phase workflow, and a sandbox that wins on cold-start time can still lose the pipeline that follows it.

- Real-workload suites: 3 repositories, each run cold on 12 replicate sandboxes through that repository's own CI tasks: clone, install dependencies, lint, typecheck, build. The repositories are Better Auth, Mastra, and OpenClaw.
- Synthetic suites: 10 versioned Phoronix Test Suite profiles. Of those, 8 are industry-standard profiles published through OpenBenchmarking.org. Each suite runs on 3 replicate sandboxes and repeats until Phoronix's own convergence criterion is met. Methodology lists what every suite measures.

[Explore any metric](#explorer) - [scan the full table](#all-metrics).

## Results explorer

The HTML page at `/hpc-sandbox-benchmarks` includes one interactive results explorer for real-world pipelines and synthetic benchmarks. Choose a statistic - min, p50, p95, or max - and the ranked bars and per-provider tooltips update from the same catalogued data the static tables below carry.

### Real-world pipeline durations

Cold, end-to-end repository pipelines on freshly created sandboxes, run through each repo's own CI tasks in the suite's execution order; totals are the sum of per-task medians.

#### Better Auth - 10 tasks

Better Auth's own CI task matrix, run cold on 12 fresh sandboxes per environment. Each task's median pools those sandboxes, and the total is the sum of the task medians. Task names are the repository's own: the repo-wide `lint types` pass and the root `typecheck` task are separate jobs.

| Provider | Total (s) | vs fastest | git clone (s) | cold install (s) | lint (Biome) (s) | lint deps (Knip) (s) | lint format (s) | lint spell (s) | lint types (s) | lint packages (s) | typecheck (s) | build (s) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Namespace | 145.6 | fastest | 0.97 | 24.3 | 2.54 | 7.81 | 2.12 | 5.19 | 24.3 | 2.06 | 30.0 | 46.3 |
| Daytona (VM) | 163.7 | x1.12 | 1.53 | 13.9 | 2.94 | 10.3 | 2.86 | 7.24 | 24.9 | 2.46 | 39.3 | 58.2 |
| Blaxel | 164.0 | x1.13 | 0.63 | 11.7 | 3.22 | 10.1 | 2.92 | 6.85 | 26.7 | 2.43 | 41.5 | 57.9 |
| Novita | 205.5 | x1.41 | 1.90 | 15.9 | 3.79 | 12.3 | 3.14 | 7.93 | 36.0 | 3.31 | 46.1 | 75.0 |
| Modal (VM) | 206.6 | x1.42 | 0.90 | 18.8 | 3.86 | 13.0 | 3.43 | 8.78 | 34.1 | 3.21 | 50.1 | 70.3 |
| microsandbox | 237.9 | x1.63 | 1.63 | 20.6 | 4.03 | 12.9 | 3.41 | 10.00 | 40.7 | 3.36 | 60.4 | 80.9 |
| Vercel | 258.0 | x1.77 | 0.82 | 19.5 | 4.28 | 14.8 | 4.39 | 11.1 | 44.3 | 3.69 | 65.4 | 89.7 |
| E2B | 300.1 | x2.06 | 1.43 | 20.2 | 5.08 | 19.0 | 5.26 | 13.4 | 52.3 | 4.27 | 76.7 | 102.5 |
| run.cloud | 301.0 | x2.07 | 1.95 | 54.9 | 4.09 | 16.7 | 4.36 | 10.9 | 48.5 | 3.41 | 65.5 | 90.7 |
| Runloop | 407.2 | x2.80 | 2.08 | 28.0 | 6.40 | 22.4 | 6.69 | 16.7 | 76.8 | 6.52 | 100.2 | 141.5 |
| Modal (gVisor) | 424.4 | x2.92 | 2.68 | 35.9 | 10.3 | 28.0 | 6.35 | 14.7 | 101.6 | 9.41 | 79.2 | 136.4 |

#### Mastra - 4 tasks

Mastra's CI tasks scoped to packages/core. A task that produced no value on any environment is excluded from every total rather than being counted as zero; the Data quality section preserves the run's failure record.

| Provider | Total (s) | vs fastest | git clone (s) | cold install (s) | lint:format (s) | build:core (s) |
|---|---|---|---|---|---|---|
| Namespace | 166.7 | fastest | 3.74 | 40.0 | 67.1 | 55.8 |
| Blaxel | 200.6 | x1.20 | 2.08 | 37.8 | 88.9 | 71.8 |
| Daytona (VM) | 205.3 | x1.23 | 2.39 | 39.0 | 93.2 | 70.7 |
| Novita | 236.9 | x1.42 | 4.06 | 46.8 | 101.8 | 84.2 |
| Modal (VM) | 260.5 | x1.56 | 2.30 | 51.6 | 115.0 | 91.5 |
| microsandbox | 273.0 | x1.64 | 3.08 | 58.6 | 116.4 | 94.9 |
| run.cloud | 342.8 | x2.06 | 3.30 | 67.2 | 150.9 | 121.4 |
| E2B | 367.3 | x2.20 | 3.35 | 69.8 | 165.8 | 128.4 |
| Vercel | 416.6 | x2.50 | 2.89 | 73.9 | 189.3 | 150.6 |
| Runloop | 457.1 | x2.74 | 5.43 | 92.4 | 199.8 | 159.4 |
| Modal (gVisor) | 464.4 | x2.79 | 6.49 | 97.2 | 189.2 | 171.5 |

#### OpenClaw - 5 tasks

OpenClaw's own CI tasks. A provider is compared only when it completed every task in the comparison, so a partial pipeline cannot appear faster because some of the work is missing.

| Provider | Total (s) | vs fastest | git clone (s) | cold install (s) | lint (extension channels) (s) | typecheck (tsgo) (s) | typecheck (test tree) (s) |
|---|---|---|---|---|---|---|---|
| Namespace | 166.5 | fastest | 6.21 | 14.7 | 51.6 | 13.7 | 80.4 |
| Daytona (VM) | 198.5 | x1.19 | 3.55 | 15.7 | 61.7 | 17.3 | 100.3 |
| Modal (VM) | 236.4 | x1.42 | 3.18 | 18.2 | 73.9 | 20.8 | 120.3 |
| Novita | 243.2 | x1.46 | 4.18 | 19.0 | 69.3 | 24.3 | 126.4 |
| microsandbox | 268.0 | x1.61 | 4.18 | 20.3 | 89.5 | 23.0 | 131.0 |
| run.cloud | 317.3 | x1.91 | 4.28 | 21.3 | 101.7 | 27.1 | 163.0 |
| Vercel | 335.3 | x2.01 | 4.11 | 19.7 | 111.0 | 30.1 | 170.5 |
| E2B | 356.5 | x2.14 | 4.48 | 19.9 | 110.3 | 37.4 | 184.4 |
| Runloop | 410.0 | x2.46 | 6.78 | 24.3 | 133.7 | 36.4 | 208.8 |
| Modal (gVisor) | 605.5 | x3.64 | 10.3 | 31.9 | 177.8 | 70.3 | 315.2 |
| Blaxel | no comparable data (failed) | - | - | - | - | - | - |

- Blaxel: failed - PTS ran but every trial failed for 4 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_types (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded

## Precision

Each row shows the single metric with that provider's widest total (min-max) spread, ranked by IDR (p10-p90). The distribution is every trial the run recorded for that metric, pooled across the provider's replicate sandboxes. Phoronix repeats a benchmark several times inside each sandbox, and all of those measurements count, so a row's n is trials rather than sandboxes. Convergence-mode suites repeat until Phoronix settles, which means a noisy sandbox contributes more trials than a quiet one and weighs on the interval accordingly. Metrics that depend on the public Internet, including WAN iperf, repository cloning, and dependency installation, are excluded so the figure reflects sandbox precision; local networking such as iperf loopback remains eligible. The suite and metric vary by provider and are named on every row. Because the units are unlike, every interval is given as a percentage of its own metric's p50.

| Environment | Worst precision item | Distribution | Total (MIN-MAX) | IQR (p25-p75) | IDR (p10-p90) |
|---|---|---|---|---|---|
| E2B | Real world / Better Auth / typecheck | p50 76.7 s; n=12 trials across 12 sandboxes | 31.8% | 11.1% | 15.0% |
| Blaxel | Synthetic / STREAM / Copy | p50 118,960 MB/s; n=58 trials across 3 sandboxes | 41.2% | 8.8% | 31.1% |
| microsandbox | Synthetic / fio / rand write 4KB, O_DIRECT (IOPS) | p50 263,500 IOPS; n=6 trials across 3 sandboxes | 57.3% | 23.6% | 41.6% |
| Runloop | Synthetic / fio / seq write 1MB, O_DIRECT (IOPS) | p50 2,934 IOPS; n=6 trials across 3 sandboxes | 67.5% | 43.2% | 62.4% |
| Daytona (VM) | Synthetic / STREAM / Scale | p50 170,975 MB/s; n=15 trials across 3 sandboxes | 76.0% | 72.0% | 74.0% |
| Novita | Synthetic / fio / rand write 4KB, O_DIRECT (MB/s) | p50 433 MB/s; n=6 trials across 3 sandboxes | 81.8% | 64.8% | 80.1% |
| Vercel | Synthetic / STREAM / Copy | p50 40,265 MB/s; n=20 trials across 3 sandboxes | 110% | 31.7% | 108% |
| run.cloud | Synthetic / Node.js web tooling / runs per second | p50 11.77 runs/s; n=45 trials across 3 sandboxes | 149% | 82.9% | 131% |
| Modal (VM) | Synthetic / iperf3 / loopback TCP, 1 stream | p50 61,652 Mbits/sec; n=6 trials across 3 sandboxes | 180% | 94.8% | 149% |
| Modal (gVisor) | Synthetic / iperf3 / loopback UDP, 10G objective | p50 169 Mbits/sec; n=6 trials across 3 sandboxes | 243% | 170% | 234% |
| Namespace | Synthetic / Hardlink / throughput | p50 5.25 bogo ops/s; n=6 trials across 3 sandboxes | 262% | 193% | 260% |

## Methodology

Every environment ran the same harness against freshly created sandboxes targeting 4 vCPU - 8 GB RAM - 40 GB disk. Synthetic suites ran on 3 replicate sandboxes each, repeating passes until Phoronix's own convergence criterion was met, so pass counts vary by how noisy the sandbox was; each real repository pipeline (Better Auth, Mastra, and OpenClaw) ran cold on 12 fresh sandboxes, through that repository's own CI tasks. Every figure is a median (p50) pooled across an environment's sandboxes, and every cell keeps the individual trials behind that median, which is what the Precision section reads.

The 10 synthetic suites are described by their own Phoronix test profiles: 8 published on OpenBenchmarking.org, 2 written for this harness and therefore without a profile page to cite.

| Name | Test type | Description | Project site |
|---|---|---|---|
| [Flexible IO Tester](https://openbenchmarking.org/test/pts/fio) v2.1.0 | Disk | FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. | [fio.readthedocs.io/en/latest/fio_doc.html](https://fio.readthedocs.io/en/latest/fio_doc.html) |
| Hardlink Throughput (harness-local) | Disk | Measures hardlink() syscall throughput using the stress-ng --link stressor. Mirrors the per-syscall pattern that pnpm install hits when materialising thousands of node_modules entries from a content-addressable store. Reports bogo ops/sec (real time) - one bogo op = one create+unlink hardlink cycle. | [github.com/ColinIanKing/stress-ng](https://github.com/ColinIanKing/stress-ng) |
| [Stream](https://openbenchmarking.org/test/pts/stream) v1.3.4 | Memory | This is a benchmark of Stream, the popular system memory (RAM) benchmark. | [cs.virginia.edu/stream](http://www.cs.virginia.edu/stream/) |
| [iPerf](https://openbenchmarking.org/test/pts/iperf) v1.2.0 | Network | iPerf is a network bandwidth throughput testing software. This test profile is used for automated testing of an iperf client and requires you have access to an iperf server. | [software.es.net/iperf](https://software.es.net/iperf/) |
| iPerf WAN (harness-local) | Network | This test measures sustained WAN TCP throughput with iperf3 against the nearest reachable server from a curated public list, chosen per run by RTT probe. It uses eight parallel streams so that high bandwidth-delay-product paths are not understated, and reports receiver-side goodput. | [software.es.net/iperf](https://software.es.net/iperf/) |
| [Node.js V8 Web Tooling Benchmark](https://openbenchmarking.org/test/pts/node-web-tooling) v1.0.1 | Processor | Running the V8 project's Web-Tooling-Benchmark under Node.js. The Web-Tooling-Benchmark stresses JavaScript-related workloads common to web developers like Babel and TypeScript and Babylon. This test profile can test the system's JavaScript performance with Node.js. | [v8.github.io/web-tooling-benchmark](https://v8.github.io/web-tooling-benchmark/) |
| [Git](https://openbenchmarking.org/test/pts/git) v1.1.0 | System | This test measures the time needed to carry out some sample Git operations on an example, static repository that happens to be a copy of the GNOME GTK tool-kit repository. | [git-scm.com](https://git-scm.com/) |
| [PostgreSQL](https://openbenchmarking.org/test/pts/pgbench) v1.15.0 | System | This is a benchmark of PostgreSQL using the integrated pgbench for facilitating the database benchmarks. | [postgresql.org](https://www.postgresql.org/) |
| [PyBench](https://openbenchmarking.org/test/pts/pybench) v1.1.3 | System | This test profile reports the total time of the different average timed test results from PyBench. PyBench reports average test times for different functions such as BuiltinFunctionCalls and NestedForLoops, with this total result providing a rough estimate as to Python's average performance on a given system. This test profile runs PyBench each time for 20 rounds. | [svn.python.org/projects/python/trunk/Tools/pybench](http://svn.python.org/projects/python/trunk/Tools/pybench/) |
| [SQLite Speedtest](https://openbenchmarking.org/test/pts/sqlite-speedtest) v1.0.1 | System | This is a benchmark of SQLite's speedtest1 benchmark program with an increased problem size of 1,000. | [sqlite.org](http://www.sqlite.org/) |

- **Pipeline totals** - The p50 is the sum of per-task medians, so it exactly matches the task numbers reported beside it. In the Results explorer, min, p95, and max come from end-to-end totals aligned within the same replicate sandbox and trial; task percentiles are never added together.
- **A column is a variant, not a vendor** - Modal (gVisor) and Modal (VM) are one account's two runtimes booting the same image. Variants attempted without validated metrics are disclosed under Data quality rather than rendered as empty columns.
- **A column is a fleet, not a machine** - Each cell pools an environment's replicate sandboxes, and the environments table flags every provider whose sandboxes reported more than one host CPU model.
- **n counts trials, r counts sandboxes** - A convergence-mode cell's n is pooled passes, not sandboxes. A sandbox that converged slowly contributes more passes to the same median. Where a sandbox or a whole suite was lost, the surviving replicates still carry the cell, which is why the replicate count in the summary above is a range (2 to 12) rather than one number; Data quality records every such loss.
- **Network figures isolate the sandbox** - iperf3 loopback, using TCP at 1 and 10 streams plus a 10 G-objective UDP datagram test, measures the sandbox's own network stack with no Internet path. The WAN pair measures both directions against the nearest curated public iperf3 server, chosen per run by RTT probe.
- **Task names are each repository's own** - Labels come from the dataset's catalog, which uses the repo's names: Better Auth's `lint types` (its repo-wide type-lint pass) is a different task from its root `typecheck`.

Data provenance: run document data/sandbox-benchmarks/runs/31066359914.json - metric catalog data/sandbox-benchmarks/catalog.json (schema @ cf002382d6be) - test profiles data/sandbox-benchmarks/pts-profiles.json (packages/schema/src/pts-profiles @ 3153398556fb) - derived by scripts/generate-sandbox-benchmark-data.ts.

- Benchmark harness and raw results: [starslingdev/hpc-sandbox-benchmarks](https://github.com/starslingdev/hpc-sandbox-benchmarks)
- Write-up: [Beyond cold starts: benchmarking sandboxes for real workloads](https://starsling.dev/blog/beyond-cold-starts-benchmarking-sandboxes-for-real-workloads)

## All metrics

Every catalogued metric this run emitted, labeled and grouped exactly as the dataset's own leaderboard labels it. Ratios are relative to the row's best provider; spread is worst / best. "(lower)" means lower is better; "(derived)" marks a value computed from other rows. Pipeline task rows are nested (-) under their suite's total.

### CPU

| Metric | Unit | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel | Spread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Node.js web tooling | runs/s | 19.75 (x1.43) | 18.75 (x1.51) | 11.54 (x2.45) | 17.26 (x1.64) | 9.74 (x2.90) | 17.96 (x1.57) | **28.25** (best) | 16.65 (x1.70) | 11.77 (x2.40) | 10.87 (x2.60) | 13.44 (x2.10) | x2.9 |

### Disk I/O

| Metric | Unit | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel | Spread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| fio rand read 4KB, O_DIRECT (IOPS) | IOPS | 218,000 (x1.47) | 271,500 (x1.18) | 47,350 (x6.78) | **321,000** (best) | 32,200 (x9.97) | 227,000 (x1.41) | 256,500 (x1.25) | 97,100 (x3.31) | 256,000 (x1.25) | 165,000 (x1.95) | 144,000 (x2.23) | x10.0 |
| fio rand read 4KB, O_DIRECT (MB/s) | MB/s | 851 (x1.48) | 1,060 (x1.18) | 185 (x6.78) | **1,255** (best) | 126 (x10.00) | 888 (x1.41) | 1,003 (x1.25) | 380 (x3.30) | 1,000 (x1.25) | 644 (x1.95) | 563 (x2.23) | x10.0 |
| fio rand write 4KB, O_DIRECT (IOPS) | IOPS | 210,000 (x1.25) | 237,500 (x1.11) | 48,350 (x5.45) | **263,500** (best) | 26,350 (x10.00) | 210,000 (x1.25) | 248,500 (x1.06) | 110,850 (x2.38) | 198,000 (x1.33) | 156,000 (x1.69) | 178,000 (x1.48) | x10.0 |
| fio rand write 4KB, O_DIRECT (MB/s) | MB/s | 822 (x1.25) | 928 (x1.11) | 189 (x5.44) | **1,028** (best) | 103 (x9.98) | 822 (x1.25) | 970 (x1.06) | 433 (x2.37) | 775 (x1.33) | 610 (x1.68) | 696 (x1.48) | x10.0 |
| fio seq read 1MB, O_DIRECT (IOPS) | IOPS | 7,667 (x2.41) | 8,408 (x2.19) | 599 (x30.80) | 6,134 (x3.01) | **18,450** (best) | 1,943 (x9.50) | 4,075 (x4.53) | 8,027 (x2.30) | 5,273 (x3.50) | 5,759 (x3.20) | 4,028 (x4.58) | x30.8 |
| fio seq read 1MB, O_DIRECT (MB/s) | MB/s | 7,668 (x1.02) | **7,810** (best) | 601 (x12.99) | 6,135 (x1.27) | - | 1,945 (x4.02) | 4,076 (x1.92) | 7,575 (x1.03) | 4,580 (x1.71) | 5,760 (x1.36) | 4,030 (x1.94) | x13.0 |
| fio seq write 1MB, O_DIRECT (IOPS) | IOPS | 5,586 (x1.05) | 3,996 (x1.47) | 600 (x9.78) | **5,864** (best) | 3,079 (x1.90) | 2,694 (x2.18) | 2,770 (x2.12) | 4,207 (x1.39) | 3,972 (x1.48) | 2,934 (x2.00) | 2,627 (x2.23) | x9.8 |
| fio seq write 1MB, O_DIRECT (MB/s) | MB/s | 5,588 (x1.05) | 3,997 (x1.47) | 601 (x9.76) | **5,866** (best) | 3,080 (x1.90) | 2,696 (x2.18) | 2,772 (x2.12) | 4,209 (x1.39) | 3,974 (x1.48) | 2,935 (x2.00) | 2,629 (x2.23) | x9.8 |
| Hardlink throughput | bogo ops/s | 20.24 (x1.27) | **25.64** (best) | 1.42 (x18.12) | 9.59 (x2.67) | 2.86 (x8.97) | 8.06 (x3.18) | 5.25 (x4.89) | 9.27 (x2.76) | 10.84 (x2.37) | 14.40 (x1.78) | 8.29 (x3.09) | x18.1 |

### Memory bandwidth

| Metric | Unit | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel | Spread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| STREAM Add | MB/s | 100,460 (x1.79) | **179,998** (best) | 49,750 (x3.62) | 58,538 (x3.07) | 61,573 (x2.92) | 68,027 (x2.65) | 33,772 (x5.33) | 53,912 (x3.34) | 30,191 (x5.96) | 41,196 (x4.37) | 46,927 (x3.84) | x6.0 |
| STREAM Copy | MB/s | 118,960 (x1.73) | **205,776** (best) | 76,198 (x2.70) | 82,389 (x2.50) | 90,131 (x2.28) | 89,690 (x2.29) | 44,095 (x4.67) | 74,267 (x2.77) | 43,125 (x4.77) | 46,008 (x4.47) | 40,265 (x5.11) | x5.1 |
| STREAM Scale | MB/s | 92,836 (x1.84) | **170,975** (best) | 45,006 (x3.80) | 49,350 (x3.46) | 50,664 (x3.37) | 66,143 (x2.58) | 30,664 (x5.58) | 51,397 (x3.33) | 27,462 (x6.23) | 38,118 (x4.49) | 43,506 (x3.93) | x6.2 |
| STREAM Triad | MB/s | 100,750 (x1.79) | **180,452** (best) | 50,072 (x3.60) | 58,304 (x3.10) | 58,958 (x3.06) | 67,409 (x2.68) | 33,846 (x5.33) | 53,899 (x3.35) | 29,869 (x6.04) | 41,825 (x4.31) | 47,919 (x3.77) | x6.0 |

### Network

| Metric | Unit | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel | Spread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iperf3 loopback TCP, 1 stream | Mbits/sec | 90,600 (x1.68) | 73,771 (x2.06) | 61,269 (x2.48) | 73,119 (x2.08) | 15,316 (x9.92) | 61,652 (x2.46) | 72,502 (x2.09) | **151,886** (best) | 53,988 (x2.81) | 41,171 (x3.69) | 75,680 (x2.01) | x9.9 |
| iperf3 loopback TCP, 10 streams | Mbits/sec | 124,399 (x1.24) | 85,968 (x1.80) | 47,099 (x3.28) | 74,663 (x2.07) | 13,806 (x11.18) | 61,712 (x2.50) | 66,590 (x2.32) | **154,342** (best) | 53,453 (x2.89) | 37,644 (x4.10) | 71,731 (x2.15) | x11.2 |
| iperf3 loopback UDP, 10G objective | Mbits/sec | **9,999** (best) | **9,999** (best) | **9,999** (best) | **9,999** (best) | 169 (x59.17) | **9,999** (best) | **9,999** (best) | **9,999** (best) | **9,999** (best) | **9,999** (best) | **9,999** (best) | x59.2 |
| iperf3 WAN download | Mbits/sec | 2,034 (x3.51) | 4,070 (x1.75) | 3,732 (x1.91) | 4,501 (x1.59) | **7,140** (best) | 1,657 (x4.31) | 1,353 (x5.28) | 4,346 (x1.64) | 893 (x8.00) | 1,697 (x4.21) | - | x8.0 |
| iperf3 WAN upload | Mbits/sec | 2,271 (x1.83) | 3,754 (x1.11) | 3,543 (x1.17) | 1,236 (x3.37) | 462 (x9.01) | **4,159** (best) | 2,615 (x1.59) | 2,237 (x1.86) | 908 (x4.58) | 741 (x5.61) | - | x9.0 |

### System

| Metric | Unit | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel | Spread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Git common operations | s (lower) | 44.0 (x1.39) | 39.4 (x1.25) | 66.1 (x2.09) | 52.6 (x1.67) | 81.0 (x2.57) | 47.5 (x1.50) | **31.6** (best) | 45.5 (x1.44) | 53.5 (x1.69) | 82.4 (x2.61) | 61.6 (x1.95) | x2.6 |
| pgbench RO (s100, 50c) | TPS | **324,111** (best) | 283,102 (x1.14) | 219,347 (x1.48) | 228,932 (x1.42) | 11,157 (x29.05) | 194,554 (x1.67) | 253,464 (x1.28) | 257,168 (x1.26) | 146,992 (x2.20) | 98,086 (x3.30) | 175,220 (x1.85) | x29.1 |
| pgbench RO latency (s100, 50c) | ms (lower) | **0** (best) | 0 (x1.15) | 0 (x1.48) | 0 (x1.42) | 4 (x29.11) | 0 (x1.67) | 0 (x1.28) | 0 (x1.26) | 0 (x2.21) | 1 (x3.31) | 0 (x1.85) | x29.1 |
| pgbench RW (s100, 50c) | TPS | 23,988 (x1.15) | 15,739 (x1.75) | 13,890 (x1.98) | 16,880 (x1.63) | 1,939 (x14.19) | 13,351 (x2.06) | **27,502** (best) | 24,726 (x1.11) | 8,017 (x3.43) | 9,565 (x2.88) | 18,660 (x1.47) | x14.2 |
| pgbench RW latency (s100, 50c) | ms (lower) | 2 (x1.15) | 3 (x1.75) | 4 (x1.99) | 3 (x1.63) | 26 (x14.19) | 4 (x2.06) | **2** (best) | 2 (x1.11) | 6 (x3.43) | 5 (x2.88) | 3 (x1.47) | x14.2 |
| PyBench | ms (lower) | 482 (x1.32) | 440 (x1.21) | 805 (x2.21) | 500 (x1.37) | 901 (x2.47) | 665 (x1.82) | **365** (best) | 481 (x1.32) | 814 (x2.23) | 1,173 (x3.22) | 770 (x2.11) | x3.2 |
| SQLite Speedtest | s (lower) | 42.0 (x1.21) | **34.7** (best) | 72.5 (x2.09) | 52.4 (x1.51) | 382.6 (x11.03) | 63.3 (x1.82) | 48.6 (x1.40) | 44.9 (x1.30) | 66.6 (x1.92) | 96.9 (x2.79) | 67.0 (x1.93) | x11.0 |

### Real-world pipelines

| Metric | Unit | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel | Spread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Better Auth: total ( task medians) | s (lower) (derived) | 164.0 (x1.13) | 163.7 (x1.12) | 300.1 (x2.06) | 237.9 (x1.63) | 424.4 (x2.92) | 206.6 (x1.42) | **145.6** (best) | 205.5 (x1.41) | 301.0 (x2.07) | 407.2 (x2.80) | 258.0 (x1.77) | x2.9 |
| - Better Auth: git clone | s (lower) | **0.63** (best) | 1.53 (x2.42) | 1.43 (x2.28) | 1.63 (x2.59) | 2.68 (x4.26) | 0.90 (x1.44) | 0.97 (x1.54) | 1.90 (x3.02) | 1.95 (x3.09) | 2.08 (x3.30) | 0.82 (x1.31) | x4.3 |
| - Better Auth: cold install | s (lower) | **11.7** (best) | 13.9 (x1.18) | 20.2 (x1.72) | 20.6 (x1.76) | 35.9 (x3.06) | 18.8 (x1.61) | 24.3 (x2.07) | 15.9 (x1.36) | 54.9 (x4.68) | 28.0 (x2.39) | 19.5 (x1.66) | x4.7 |
| - Better Auth: lint (Biome) | s (lower) | 3.22 (x1.27) | 2.94 (x1.16) | 5.08 (x2.00) | 4.03 (x1.59) | 10.3 (x4.05) | 3.86 (x1.52) | **2.54** (best) | 3.79 (x1.49) | 4.09 (x1.61) | 6.40 (x2.52) | 4.28 (x1.69) | x4.1 |
| - Better Auth: lint deps (Knip) | s (lower) | 10.1 (x1.29) | 10.3 (x1.32) | 19.0 (x2.43) | 12.9 (x1.65) | 28.0 (x3.58) | 13.0 (x1.66) | **7.81** (best) | 12.3 (x1.58) | 16.7 (x2.14) | 22.4 (x2.86) | 14.8 (x1.89) | x3.6 |
| - Better Auth: lint format | s (lower) | 2.92 (x1.37) | 2.86 (x1.35) | 5.26 (x2.48) | 3.41 (x1.61) | 6.35 (x2.99) | 3.43 (x1.62) | **2.12** (best) | 3.14 (x1.48) | 4.36 (x2.06) | 6.69 (x3.15) | 4.39 (x2.07) | x3.1 |
| - Better Auth: lint spell | s (lower) | 6.85 (x1.32) | 7.24 (x1.40) | 13.4 (x2.59) | 10.00 (x1.93) | 14.7 (x2.82) | 8.78 (x1.69) | **5.19** (best) | 7.93 (x1.53) | 10.9 (x2.09) | 16.7 (x3.22) | 11.1 (x2.13) | x3.2 |
| - Better Auth: lint types | s (lower) | 26.7 (x1.10) | 24.9 (x1.02) | 52.3 (x2.15) | 40.7 (x1.67) | 101.6 (x4.17) | 34.1 (x1.40) | **24.3** (best) | 36.0 (x1.48) | 48.5 (x1.99) | 76.8 (x3.15) | 44.3 (x1.82) | x4.2 |
| - Better Auth: lint packages | s (lower) | 2.43 (x1.18) | 2.46 (x1.19) | 4.27 (x2.07) | 3.36 (x1.63) | 9.41 (x4.57) | 3.21 (x1.56) | **2.06** (best) | 3.31 (x1.61) | 3.41 (x1.66) | 6.52 (x3.17) | 3.69 (x1.79) | x4.6 |
| - Better Auth: typecheck | s (lower) | 41.5 (x1.38) | 39.3 (x1.31) | 76.7 (x2.56) | 60.4 (x2.01) | 79.2 (x2.64) | 50.1 (x1.67) | **30.0** (best) | 46.1 (x1.54) | 65.5 (x2.18) | 100.2 (x3.34) | 65.4 (x2.18) | x3.3 |
| - Better Auth: build | s (lower) | 57.9 (x1.25) | 58.2 (x1.26) | 102.5 (x2.21) | 80.9 (x1.75) | 136.4 (x2.95) | 70.3 (x1.52) | **46.3** (best) | 75.0 (x1.62) | 90.7 (x1.96) | 141.5 (x3.06) | 89.7 (x1.94) | x3.1 |
| Mastra: total ( task medians) | s (lower) (derived) | 200.6 (x1.20) | 205.3 (x1.23) | 367.3 (x2.20) | 273.0 (x1.64) | 464.4 (x2.79) | 260.5 (x1.56) | **166.7** (best) | 236.9 (x1.42) | 342.8 (x2.06) | 457.1 (x2.74) | 416.6 (x2.50) | x2.8 |
| - Mastra: git clone | s (lower) | **2.08** (best) | 2.39 (x1.15) | 3.35 (x1.61) | 3.08 (x1.48) | 6.49 (x3.11) | 2.30 (x1.10) | 3.74 (x1.80) | 4.06 (x1.95) | 3.30 (x1.58) | 5.43 (x2.60) | 2.89 (x1.39) | x3.1 |
| - Mastra: cold install | s (lower) | **37.8** (best) | 39.0 (x1.03) | 69.8 (x1.84) | 58.6 (x1.55) | 97.2 (x2.57) | 51.6 (x1.36) | 40.0 (x1.06) | 46.8 (x1.24) | 67.2 (x1.78) | 92.4 (x2.44) | 73.9 (x1.95) | x2.6 |
| - Mastra: lint:format | s (lower) | 88.9 (x1.32) | 93.2 (x1.39) | 165.8 (x2.47) | 116.4 (x1.74) | 189.2 (x2.82) | 115.0 (x1.71) | **67.1** (best) | 101.8 (x1.52) | 150.9 (x2.25) | 199.8 (x2.98) | 189.3 (x2.82) | x3.0 |
| - Mastra: build:core | s (lower) | 71.8 (x1.29) | 70.7 (x1.27) | 128.4 (x2.30) | 94.9 (x1.70) | 171.5 (x3.08) | 91.5 (x1.64) | **55.8** (best) | 84.2 (x1.51) | 121.4 (x2.18) | 159.4 (x2.86) | 150.6 (x2.70) | x3.1 |
| OpenClaw: total ( task medians) | s (lower) (derived) | - | 198.5 (x1.19) | 356.5 (x2.14) | 268.0 (x1.61) | 605.5 (x3.64) | 236.4 (x1.42) | **166.5** (best) | 243.2 (x1.46) | 317.3 (x1.91) | 410.0 (x2.46) | 335.3 (x2.01) | x3.6 |
| - OpenClaw: git clone | s (lower) | **2.37** (best) | 3.55 (x1.50) | 4.48 (x1.89) | 4.18 (x1.76) | 10.3 (x4.35) | 3.18 (x1.34) | 6.21 (x2.62) | 4.18 (x1.76) | 4.28 (x1.80) | 6.78 (x2.86) | 4.11 (x1.73) | x4.3 |
| - OpenClaw: cold install | s (lower) | **11.3** (best) | 15.7 (x1.39) | 19.9 (x1.77) | 20.3 (x1.80) | 31.9 (x2.83) | 18.2 (x1.62) | 14.7 (x1.30) | 19.0 (x1.69) | 21.3 (x1.89) | 24.3 (x2.16) | 19.7 (x1.75) | x2.8 |
| - OpenClaw: lint (extension channels) | s (lower) | 61.4 (x1.19) | 61.7 (x1.20) | 110.3 (x2.14) | 89.5 (x1.74) | 177.8 (x3.45) | 73.9 (x1.43) | **51.6** (best) | 69.3 (x1.34) | 101.7 (x1.97) | 133.7 (x2.59) | 111.0 (x2.15) | x3.4 |
| - OpenClaw: typecheck (tsgo) | s (lower) | 17.3 (x1.26) | 17.3 (x1.26) | 37.4 (x2.72) | 23.0 (x1.67) | 70.3 (x5.12) | 20.8 (x1.52) | **13.7** (best) | 24.3 (x1.77) | 27.1 (x1.97) | 36.4 (x2.65) | 30.1 (x2.19) | x5.1 |
| - OpenClaw: typecheck (test tree) | s (lower) | - | 100.3 (x1.25) | 184.4 (x2.29) | 131.0 (x1.63) | 315.2 (x3.92) | 120.3 (x1.50) | **80.4** (best) | 126.4 (x1.57) | 163.0 (x2.03) | 208.8 (x2.60) | 170.5 (x2.12) | x3.9 |

## Observed environments

| Spec | Blaxel | Daytona (VM) | E2B | microsandbox | Modal (gVisor) | Modal (VM) | Namespace | Novita | run.cloud | Runloop | Vercel |
|---|---|---|---|---|---|---|---|---|---|---|---|
| vCPUs | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| CPU model | AMD EPYC | AMD EPYC | Intel(R) Xeon(R) Processor @ 2.60GHz | Intel(R) Xeon(R) Processor | unknown | 8 models across replicates | AMD EPYC | AMD EPYC | AMD EPYC | 4 models across replicates | 2 models across replicates |
| Isolation | vm | vm | vm | vm | gvisor | vm | vm | vm | vm | vm | vm |
| Virtualization | firecracker | firecracker | firecracker | libkrun | not-observable | cloud-hypervisor | firecracker | firecracker | firecracker | cloud-hypervisor | firecracker |
| Memory (GB) | 7.78 | 7.78 | 7.77 | 7.78 | 8 | 7.77 | 7.82 | 7.78 | 7.78 | 7.78 | 8.21 |
| Disk (GB) | 39.9 | 39.1 | 104.2 | 39.3 | - | 511 | 63 | 103.6 | 39.3 | 63 | 63.9 |
| File system | xfs | ext4 | ext4 | overlayfs | overlayfs | ext4 | overlayfs | ext4 | ext4 | ext4 | xfs |
| Mount options | allocsize=64k,attr2,inode64,lazytime,logbsize=256k,logbufs=8,noatime,nodiratime,noquota,rw | relatime,rw | discard,relatime,rw | - | - | discard,noatime,prjquota,rw | - | relatime,rw | relatime,rw | noinit_itable,relatime,rw | attr2,inode64,logbsize=32k,logbufs=8,noquota,nouuid,relatime,rw |
| I/O scheduler | MQ-DEADLINE | - | - | - | - | - | - | - | - | NONE | MQ-DEADLINE |
| Block size | 4096 | 4096 | 4096 | - | - | 4096 | - | 4096 | 4096 | 4096 | 4096 |
| Kernel | 6.1.166 | 6.19.14 | 6.1.158+ | 6.12.98 | 4.19.0-gvisor | 6.12.8+ | 7.0.1 | 6.1.158+ | 6.18.36+ | 6.18.32 | 6.12.76 |
| OS | Debian GNU/Linux 12 (bookworm) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) | Debian GNU/Linux 13 (trixie) |
| Egress family | IPv6 | IPv4 | IPv4 | IPv4 | IPv4 | IPv4 | IPv4 | IPv4 | IPv4 | IPv4 | IPv4 |
| Egress ASN | AS396356 | AS396356 | AS396982 | AS396982 | AS8075 | AS31898 | AS401483 | AS31898 | AS24940 | AS16509 | AS16509 |
| ASN name | Latitude.sh | Latitude.sh | Google LLC | Google LLC | Microsoft Corporation | Oracle Corporation | Namespace Labs | Oracle Corporation | Hetzner Online GmbH | Amazon.com, Inc. | Amazon.com, Inc. |
| Geo location | Ashburn, Virginia, US | Los Angeles, California, US | The Dalles, Oregon, US | Dallas, Texas, US | San Antonio, Texas, US | Ashburn, Virginia, US | Leesburg, Virginia, US | Phoenix, Arizona, US | Helsinki, Uusimaa, FI | Columbus, Ohio, US | - |
| Spec vs target | matched | matched | matched | matched | matched | matched | matched | matched | matched | matched | matched |

Heterogeneous fleets: Modal (VM) saw 8 host CPU models across its replicate sandboxes (AMD EPYC 9455 48-Core Processor; AMD EPYC 9455P 48-Core Processor; AMD EPYC 9554 64-Core Processor; AMD EPYC 9554P 64-Core Processor; AMD EPYC 9654 96-Core Processor; AMD EPYC 9J45 128-Core Processor; Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz; Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz) - Runloop saw 4 host CPU models across its replicate sandboxes (Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz; Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz; Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz; Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz) - Vercel saw 2 host CPU models across its replicate sandboxes (Intel(R) Xeon(R) Processor @ 2.50GHz; Intel(R) Xeon(R) Processor @ 2.90GHz). Every median for each of these environments pools across those machines, so read their cells as the fleet's typical draw, not one machine's speed.

## Data quality

Only providers marked validated by the dataset render as comparison columns. Every skipped or failed suite record below comes from this run, including universally unrecorded tasks and provider-specific failures; no older run fills a missing cell. Providers attempted without any validated metrics are disclosed after the table instead of becoming empty comparison columns.

| Provider | Suite | Outcome | Reason |
|---|---|---|---|
| Blaxel | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Blaxel | realworld-openclaw | failed | PTS ran but every trial failed for 4 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_types (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| Daytona (VM) | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Daytona (VM) | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| E2B | pgbench | failed | Step "clone repo" failed with exit code 128 |
| E2B | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| E2B | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| microsandbox | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| microsandbox | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| Modal (gVisor) | disk | failed | PTS duplicate-value dedup dropped 1 fio twin result (MB/s == IOPS at this block size, so the duplicate-valued <Result> was never written): fio_type_sequential_read_engine_linux_aio_direct_yes_block_size_1mb_job_count_1_disk_target_default_test_directory_mb_per_s (twin survived in disk/pts_fio-seq-read.xml) |
| Modal (gVisor) | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Modal (gVisor) | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| Modal (gVisor) | realworld-openclaw | failed | Step "mise run benchmark:realworld:pts:openclaw" lost its sandbox: 12 consecutive detached polls failed (last: done-file fs exists) - the sandbox stopped responding, not a quiet long step |
| Modal (VM) | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Modal (VM) | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| Namespace | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Namespace | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| Novita | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Novita | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| run.cloud | disk | failed | PTS duplicate-value dedup dropped 1 fio twin result (MB/s == IOPS at this block size, so the duplicate-valued <Result> was never written): fio_type_sequential_read_engine_linux_aio_direct_yes_block_size_1mb_job_count_1_disk_target_default_test_directory_mb_per_s (twin survived in disk/pts_fio-seq-read.xml) |
| run.cloud | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| run.cloud | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| run.cloud | realworld-openclaw | failed | Step "mise run benchmark:realworld:pts:openclaw" lost its sandbox: 12 consecutive detached polls failed (last: done-file cat poll) - the sandbox stopped responding, not a quiet long step |
| Runloop | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Runloop | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |
| Vercel | network | failed | pts_iperf-wan-download: pts_iperf-wan-download did not produce 1 numeric metric value(s) |
| Vercel | network | failed | pts_iperf-wan-upload: PTS batch-run of local/iperf-wan-1.0.0 completed but every trial errored (composite carries no values) |
| Vercel | network | failed | Step "mise run benchmark:network:suite" failed with exit code 1 |
| Vercel | realworld-mastra | failed | PTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) - attempted, no value recorded |
| Vercel | realworld-openclaw | failed | PTS ran but every trial failed for 3 of 8 declared metrics: realworld_openclaw_task_lint_oxlint (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_shrinkwrap_check (realworld-openclaw/pts_realworld-openclaw.xml), realworld_openclaw_task_test_unit_fast (realworld-openclaw/pts_realworld-openclaw.xml) - attempted, no value recorded |

Also in this run, not in the tables above: Daytona (container) - pending, 0 metrics reported; microsandbox (local) - pending, 0 metrics reported. An environment that reported nothing gets no column.