We raised $3M for agent-native runners
Pricing
Products
Skills
Resources
Get Started
Technical report

High performance sandbox benchmarks

Compare 13 sandbox providers on one target hardware shape, across real developer, CI/CD, and reinforcement-learning environment workloads.

View in Markdown
starslingdev/hpc-sandbox-benchmarksOpen sourceThe harness that produced every number on this page, with the raw run documents it recomputes from. Clone it and run the suites yourself.Read the write-up/Beyond cold starts: benchmarking sandboxes for real workloads
Run ID
36498291449
Commit
d02dfe89375e
Run date
2026-09-29
Target spec
4 vCPU · 8 GB · 40 GB
Replicates
n = 2 to 12 sandboxes / case
Providers
13 validated

We measured the end-to-end time developers and agents experience across environments from 13 sandbox providers, including Firecracker- and libkrun-based stacks, as they complete software engineering tasks. Each repository run starts with a clone and cold dependency install, then continues through that repository's own lint, typecheck, build, and test tasks.

  • Real-workload suites: 3 repositories, each run cold on 12 replicate sandboxes through that repository's own CI tasks: clone, install dependencies, lint, typecheck, build, test. The repositories are Better Auth, Mastra, and OpenClaw.
  • Synthetic suites: 10 versioned Phoronix Test Suite profiles. Of those, 8 are industry-standard profiles published through OpenBenchmarking.org. Each suite runs on 3 replicate sandboxes (5 for CPU) and repeats until Phoronix's own convergence criterion is met. Methodology lists what every suite measures.

Explore any metric · scan the full table.

Results explorer

Choose a statistic (min, p50, p95, or max), then a real-world pipeline or synthetic benchmark. Bars rank providers best-first against the leader. Hover or focus a row for instance metadata and the raw sample distribution; select it to jump to that metric in the table below.

starsling.dev/hpc-sandbox-benchmarks

High performance sandbox benchmarks

Real-world pipelines

s · lower is better · p50

Every environment · target 4 vCPU · 8 GB · 40 GB

Real world · Better Auth · Cold · total (Σ task medians) · Metric p50

light to dark = execution order

Metric
Real
Synthetic
Pick a synthetic benchmark family to chart a metric

What these sandbox benchmarks answer

Findings from run 36498291449, tested 2026-09-29, targeting 4 vCPU · 8 GB · 40 GB.

Key findings

Firecracker vs libkrun

On Node.js Web Tooling, the highest measured Firecracker result is Daytona (VM): 21.52 runs/s (p50, 5 sandboxes); the highest measured libkrun result is microsandbox: 21.14 runs/s (p50, 5 sandboxes). Hardware, storage, and region vary between these environments. This measures the deployed stacks, not the performance cost of either runtime in isolation.

Cold start vs workload performance

Namespace has the shortest Better Auth cold dependency install: 10.9 s at p50. boat has the shortest full workflow: 143.8 s, summing task medians. The install leader is not the workflow leader. Cold install starts inside a running sandbox; sandbox creation-to-ready time was not measured.

CPU, RAM, disk, and network tradeoffs

boat leads Node.js Web Tooling at 26.69 runs/s; tama leads STREAM Triad at 224,204 MB/s. A CPU result does not predict memory bandwidth. Use the disk and WAN rows for their specific I/O patterns; neither a 4 KB random-write test nor transfer throughput is a general application-speed score.

Which dimensions matter for agents and CI?

For a fresh agent sandbox, compare setup tasks such as clone and dependency install. For a complete CI run, compare the repository pipeline closest to yours in the Results explorer. Synthetic CPU, memory, disk, and network tests are separate measurements; this run does not establish which resource caused a task to take longer.

Not measured in this run: sandbox creation-to-ready time, runtime RAM overhead, or network round-trip latency. STREAM measures memory bandwidth; iperf3 measures throughput. These results cannot establish which runtime boots fastest or uses the least RAM.

Measured summary

Highest throughput or shortest duration among measured environments, selected separately for each row. This is not an overall provider ranking. Non-total measurements and pipeline task values are p50; workflow totals sum per-task p50 medians. Run 36498291449, tested 2026-09-29; target 4 vCPU · 8 GB · 40 GB. Raw run and harness source
Dimension and measurementStrongest observed result
CPUNode.js web tooling (runs/s)Phoronix Node.js Web Tooling; replicate sandboxesMetric rowMethod26.69 runs/sboat5 replicate sandboxes
Memory bandwidthSTREAM Triad (MB/s)STREAM Triad; replicate sandboxesMetric rowMethod224,204 MB/stama3 replicate sandboxes
Disk I/Ofio rand write 4KB, O_DIRECT (MB/s)fio random write, 4 KB blocks, O_DIRECT; replicate sandboxesMetric rowMethod1,487 MB/sboat3 replicate sandboxes
Network throughputiperf3 WAN download (Mbits/sec)iperf3 WAN download to the nearest curated public server; replicate sandboxesMetric rowMethod8,465 Mbits/secVercel3 replicate sandboxes
Network throughputiperf3 WAN upload (Mbits/sec)iperf3 WAN upload to the nearest curated public server; replicate sandboxesMetric rowMethod5,594 Mbits/secModal (VM)3 replicate sandboxes
Filesystem behaviorHardlink throughput (bogo ops/s)Hardlink throughput; replicate sandboxesMetric rowMethod24.68 bogo ops/sDaytona (VM)3 replicate sandboxes
Cold workload taskBetter Auth: cold install (s)Better Auth cold dependency install; fresh pipeline sandboxesMetric rowMethod10.9 sNamespace12 replicate sandboxes
End-to-end workflowBetter Auth: total (s)Better Auth cold pipeline; total sums per-task mediansMetric rowMethod143.8 sboat12 replicate sandboxes

Precision

Each row shows the single metric with that provider's widest total (min-max) spread, ranked by IDR (p10-p90). The distribution is every trial the run recorded for that metric, pooled across the provider's replicate sandboxes. Phoronix repeats a benchmark several times inside each sandbox, and all of those measurements count, so a row's n is trials rather than sandboxes. Convergence-mode suites repeat until Phoronix settles, which means a noisy sandbox contributes more trials than a quiet one and weighs on the interval accordingly. Metrics that depend on the public Internet, including WAN iperf, repository cloning, and dependency installation, are excluded so the figure reflects sandbox precision; local networking such as iperf loopback remains eligible. The suite and metric vary by provider and are named on every row. Because the units are unlike, every interval is given as a percentage of its own metric's p50. Dots are individual trials and the vertical line marks p50; select Total, IQR, or IDR to highlight and read that range, or hover a row for its numerical endpoints.

Worst precision item per provider

Ranked by IDR (p10-p90) · lower is more precise

Range
Provider · worst itemIDR spread
  1. 1

    E2B

    Real world · Better Auth · lint format

    p50 5.12 s · n=12 trials · 12 sandboxes

    IDR spread36.5%
  2. 2

    boat

    Real world · Better Auth · lint spell

    p50 5.46 s · n=12 trials · 12 sandboxes

    IDR spread41.5%
  3. 3

    Blaxel

    Synthetic · iperf3 · loopback TCP, 1 stream

    p50 119,105 Mbits/sec · n=6 trials · 3 sandboxes

    IDR spread41.7%
  4. 4

    Runloop

    Real world · OpenClaw · typecheck (tsgo)

    p50 46.7 s · n=12 trials · 12 sandboxes

    IDR spread46.9%
  5. 5

    Vercel

    Real world · Better Auth · build

    p50 113.4 s · n=12 trials · 12 sandboxes

    IDR spread59.8%
  6. 6

    Daytona (VM)

    Synthetic · iperf3 · loopback TCP, 10 streams

    p50 77,472 Mbits/sec · n=6 trials · 3 sandboxes

    IDR spread61.5%
  7. 7

    Novita

    Synthetic · STREAM · Copy

    p50 58,485 MB/s · n=6 trials · 3 sandboxes

    IDR spread81.4%
  8. 8

    microsandbox

    Synthetic · fio · rand read 4KB, O_DIRECT (IOPS)

    p50 177,000 IOPS · n=6 trials · 3 sandboxes

    IDR spread94.7%
  9. 9

    run.cloud

    Synthetic · fio · seq write 1MB, O_DIRECT (IOPS)

    p50 3,515 IOPS · n=6 trials · 3 sandboxes

    IDR spread95.2%
  10. 10

    Namespace

    Synthetic · SQLite Speedtest · execution time

    p50 36.1 s · n=6 trials · 3 sandboxes

    IDR spread150%
  11. 11

    tama

    Synthetic · SQLite Speedtest · execution time

    p50 58.9 s · n=6 trials · 3 sandboxes

    IDR spread214%
  12. 12

    Modal (VM)

    Synthetic · iperf3 · loopback TCP, 1 stream

    p50 23,932 Mbits/sec · n=6 trials · 3 sandboxes

    IDR spread252%
  13. 13

    Modal (gVisor)

    Synthetic · pgbench · RO latency (s100, 50c)

    p50 1 ms · n=6 trials · 3 sandboxes

    IDR spread536%
trialp50selected p10-p90total MIN-MAX

Methodology

Every environment ran the same harness against freshly created sandboxes targeting 4 vCPU · 8 GB RAM · 40 GB disk. Synthetic suites ran on 3 replicate sandboxes (5 for CPU), each repeating passes until Phoronix's own convergence criterion was met, so pass counts vary by how noisy the sandbox was; each real repository pipeline (Better Auth, Mastra, and OpenClaw) ran cold on 12 fresh sandboxes, through that repository's own CI tasks. Every figure is a median (p50) pooled across an environment's sandboxes, and every cell keeps the individual trials behind that median, which is what the Precision section reads.

The 10 synthetic suites are described by their own Phoronix test profiles: 8 published on OpenBenchmarking.org, 2 written for this harness and therefore without a profile page to cite.

NameTest typeDescriptionProject site
Flexible IO Testerv2.1.0DiskFIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers.fio.readthedocs.io
Hardlink Throughputharness-localDiskMeasures hardlink() syscall throughput using the stress-ng --link stressor. Mirrors the per-syscall pattern that pnpm install hits when materialising thousands of node_modules entries from a content-addressable store. Reports bogo ops/sec (real time) — one bogo op = one create+unlink hardlink cycle.github.com/ColinIanKing
Streamv1.3.4MemoryThis is a benchmark of Stream, the popular system memory (RAM) benchmark.cs.virginia.edu
iPerfv1.2.0NetworkiPerf is a network bandwidth throughput testing software. This test profile is used for automated testing of an iperf client and requires you have access to an iperf server.software.es.net
iPerf WANharness-localNetworkThis test measures sustained WAN TCP throughput with iperf3 against an RTT-ranked reachable server from a curated public list, chosen per run by RTT probe. It uses eight parallel streams so that high bandwidth-delay-product paths are not understated, and reports receiver-side goodput.software.es.net
Node.js V8 Web Tooling Benchmarkv1.0.1ProcessorRunning the V8 project's Web-Tooling-Benchmark under Node.js. The Web-Tooling-Benchmark stresses JavaScript-related workloads common to web developers like Babel and TypeScript and Babylon. This test profile can test the system's JavaScript performance with Node.js.v8.github.io
Gitv1.1.0SystemThis test measures the time needed to carry out some sample Git operations on an example, static repository that happens to be a copy of the GNOME GTK tool-kit repository.git-scm.com
PostgreSQLv1.15.0SystemThis is a benchmark of PostgreSQL using the integrated pgbench for facilitating the database benchmarks.postgresql.org
PyBenchv1.1.3SystemThis test profile reports the total time of the different average timed test results from PyBench. PyBench reports average test times for different functions such as BuiltinFunctionCalls and NestedForLoops, with this total result providing a rough estimate as to Python's average performance on a given system. This test profile runs PyBench each time for 20 rounds.svn.python.org
SQLite Speedtestv1.0.1SystemThis is a benchmark of SQLite's speedtest1 benchmark program with an increased problem size of 1,000.sqlite.org
Pipeline totals
The p50 is the sum of per-task medians, so it exactly matches the task numbers reported beside it. In the Results explorer, min, p95, and max come from end-to-end totals aligned within the same replicate sandbox and trial; task percentiles are never added together.
A column is a variant, not a vendor
Modal (gVisor) and Modal (VM) are one account's two runtimes booting the same image. Variants attempted without validated metrics are disclosed under Data quality rather than rendered as empty columns.
A column is a fleet, not a machine
Each cell pools an environment's replicate sandboxes, and the environments table flags every provider whose sandboxes reported more than one host CPU model.
n counts trials, r counts sandboxes
A convergence-mode cell's n is pooled passes, not sandboxes. A sandbox that converged slowly contributes more passes to the same median. Where a sandbox or a whole suite was lost, the surviving replicates still carry the cell, which is why the replicate count in the summary above is a range (2 to 12) rather than one number; Data quality records every such loss.
Network figures isolate the sandbox
iperf3 loopback, using TCP at 1 and 10 streams plus a 10 G-objective UDP datagram test, measures the sandbox's own network stack with no Internet path. The WAN pair measures both directions against the nearest curated public iperf3 server, chosen per run by RTT probe.
Task names are each repository's own
Labels come from the dataset's catalog, which uses the repo's names: Better Auth's lint types (its repo-wide type-lint pass) is a different task from its root typecheck.

All metrics

Every catalogued metric this run emitted, labeled and grouped exactly as the dataset's own leaderboard labels it. Each cell shows the median and its ratio to the row's best; amber deepens the further a provider sits behind, teal marks the best. ↓ means lower is better; ∑ marks a value derived from other rows. Click a metric name to open it in the results explorer.

MetricBlaxelboatDaytona (VM)E2BmicrosandboxModal (gVisor)Modal (VM)NamespaceNovitarun.cloudRunlooptama †VercelSpread
CPU
runs/s ↑headline19.48×1.3726.69best21.52×1.2412.61×2.1221.14×1.2612.79×2.0915.54×1.7221.79×1.2319.52×1.3714.73×1.818.29×3.2216.77×1.5912.96×2.06×3.2
Disk I/O
IOPS ↑220,500×2.07456,500best243,500×1.8758,700×7.78177,000×2.58222,000×2.06227,500×2.01102,950×4.4379,700×5.73170,000×2.69147,000×3.1193,000×4.91248,000×1.84×7.8
MB/s ↑904×2.071,870best996×1.88241×7.77725×2.58908×2.06932×2.01422×4.43327×5.73697×2.68601×3.11381×4.911,015×1.84×7.8
IOPS ↑205,000×1.77363,000best229,000×1.5958,650×6.19126,500×2.87214,000×1.70151,500×2.40105,350×3.4580,000×4.54189,000×1.92136,500×2.6663,900×5.68281,000×1.29×6.2
MB/s ↑headline839×1.771,487best937×1.59240×6.19518×2.87876×1.70621×2.40431×3.45328×4.54773×1.92558×2.66262×5.671,150×1.29×6.2
IOPS ↑12,900×1.8018,500×1.258,889×2.61599×38.7321,900×1.0623,200best1,047×22.162,290×10.1312,150×1.917,511×3.095,483×4.23642×36.145,002×4.64×38.7
MB/s ↑13,529×1.8019,435×1.259,322×2.61630×38.5922,924×1.0624,320best1,099×22.122,403×10.1212,778×1.907,876×3.095,750×4.23674×36.075,247×4.64×38.6
IOPS ↑6,458×2.552,693×6.133,885×4.25599×27.5516,500best5,311×3.11838×19.691,718×9.606,502×2.543,515×4.692,951×5.591,260×13.104,365×3.78×27.5
MB/s ↑6,773×2.552,824×6.124,075×4.24630×27.4317,287best5,571×3.10880×19.641,804×9.596,820×2.533,687×4.693,096×5.581,322×13.074,578×3.78×27.4
bogo ops/s ↑20.07×1.2311.86×2.0824.68best1.71×14.398.48×2.914.94×4.998.07×3.0615.63×1.5812.15×2.037.63×3.2414.23×1.737.46×3.3110.72×2.30×14.4
Memory bandwidth
MB/s ↑97,174×2.2332,816×6.59176,029×1.2346,807×4.6257,023×3.7948,368×4.47106,420×2.0333,021×6.5554,015×4.0044,901×4.8238,452×5.62216,260best45,084×4.80×6.6
MB/s ↑102,256×1.9543,195×4.63199,868best74,177×2.6987,749×2.2875,189×2.66107,489×1.8643,468×4.6058,485×3.4261,537×3.2541,401×4.83190,257×1.0539,807×5.02×5.0
MB/s ↑85,908×2.3529,851×6.76166,697×1.2143,995×4.5852,732×3.8342,033×4.80102,657×1.9630,156×6.6951,762×3.9040,514×4.9836,572×5.52201,715best41,211×4.89×6.8
MB/s ↑headline97,795×2.2932,865×6.82176,705×1.2746,622×4.8157,075×3.9348,220×4.65107,552×2.0833,120×6.7754,229×4.1344,961×4.9939,140×5.73224,204best45,668×4.91×6.8
Network
Mbits/sec ↑119,105×1.3194,557×1.6580,132×1.9552,200×3.0084,649×1.8532,296×4.8523,932×6.5435,888×4.36156,485best35,792×4.3739,610×3.9562,953×2.4963,137×2.48×6.5
Mbits/sec ↑148,679best86,842×1.7177,472×1.9252,966×2.81100,335×1.4827,477×5.4124,839×5.9939,911×3.73147,313×1.0141,372×3.5937,443×3.9742,962×3.4648,271×3.08×6.0
Mbits/sec ↑9,999best9,999best9,999best9,999best9,999best501×19.969,999best9,999best9,999best9,999best9,999best9,999best9,999best×20.0
Mbits/sec ↑headline1,361×6.22893×9.486,717×1.261,206×7.025,117×1.651,994×4.251,464×5.78876×9.675,834×1.45935×9.051,686×5.024,552×1.868,465best×9.7
Mbits/sec ↑headline4,385×1.28790×7.084,711×1.191,227×4.562,027×2.761,265×4.425,594best3,818×1.473,431×1.63934×5.992,035×2.751,518×3.685,515×1.01×7.1
System
s ↓headline41.5×1.1934.8best36.2×1.0465.7×1.8940.1×1.1581.6×2.3541.9×1.2137.6×1.0844.7×1.2940.2×1.1696.0×2.7651.0×1.4763.0×1.81×2.8
TPS ↑337,182×1.34384,743×1.17281,830×1.60226,079×1.99244,429×1.8478,759×5.72196,681×2.29218,996×2.06296,502×1.52184,083×2.45105,029×4.29450,356best128,004×3.52×5.7
ms ↓0×1.330×1.170×1.590×1.980×1.831×5.700×2.280×2.050×1.510×2.430×4.270best0×3.50×5.7
TPS ↑22,961×1.1824,106×1.1215,289×1.7713,085×2.0616,012×1.698,344×3.2415,492×1.7417,753×1.5226,994best15,633×1.738,405×3.2114,140×1.9112,792×2.11×3.2
ms ↓2×1.172×1.123×1.764×2.063×1.696×3.233×1.743×1.532best3×1.736×3.214×1.914×2.11×3.2
ms ↓456×1.23418×1.13408×1.10635×1.72452×1.22901×2.44477×1.29370best480×1.30493×1.331,201×3.25533×1.44770×2.08×3.2
s ↓38.2×1.2348.7×1.5731.0best71.4×2.3147.5×1.53450.0×14.5335.9×1.1636.1×1.1740.8×1.3274.3×2.4092.6×2.9958.9×1.9067.1×2.17×14.5
Real-world pipelines
s ↓ ∑191.5×1.33143.8best155.7×1.08296.0×2.06202.0×1.40424.4×2.95212.7×1.48160.9×1.12187.7×1.31273.5×1.90562.6×3.91195.3×1.36312.1×2.17×3.9
s ↓0.74×1.131.71×2.611.61×2.461.65×2.521.13×1.731.81×2.771.12×1.720.65best2.01×3.083.61×5.522.72×4.161.43×2.191.02×1.57×5.5
s ↓12.5×1.1413.5×1.2312.8×1.1720.3×1.8614.6×1.3434.7×3.1819.8×1.8110.9best15.1×1.3822.5×2.0629.9×2.7418.2×1.6723.6×2.16×3.2
s ↓3.40×1.432.39best2.86×1.204.95×2.083.39×1.429.88×4.143.83×1.602.72×1.143.22×1.354.37×1.837.78×3.264.62×1.945.12×2.15×4.1
s ↓10.8×1.427.66best9.47×1.2418.5×2.4211.6×1.5224.3×3.1813.0×1.7010.2×1.3311.2×1.4614.5×1.8924.9×3.2515.9×2.0717.5×2.29×3.3
s ↓3.15×1.422.22best2.57×1.165.12×2.303.10×1.406.29×2.833.56×1.602.34×1.052.99×1.343.79×1.717.17×3.234.32×1.955.32×2.39×3.2
s ↓7.81×1.435.46best6.49×1.1913.4×2.467.95×1.4616.0×2.948.90×1.635.96×1.097.71×1.4110.6×1.9420.6×3.7710.6×1.9413.9×2.54×3.8
s ↓32.8×1.5624.5×1.1624.5×1.1750.9×2.4238.0×1.80107.3×5.1034.8×1.6529.4×1.4031.5×1.5053.8×2.56111.7×5.3021.1best53.8×2.56×5.3
s ↓2.79×1.222.50×1.102.41×1.064.34×1.903.00×1.318.30×3.643.25×1.432.28best2.65×1.163.95×1.749.00×3.952.50×1.103.95×1.73×3.9
s ↓47.8×1.4533.0best39.6×1.2074.9×2.2750.5×1.5380.1×2.4351.1×1.5539.4×1.2044.9×1.3665.9×2.00163.1×4.9551.5×1.5674.5×2.26×4.9
s ↓69.7×1.3750.9best53.3×1.05101.8×2.0068.8×1.35135.5×2.6673.3×1.4457.0×1.1266.4×1.3090.5×1.78185.7×3.6565.3×1.28113.4×2.23×3.6
s ↓ ∑1168.4×1.161007.9best1102.7×1.091849.5×1.841205.0×1.201769.6×1.761380.2×1.371139.0×1.131228.8×1.221688.8×1.682561.6×2.541432.6×1.421719.2×1.71×2.5
s ↓1.72×1.022.95×1.753.36×1.993.79×2.252.71×1.614.14×2.462.39×1.421.69best3.48×2.063.21×1.915.72×3.393.54×2.103.00×1.78×3.4
s ↓headline38.2best43.6×1.1438.3×1.0074.3×1.9548.7×1.2870.7×1.8551.9×1.3640.2×1.0544.5×1.17127.1×3.33100.0×2.6296.9×2.5465.6×1.72×3.3
s ↓96.7×1.3870.2best84.2×1.20167.6×2.39116.5×1.66149.0×2.12114.4×1.6379.6×1.1396.2×1.37138.3×1.97217.6×3.10114.4×1.63165.8×2.36×3.1
s ↓75.7×1.2660.3best68.8×1.14132.6×2.2084.6×1.40127.0×2.1193.0×1.5466.9×1.1177.3×1.28108.8×1.80276.9×4.5999.5×1.65125.8×2.09×4.6
s ↓956.0×1.15830.9best908.1×1.091471.2×1.77952.5×1.151418.7×1.711118.5×1.35950.6×1.141007.2×1.211311.4×1.581961.4×2.361118.2×1.351359.0×1.64×2.4
s ↓ ∑617.3×1.38445.7best557.7×1.251124.7×2.52628.4×1.41878.9×1.97649.8×1.46639.8×1.44671.6×1.51806.4×1.811561.1×3.50–905.4×2.03×3.5
s ↓2.70×1.055.98×2.334.43×1.734.90×1.913.72×1.455.12×1.994.52×1.762.56best4.32×1.6910.7×4.187.17×2.80–4.29×1.67×4.2
s ↓12.1×1.0618.7×1.6411.8×1.0320.2×1.7717.5×1.5324.5×2.1414.8×1.2911.4best14.6×1.2720.4×1.7824.8×2.17–20.7×1.81×2.2
s ↓314.6×1.50210.2best288.8×1.37586.9×2.79314.9×1.50432.3×2.06331.5×1.58327.0×1.56344.0×1.64405.6×1.93914.7×4.35–435.2×2.07×4.4
s ↓163.6×1.39117.7best145.7×1.24304.4×2.59164.2×1.40242.4×2.06172.0×1.46164.8×1.40175.3×1.49215.6×1.83343.9×2.92–255.2×2.17×2.9
s ↓17.5×1.3013.5best15.9×1.1834.1×2.5318.8×1.4025.6×1.9018.1×1.3419.4×1.4421.2×1.5823.6×1.7546.7×3.47–29.5×2.19×3.5
s ↓106.9×1.3479.7best91.1×1.14174.2×2.19109.1×1.37148.9×1.87108.9×1.37114.5×1.44112.2×1.41130.5×1.64223.8×2.81–160.6×2.02×2.8

Observed environments

Specs, storage shape, hardware mixtures, and egress identity come directly from the run's embedded in-sandbox probes and Phoronix metadata. When replicate sandboxes reported different host CPU models, the CPU row shows the fleet count and the disclosure below lists every model. Environment flags mark an off-target resource, gVisor isolation, or a heterogeneous fleet; missing probe values remain blank rather than being filled from another run.

SpecBlaxelboatDaytona (VM)E2BmicrosandboxModal (gVisor)Modal (VM)NamespaceNovitarun.cloudRunlooptama †Vercel
vCPUs44444444444964
CPU modelAMD EPYCAMD Ryzen 9 9950X 16-Core ProcessorAMD EPYCIntel(R) Xeon(R) Processor @ 2.60GHzAMD EPYCunknown6 models across replicates ⚠AMD EPYCAMD EPYCAMD EPYC2 models across replicates ⚠2 models across replicates ⚠2 models across replicates ⚠
Isolationfirecrackerqemu-kvmfirecrackerfirecrackerlibkrungvisorcloud-hypervisoroci-containerfirecrackerfirecrackercloud-hypervisoroci-containerfirecracker
Virtualizationfirecrackerqemu-kvmfirecrackerfirecrackerlibkrunnot-observablecloud-hypervisorfirecrackerfirecrackerfirecrackercloud-hypervisorunknownfirecracker
Memory (GB)7.787.767.787.777.7887.777.817.787.787.781511.528
Disk (GB)40.268.639.1104.239.3–51162.9103.639.36348.963.9
File systemxfsext4ext4ext4ext4overlayfsext4overlayfsext4ext4ext4overlayfsxfs
Mount options–relatime,rwrelatime,rwdiscard,relatime,rwrelatime,rw–discard,noatime,prjquota,rw–relatime,rwrelatime,rwnoinit_itable,relatime,rw–inode64,logbsize=32k,logbufs=8,noatime,noquota,nouuid,rw
I/O scheduler–NONEMQ-DEADLINE–NONE–––––NONENONEMQ-DEADLINE
Block size–4096409640964096–4096–409640964096–4096
Kernel6.1.1666.8.0-117-generic6.19.146.1.158+6.12.1094.19.0-gvisor7.2.67.1.46.1.158+6.18.36+6.18.325.15.0-174-generic6.18.49
OSDebian GNU/Linux 13 (trixie)Ubuntu 24.04.4 LTSDebian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)Debian GNU/Linux 13 (trixie)
Egress familyIPv6IPv4IPv4IPv4IPv4IPv4IPv4IPv4IPv4IPv4IPv4IPv4IPv4
Egress ASNAS396356AS16276AS396356AS396982AS20473AS8075AS31898AS401483AS31898AS24940AS16509AS16276AS14618
ASN nameLatitude.shOVH SASLatitude.shGoogle LLCThe Constant Company, LLCMicrosoft CorporationOracle CorporationNamespace LabsOracle CorporationHetzner Online GmbHAmazon.com, Inc.OVH SASAmazon.com, Inc.
Geo locationAshburn, Virginia, USRoubaix, Hauts-de-France, FRLos Angeles, California, USThe Dalles, Oregon, USElk Grove Village, Illinois, USSan Antonio, Texas, USFrankfurt am Main, Hesse, DELeesburg, Virginia, USPhoenix, Arizona, USFalkenstein, Saxony, DEColumbus, Ohio, USReston, Virginia, USAshburn, Virginia, US
Spec vs targetmatchedmatchedmatchedmatchedmatchedmatchedmatchedmatchedmatchedmatchedmatchedmismatchmatched

⚠ Heterogeneous fleets: Modal (VM) saw 6 host CPU models across its replicate sandboxes (AMD EPYC 9455P 48-Core Processor; AMD EPYC 9554 64-Core Processor; AMD EPYC 9554P 64-Core Processor; AMD EPYC 9654 96-Core Emb Processor; AMD EPYC 9654 96-Core Processor; AMD EPYC 9J45 128-Core Processor) · Runloop saw 2 host CPU models across its replicate sandboxes (Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz; Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz) · tama saw 2 host CPU models across its replicate sandboxes (INTEL(R) XEON(R) GOLD 6526Y; INTEL(R) XEON(R) GOLD 6542Y) · Vercel saw 2 host CPU models across its replicate sandboxes (Intel(R) Xeon(R) Processor @ 2.50GHz; Intel(R) Xeon(R) Processor @ 2.90GHz). Every median for each of these environments pools across those machines, so read their cells as the fleet's typical draw, not one machine's speed.

Data quality

Only providers marked validated by the dataset render as comparison columns. Every skipped or failed suite record below comes from this run, including universally unrecorded tasks and provider-specific failures; no older run fills a missing cell. Providers attempted without any validated metrics are disclosed after the table instead of becoming empty comparison columns.

ProviderSuiteOutcomeReason
E2Brealworld-mastrafailedPTS ran but every trial failed for 1 of 5 declared metrics: realworld_mastra_task_test_core (realworld-mastra/pts_realworld-mastra.xml) — attempted, no value recorded
E2Brealworld-mastrafailedPartial publication withheld unverified measurements: realworld_mastra_task_test_core
Modal (gVisor)cpu-nodefailedFailed to create sandbox: computesdk created-request preparation and verification failed: Modal control operation exceeded 5000ms
Modal (gVisor)cpu-nodefailedPartial publication withheld unverified measurements: node_web_tooling_runs_per_s
Modal (gVisor)networkfailedcomputesdk exec failed: Modal control operation exceeded 5000ms
Modal (gVisor)networkfailedPartial publication withheld unverified measurements: iperf_server_address_localhost_server_port_5201_duration_10_seconds_test_tcp_parallel_1, iperf_server_address_localhost_server_port_5201_duration_10_seconds_test_tcp_parallel_10, iperf_server_address_localhost_server_port_5201_duration_10_seconds_test_udp_10000mbit_objective_parallel_1, iperf_wan_direction_download, iperf_wan_direction_upload
Modal (gVisor)realworld-mastrafailedFailed to create sandbox: computesdk created-request preparation and verification failed: Channel has been shut down
Modal (gVisor)realworld-mastrafailedPartial publication withheld unverified measurements: realworld_mastra_task_git_clone, realworld_mastra_task_cold_install, realworld_mastra_task_lint_format, realworld_mastra_task_build_core, realworld_mastra_task_test_core
Modal (gVisor)systemfailedFailed to create sandbox: computesdk created-request preparation and verification failed: Modal control operation exceeded 5000ms
Modal (gVisor)systemfailedPartial publication withheld unverified measurements: pybench_milliseconds, sqlite_speedtest_seconds, git_seconds
Modal (VM)realworld-mastrafailedFailed to create sandbox: computesdk created-request preparation and verification failed: Channel has been shut down
Modal (VM)realworld-mastrafailedPartial publication withheld unverified measurements: realworld_mastra_task_git_clone, realworld_mastra_task_cold_install, realworld_mastra_task_lint_format, realworld_mastra_task_build_core, realworld_mastra_task_test_core
tama †networkfailedFailed to create sandbox: tama new bench-7c080ec3-c7c9-447d-862c-b5bd3620f9fd --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-7c080ec3-c7c9-447d-862c-b5bd3620f9fd failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †networkfailedPartial publication withheld unverified measurements: iperf_server_address_localhost_server_port_5201_duration_10_seconds_test_tcp_parallel_1, iperf_server_address_localhost_server_port_5201_duration_10_seconds_test_tcp_parallel_10, iperf_server_address_localhost_server_port_5201_duration_10_seconds_test_udp_10000mbit_objective_parallel_1, iperf_wan_direction_download, iperf_wan_direction_upload
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-27569028-1a38-46f9-b36c-549b24692577 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-27569028-1a38-46f9-b36c-549b24692577 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedPartial publication withheld unverified measurements: realworld_better_auth_task_git_clone, realworld_better_auth_task_cold_install, realworld_better_auth_task_lint_biome, realworld_better_auth_task_lint_deps_knip, realworld_better_auth_task_lint_format, realworld_better_auth_task_lint_spell, realworld_better_auth_task_lint_types, realworld_better_auth_task_lint_packages, realworld_better_auth_task_typecheck, realworld_better_auth_task_build
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-6bc7cabf-3777-4327-b065-0dffd618d134 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-6bc7cabf-3777-4327-b065-0dffd618d134 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-2aa81f6a-2df9-4762-bdeb-bf9a21938d7c --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-2aa81f6a-2df9-4762-bdeb-bf9a21938d7c failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-f3a9f147-374c-4af5-b645-62d10d7f085f --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-f3a9f147-374c-4af5-b645-62d10d7f085f failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-b877cf39-0970-4eb3-9575-c2c5ec975f79 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-b877cf39-0970-4eb3-9575-c2c5ec975f79 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-df1c5e5c-8f6d-46a5-b834-9df476585564 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-df1c5e5c-8f6d-46a5-b834-9df476585564 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-116e522d-ff95-414b-9d2f-c1627adb06f6 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-116e522d-ff95-414b-9d2f-c1627adb06f6 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-better-authfailedFailed to create sandbox: tama new bench-570baaf0-76ca-4f34-b78b-efd484412750 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-570baaf0-76ca-4f34-b78b-efd484412750 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-707fc315-f423-4c7a-a04c-633477b1de5a --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-707fc315-f423-4c7a-a04c-633477b1de5a failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedPartial publication withheld unverified measurements: realworld_openclaw_task_git_clone, realworld_openclaw_task_cold_install, realworld_openclaw_task_lint_oxlint, realworld_openclaw_task_lint_extensions_all, realworld_openclaw_task_typecheck, realworld_openclaw_task_test_types
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-da57ec1f-67e5-403f-8ba3-b1e466a32e04 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-da57ec1f-67e5-403f-8ba3-b1e466a32e04 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-1ea57b41-edba-4f5b-ad14-8d5735a07de7 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-1ea57b41-edba-4f5b-ad14-8d5735a07de7 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-6861eb6b-3fe3-42eb-8f16-575c363f5bed --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-6861eb6b-3fe3-42eb-8f16-575c363f5bed failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-c0ac6ac4-2528-488c-a8f0-fe19555adec0 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-c0ac6ac4-2528-488c-a8f0-fe19555adec0 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-1ab1e03f-f9a3-4490-852c-a11f565fb4be --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-1ab1e03f-f9a3-4490-852c-a11f565fb4be failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-e3b62363-3636-4504-9d28-14a52f315354 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-e3b62363-3636-4504-9d28-14a52f315354 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-2b54d6b9-c2f0-4fbb-835e-321fe10557bc --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-2b54d6b9-c2f0-4fbb-835e-321fe10557bc failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-f4310d55-9df4-49b5-b31d-b4c4ee5d319e --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-f4310d55-9df4-49b5-b31d-b4c4ee5d319e failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-89e60cda-2d9e-421c-af26-60541f2a8eb8 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-89e60cda-2d9e-421c-af26-60541f2a8eb8 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-2cb1e1cc-f0bc-4056-93f6-c4d78da16b28 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-2cb1e1cc-f0bc-4056-93f6-c4d78da16b28 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1
tama †realworld-openclawfailedFailed to create sandbox: tama new bench-a4c5c528-2ee6-4b6e-9e7b-ecb7634c0062 --ttl 0 --json --image ghcr.io/starslingdev/sandbox-benchmarks-toolchain:v8 --cpu 4 --memory 8192: exit 1; tama: bench-a4c5c528-2ee6-4b6e-9e7b-ecb7634c0062 failed to provision; inspect it in the console; provisioning: status=failed; process exit 1

Also in this run, not in the tables above: Daytona (container): pending, 0 metrics reported. An environment that reported nothing gets no column.