Momento Benchmarks

AI coding benchmarks from real production work

We test models and harnesses on real tasks from production repositories. Every run is metered for cost and scored by an independent judge panel.

Explore mo eval
We measure the whole toolchain
  • Model
  • Harness
  • Provider
  • Skills
  • Tools
  • Context

Metered cost and judged quality

Benchmark Results

Compare cost, quality, and speed side by side

Model comparison

Metered cost per task set, indexed to Claude Opus 4.8

Opus 4.8 1.0×
GLM-5.2 ~0.4×

About 60% lower cost with GLM-5.2, with similar judged quality (4.2 vs. 4.3 out of 5). Both passed 6 of 6 automated suites.

Harness comparison

mo vs. Claude Code · Same task and test criteria

2.5×Cheaper
2.6×Faster

Model comparison from six production repositories, scored by an independent multi-model judge panel.

See the harness comparison in action

Watch it happen: one task, same prompt, same hidden test gate, captured from a real run and shown accelerated. mo phase-shifts (Opus plans, GLM-5.2 builds) and lands the same result 2.5× cheaper, 2.6× faster.

How we test

One change at a time

Each benchmark holds everything constant except the one part being tested. The same tasks run repeatedly, and the results are compared side by side.

  • Real tasks

    Tasks from production repositories rather than public benchmark sets. Each task has a hidden test gate the agent never sees.

  • A baseline and a candidate

    Runs the same tasks, harness, skills, and tools. Only one part changes, such as the model.

  • Repeated runs

    Every task runs several times, so a single lucky or unlucky run can't decide the result.

Run your own benchmarks

Use mo eval to compare coding-agent setups on your own tasks, with metered cost, judged quality, and speed.

Run a free evaluation

Seattle Systems talk

Evals Is the New Black (Box)

Khawaja Shams and Daniela Miao discuss what it takes to evaluate coding agents and why the whole setup matters.

View the talk and captions