mo's gateway isn't new. The traffic is.

If you're evaluating mo, the enterprise control plane and harness, you've probably wondered why a company known for distributed caching ended up building the routing layer for coding agents. The short version is that we didn't pivot into a new business. We directed our caching expertise and production systems towards a new kind of request. Coding agents have moved from side experiments to mission-critical infrastructure for engineering organizations. This means the infrastructure they rely on needs the same rigor Momento’s caching infrastructure has always run on.

Caching made us care about the request path first.

Long before "agentic" was a word anyone used, Momento's business was making a request for cached data come back reliably, fast, at scale, and without the requester knowing where that data lived. On paper that's caching. In practice it's routing: every read has to find the right node, partition, and tier, without the client caring how.

Inference has the same shape. During prefill, a model consumes context and produces a KV cache, the attention state that the decode process reads back one token at a time. Once prefill and decode run on separate hardware, and that cache gets too large to move on every hop, you need something that behaves like a caching layer: shared, tiered, accessible storage between the compute that produces state and the compute that consumes it. We saw the same pattern over a decade ago when storage split from compute in data warehousing. KV cache is maturing from a private scratchpad on one GPU into durable, shared infrastructure.

It’s also why cache-aware behavior shows up on nearly every AI gateway's feature list. Being able to cache and reuse content directly supports cost controls and improves the speed of the workflow. Once work doesn't have to wait on a human watching a screen, tuning the same model for throughput instead of latency can cut cost by up to 50x. Getting that advantage means knowing what's cached, what's in flight, and where it lives. mo's gateway follows the same logic. Routing and cache-awareness were never separate concerns for us.

The gateway has always been the product.

The router is an integral part of every existing Momento production deployment. Our caching platform runs client requests through a router that authenticates, streams, and serves hot keys from an L1 cache before a request reaches storage. That's the core of the product, and it's the exact same architecture we ship as Valkey Router for self-hosting.

mo’s gateway takes that same router architecture and aims it at a new caller: a developer’s coding agent requesting a model call instead of an application requesting a cache key. Because the caller is an engineer's agent rather than anonymous internet traffic, mo leads with per-developer key attribution, per-ticket cost tracking, and mid-session failover, whereas a typical application gateway leads with static response caching and end-user rate limits. The gateway’s tracking choices are intentional governance questions: which keys were touched, what a call cost, and whether the data is visible afterwards. What sits on the other end might be a frontier model, an open-weight model, a managed provider, or your own GPUs, but the critical questions the mo gateway answers remain the same: which route, what cost, how fast. Building a gateway wasn't new for us, only the caller is new.

We don't trust numbers we haven't measured.

Our infrastructure team spent the better part of a year taking our platform's p99 latency from 2.7 milliseconds to 100 microseconds. These improvements came from a series of small, highly technical changes such as changing transport protocols, removing lock contention in the HTTP/2 implementation, and overriding kernel settings to pin interrupts and worker threads to dedicated cores. Each improvement required careful benchmarking before and after each change.

mo eval captures that same discipline. A gateway that only forwards requests can't tell you whether the model on the other end is the most efficient or cost effective choice. With mo, the same gateway that routes and meters your agents' traffic also runs the evaluation deciding where traffic goes next, testing every candidate model using real repository tasks, on infrastructure you already pay for, with cost pulled from the gateway's own metering rather than a self-reported number from whoever's trying to win the comparison. When a provider ships a change, you re-run the test suite and the routing updates with it.

We hold outside benchmarks to the same bar. Comparing KV-cache transfer approaches for disaggregated inference, we ran NIXL, UCCL, and Mooncake's transfer engine against each other ourselves rather than trusting any vendor's numbers, because those tools report very different results depending on message size and topology. That's the same standard mo eval applies to Claude, GPT, GLM, and whatever ships next: no benchmark, including ours, gets trusted until it's re-run on your own code.

Same job, new traffic.

None of this is a pivot. Caching taught us why shared, reachable state matters more than raw compute. Running a router taught us how to operate a gateway at hyperscale without becoming the bottleneck. Refusing to trust an unmeasured number taught us to build evaluation into the routing layer, not a report someone reads quarterly.

mo's task-to-model routing, its zero-data-retention gateway, and mo eval aren't three new products. They're the same infrastructure disciplines we've run for years, now pointed at a caller that happens to be a coding agent instead of an application. For engineering leaders, it means predictable spend, zero vendor lock-in, and instant governance, without changing a single line of how developers work.

With mo, the type of traffic has changed, but the job is still the same.