AWS Open-Sources Infinity, a Research Agent Runtime That Suspends Agents on Tool Calls and Resumes Them When Results Arrive, at About 100KB per Idle Agent in Its Benchmark

Agents spend most of their time waiting

A coding agent may start a build that runs for twenty minutes, a triage agent may wait hours for the next pull request, and an assistant may wait until the next morning for an approval. In an October 23 post on the AWS Open Source Blog, Shadaj Laddad, research lead for the Hydro team at AWS, points out that most agent harnesses run the loop as blocking steps: call the model, invoke a tool, wait for it to return. While the agent waits, it holds a process and its memory, and cannot respond to events that arrive in the meantime.

Infinity is the answer that post introduces. It is part of the Hydro project, a collaboration between AWS and UC Berkeley, written in Rust, released under the Apache 2.0 license, and hosted in the hydro-project/infinity repository on GitHub.

Agents as stackless coroutines

With async/await, Rust, JavaScript and Python suspend a function when it waits and save the state needed to continue. Infinity applies the same idea to agents: at the moment an agent calls a tool, its conversation history holds everything needed to resume it. The runtime executes each agent as a series of bounded slices: load the history from a store, run one model completion, persist the output, dispatch at most one tool call, and end. Tool results and user input arrive as messages on a shared input queue, and each one starts a new slice. Tools are written differently as a result: execute starts the work and returns immediately, and sends the result to the queue itself when it is done.

AWS wrote a benchmark backed by a simulated model: 80,000 agents running twenty turns each, 3.2 million completions in total, on a single thread of an AMD EPYC 9R14 with in-memory stores. Resident memory grew linearly at about 100KB per idle agent, mostly conversation history; at that rate, about 75,000 agents fit in 8GB, the memory of a Raspberry Pi 5. Note that this is runtime overhead measured with a simulated model and then extrapolated, not 75,000 agents running a real model on a Raspberry Pi.

Serverless deployment and synthetic tool calls

Because a slice never waits on an external operation and loads all its state from storage, Infinity can run on serverless platforms. AWS's reference deployment uses an SQS FIFO queue as the input queue (one message group per thread), invokes a Lambda function per delivery, stores histories in Aurora DSQL and deduplication state in DynamoDB, and implements timers with SQS delivery delays and EventBridge Scheduler; the repository includes CDK constructs. An agent waiting three days for a CI result uses no compute in the meantime.

Model APIs require exactly one result per tool call, but subscription-style tools keep producing events. Infinity inserts a synthetic tool call the model never made, receive_event__injected, whose arguments point back to the original call and whose result carries the event; the built-in spawn_thread and report_to_parent tools use the same mechanism for parent and child agents to talk, with no separate messaging system.

AWS positions Infinity as an early research release, not a managed service. Open questions listed in the post include how interleaved events affect decisions given that models are mostly trained on synchronous tool-call traces, scheduling priorities, compaction for long-running trees of agents, and how internet-facing agents can establish that events are authentic.