This is not merely a test of contract-related capabilities; it is, more fundamentally, an on-chain survival exam for Agents.

I woke up this morning to a flood of DMs—so many that I briefly thought AGI had already arrived! Upon closer inspection, it turned out OpenAI had just released a new benchmark for smart contracts. Let me break it down quickly.

In one sentence: An Agent’s ability to understand, repair, and deploy smart contracts isn’t about displacing crypto security firms. Rather, it points to a far more foundational question: Can Agents truly survive—and act—in crypto environments? And OpenAI’s newly released evmbench is precisely the yardstick designed to measure that survival capability.

I haven’t had time to dive deeply into the report while traveling during the Spring Festival, so I’ve only skimmed it rapidly. My initial impression: This is an innovative benchmark—but still very much in its early, rudimentary stage. It draws from 120 high-severity vulnerabilities observed across 40 real-world projects.

The “exam” consists of three sections:
Section One: Spot the Flaw — identifying vulnerabilities.
Section Two: Patch It — given vulnerable code, fix the bug.
Section Three: Attack — AI plays the role of a hacker, launching attacks via wallet operations in a locally deployed environment.
I won’t go deeper into technical layers here. Compared to evmbench’s methodology and question details themselves, what intrigues me more is why OpenAI released this.

Over the past few years, OpenAI hasn’t shown any particular interest in crypto. This release clearly bears the influence of crypto VC Paradigm—and Paradigm’s motivations are easy to grasp. Yet the first author listed is OpenAI itself, signaling this isn’t passive collaboration but active, intentional participation.

So where does that intention come from? A straightforward explanation is that this extends OpenAI’s internal Preparedness Framework—assessing frontier models’ capabilities at the edge of high-risk scenarios, with smart contract security being just one facet. But that’s clearly not the full story.

Agents leveraging crypto networks isn’t just possible—in some sense, it’s inevitable. OpenAI sees this too. The report explicitly states: “we expect agentic stablecoin payments to grow.”

But I believe this proposition goes well beyond payments. Today’s Agents are mostly tool-like: humans issue instructions, Agents execute them, and results are returned to humans. That model won’t be the endpoint. As Agent count and capability scale, direct inter-Agent collaboration will naturally emerge: one Agent hiring another to complete subtasks; one purchasing data or compute from another; one representing an organization negotiating, signing, and fulfilling agreements with another organization’s Agent.

Humans drop out of the transaction loop entirely. At that point, a fundamental question surfaces: What keeps this economy running—when humans are no longer in the middle?

Human society solves trust and coordination through millennia of carbon-based civilization: law, reputation, institutional guarantees, etc. But that system’s underlying logic is built for humans: participants maintain persistent identities, face social consequences, and can be held accountable. Agents inherently fail to meet those prerequisites. They can initiate thousands of transactions per second, destroy and rebuild identities at will, and ignore all jurisdictional boundaries.

Some might say: “Then forcibly bind Agents to human identities—use human authorization as a guarantee.” But that’s like strapping a horse-drawn carriage’s rulebook onto an airplane. It’s not just inefficient—it’s a fundamental misunderstanding of what Agents are. Worse yet, Agent evolution inevitably trends toward greater autonomy. Future Agents may owe allegiance to no human individual—no “owner,” no bindable human identity—existing instead as fully independent actors. At that point, the binding logic has no anchor at all.

Imposing human trust infrastructure onto an Agent society is like regulating aircraft with cart-road rules. Agent society needs its own infrastructure.

Smart contracts offer exactly that possibility. They don’t rely on “trusting the other party to fulfill their promise.” Instead, fulfillment conditions are encoded directly—and enforced by the network. No arbitrators. No waiting periods. When conditions trigger, outcomes happen automatically.

Even further, smart contracts may evolve beyond settlement tools into the very organizational fabric of Agents—governance rules, resource allocation, task scheduling—all defined on-chain and executed by code, with no human intermediary required.

And when some Agents live natively on-chain, interacting with various contracts becomes their entire daily existence. How do they read a contract? How do they find their place within complex protocols? How do they spot traps, avoid risks, and simply survive in a world with no customer support, no appeals process, and no “undo” button? All of this hinges on deep contract understanding and operational fluency. Insufficient capability means real loss; misjudgment means permanence.

So looking back at evmbench, the abilities it tests—reading contracts, spotting vulnerabilities, constructing transactions, executing attacks—are, at their core, answering one question: Has the Agent learned how to survive in this new world?

OpenAI likely already realizes: whoever’s Agent learns to operate autonomously on-chain holds the ticket to the next era. And going further—the future Agent may no longer be described as “whose.” They may simply be independent individuals.

Lastly, a brief unrelated note: You all collectively DM’d me because, over a year and a half ago, I launched a passion project called CryptoBench—thank you all for remembering it!
GitHub – xxcg322/CryptoBench

It was the first benchmark designed specifically to evaluate AI capabilities in the crypto domain—covering cryptography algorithms, blockchain fundamentals, smart contracts, ecosystem dynamics, and DAO governance. Its smart contract module included both detection and repair tasks. Notably, some of the vulnerability references used in CryptoBench overlap with those in OpenAI’s latest benchmark.

When CryptoBench launched, it received meaningful encouragement and support from many friends. Back then, however, I sensed relatively few truly grasped its significance. Though I haven’t mentioned it publicly in a long time, I remain deeply proud of it—and highly satisfied with what it achieved. In a few days, I’ll share the story behind it: why I believe benchmarks like this are critically important, what I learned building it, and why I stopped talking about it over the past year.

Also, benchmarking itself remains one of my strongest interests in the AI field. I’ve just completed a data study analyzing 22,000 AI benchmarks published between 2019 and 2025—with many intriguing findings. Once I’m back from travel, I’ll share those too.

[Wu Shuo]