Same Model, Better Harness: Lower Costs, Better Results

In August, NVIDIA published a result that changes how businesses should approach harnessing AI.

Their researchers took Claude Opus 5, left the model completely untouched, and ran it inside an agent system they built called AVO. The test was ARC-AGI-3, a set of 2D games that ship with no instructions, where the system has to work out the rules and win the way a person would. Opus 5 scores about 30% on its own. Inside AVO it scored 100%, clearing all 183 levels across 25 environments in 6,624 actions.

Two design choices account for most of that gap. AVO holds persistent memory, carrying prior attempts, evaluation results, and accumulated reasoning forward so the agent resumes from what it already knows rather than rebuilding its understanding every session. It also runs a supervisor that watches the full trajectory, detects stagnation and repeated unproductive cycles, and redirects the agent toward a different strategy.

NVIDIA’s conclusion is the part worth keeping: long-horizon capability is a property of the whole system, not of the model by itself. TechCrunch put it more bluntly, calling the harness rather than the model the real hero.

The Same Engineering Lowers Your Bill

This matters commercially because harness quality shows up on your invoice.

Databricks benchmarked coding agents against its own multi-million-line codebase and found that cost per task varied by more than 2x depending on which harness ran the model, holding the model, the thinking effort, and the output quality constant. The cheaper harness sent roughly 3x less context per turn, kept a tighter working set, and finished in fewer runs. In separate work on harness tuning and caching, Databricks reported close to 50% fewer generated tokens with no observed quality degradation.

Most performance improvements cost money. This one returns it, and it requires no migration, no new contract, and no larger model.

Build on Open Standards

We argued in April that AI agnostic is the only way, and in July that reliability comes from the engineering around the model rather than the model itself. NVIDIA’s result connects those two positions, with one condition attached: the harness has to be portable, or you have rebuilt vendor lock-in one layer higher.

Open standards are how you avoid that. The Agentic AI Foundation, now part of the Linux Foundation, stewards Model Context Protocol for tool and data access along with AGENTS.md for project context. Agent Skills packages procedural knowledge as a SKILL.md folder, is developed in the open, and already works across Claude Code, Cursor, Copilot, Gemini CLI, Codex, and dozens of other clients. Agent Plugins, governed by a technical steering committee spanning Amazon, Cursor, Microsoft, OpenAI, and Vercel, bundles those pieces into a single portable format.

Standards are what make harness work an asset rather than a sunk cost. Your memory design, supervisor logic, and codified procedures stay yours when the model underneath changes, and it will change. The market is already building for that reality: Ramp shipped a model router covering eight providers through one API.

Our JS harnesses are assembled on these standards, which is how our daily AI news engine runs a full agentic team for under $0.33 a day.

Recommendations to Consider

Measure cost per completed task instead of cost per token. Tokens are an input, while an accepted deliverable is the unit your business cares about. Start logging both.
Examine memory before you evaluate models. Find out where your agent’s state lives between runs. If it rebuilds context from scratch each session, you are paying twice for the same reasoning.
Put a supervisor check on your most important workflow. A second pass that asks whether the work is progressing will catch the stall loops that consume budget quietly.
Write an AGENTS.md for your projects. A useful first version takes under an hour, and every compatible agent reads it.
Turn one repeatable process into a Skill. Choose the task you explain most often and write it down once as a SKILL.md.
Cut context on purpose. Tighter working sets were the mechanism behind the 2x cost difference Databricks measured, so send less and expect the same result.
Read benchmark scores as direction rather than as a target. A 100% result reflects a heavily engineered research setup. What transfers to your business is the method: memory, supervision, and verification.

None of this requires a research lab. Memory design, supervision, and context discipline are engineering decisions any capable small business can make; the gains are measurable in both reliability and cost, and the standards now exist to keep the results yours.

Need help optimizing your harness? Let’s talk.

Same Model, Better Harness: Lower Costs, Better Results

The Same Engineering Lowers Your Bill

Build on Open Standards

Recommendations to Consider

Sources