Meta-Harness: Automated LLM Code Optimization Breaks Barriers

Meta-Harness is an outer-loop system that automatically searches over harness code for LLM applications. In this study, Meta-Harness improved online text classification by 7.7 points while using 4x fewer context tokens. This matters because it automates harness engineering to outperform manual methods by leveraging rich diagnostic data.

Changing the code harness around a fixed large language model can create a 6x performance gap, yet this process remains largely manual. Let me break down how Meta-Harness automated LLM harness optimization code uses a filesystem to search over harnesses and achieve state-of-the-art results without relying on compressed feedback.

How Does Meta-Harness Automated Optimization Compare to Manual Engineering?

The results from this 2026 Stanford and MIT research are quite striking. On online text classification, harnesses discovered by Meta-Harness improved over Agentic Context Engineering (ACE) by 7.7 points while using 4x fewer context tokens. It matched the next-best text optimizer's final performance after 60 proposals with only four. In retrieval-augmented math reasoning, a single discovered harness improved accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On TerminalBench-2, the discovered harness surpassed the hand-engineered Terminus-KIRA and ranked number one among all Haiku 4.5 agents.

How Does the Meta-Harness Filesystem Access Work?

Here is the fascinating part. Unlike existing text optimizers that compress feedback too aggressively, Meta-Harness uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. This allows the system to reason over raw prior code rather than relying on lossy summaries. In the most demanding setting, the proposer reads a median of 82 files per iteration, referencing over 20 prior candidates per step. A single evaluation can produce up to 10,000,000 tokens of diagnostic information, which is roughly three orders of magnitude beyond the largest feedback budgets used in prior text optimization settings. This builds on earlier research in credit assignment and meta-learning, applying it to the specific domain of harness engineering.

What Are the Applications and Limitations of Automated Harness Search?

Beyond outperforming existing harnesses, the discovered strategies generalize to out-of-distribution classification datasets and unseen base models in the math setting. The search run completes in a few hours of wall-clock time and produces readable, transferable strategies. However, the authors acknowledge that overfitting in code space is a concern, though it is more inspectable than weight-space overfitting. The experiments demonstrate that harness search works with one particularly strong coding-agent proposer, Claude Code, but a broader study of how the effect varies across proposer agents remains for future work. A natural next step is co-evolving the harness and the model weights.