EnvOpt: Automated Optimization for Agent Environments

The central idea

An LLM agent’s cost and success rate depend on three things: the model parameters, the harness, and the structure of the environment it interacts with. The first two are actively optimized through parameter tuning and harness optimization, while the environment is often considered immutable. Yet, the same information can be given to an agent in very different forms. For example, a company’s Slack history can either sit behind a remote server and be accessible via MCP or exported in plain text files that can be read and searched with terminal tools like grep. This choice alone can significantly affect how much work the agent needs to perform to solve a task.

Stationary environments refer to environments with large collections of artifacts that remain the same or are similar across many tasks. For example, a banking agent uses the same enterprise platforms and databases for a massive number of queries; many tasks for a company’s Slack assistant depend on the same Slack workspace; feature requests for a toolkit can rely on the same code repository. We aim to optimize how the stationary environments are organized and represented to agents.

Using training tasks without verifiers, we let a coding agent, equipped with access and budget to run a target agent on the stationary environment, optimize the environment representation for the target agent. The optimization target includes what the environment holds (e.g. adding a database to the environment or restructuring it) and ways it is exposed to the target agent (e.g. new API endpoints). The optimized environment is monitored with a validation set with verifiers to track cost and pass rate, and tested on a held-out test set.

Results

Across five benchmarks covering workspace tasks, data engineering tasks, and software engineering tasks, EnvOpt reduces the cost of the target agent on unseen test sets by an average of 48%, with higher or similar pass rates.

On AutomationBench, EnvOpt outperforms Meta-Harness by 9.67% pass rate with only 15% of its cost.

The environments optimized for the target agents can transfer to other agents, even to those with different harnesses and to models stronger than the one proposing the optimizations.

We applied the environment from EnvOpt on top of Meta-Harness and observe similar cost reductions and 16% improvement in pass rate, outperforming either Meta-Harness or EnvOpt alone.

EnvOpt costs around USD $150 for an offline optimization of the stationary environment in AutomationBench, and saves cost of running Claude Opus 4.8 by 77.5%, easily saving many times the cost of running EnvOpt. Lines are derived by random sampling actual cost of running the test set tasks.

Where the cost goes

To illustrate where the cost reduction of EnvOpt comes from, we break down input tokens and output tokens into categories, including different kinds of tool calls, tool execution results (environment feedback), thinking tokens, agent history, etc. We observe that the most significant token cost reduction comes from the environment feedback categories for the input tokens. There are also significant savings on the tool call categories of the output token side.

How it works

EnvOpt optimizes the environment in multiple steps, with each step using a disjoint set of training tasks without verifiers.

In each step, a research agent is launched in a sandbox and given the original environment as a local directory and the permission to run target agent in another sandbox. The research agent is instructed to run the target agent, analyze the trajectories, and edit the environment as a directory in its filesystem to reduce the cost of the target agent without hurting the pass rate (which the research agent judges on itself by proxies or answer consistency, since no verifier is provided for the training tasks). Notably, we do not allow the research agent to modify the harness of the target agent. We provide disjoint sets for tasks for each step to prevent overfitting.

To prevent the research agent from reward hacking during a training step (such as writing down the answers to the training tasks to lower the cost directly), we hide all the verifiers for the tasks used in training. This also mitigates the scarcity of high quality tasks for some stationary environments, as we can synthesize in-distribution queries (without synthesizing the verifiers) for training (we used synthetic training tasks for Workspace-Bench and Data-Eng-Bench, see paper Appendix). At the end of each step, we run validation queries (with verifiers) to decide if EnvOpt has converged or should continue running.

What the research agent built

Converting a similar unseen environment.

We do not require the environments to be exactly the same: slightly different yet highly similar environments can be converted using an automated converter written by the research agent during EnvOpt. For AutomationBench, the converter is a short program that runs at the start of every task. It calls the same API the agent uses to read that task’s data and rewrites the results into one compact text block placed at the top of the agent’s prompt, so the agent skips the search and listing steps and acts directly.

Making Cheatsheets.

Cheatsheets that store important information for future tasks are created during training. For example, a cheatsheet for AutomationBench stores all the API endpoints the agent needs to create Jira issues, post messages on Slack, record a Xero payment, etc., so it does not need to call api_search or api_fetch to get the endpoints.

Indexing.

For Workspace-Bench, the agent is deployed with massive amount of files simulating an enterprise employee’s computer. The research agent thus builds an indexing map as a single text file, that stores the paths, sizes, and types of all files needed, so the agent can simply search over the indexing file instead of spending tokens and time on the huge filesystem.

Building Database.

On Data-Eng-Bench, the research agent built an SQLite database that stores the locations of tables, which files each code script writes to, what are the types of information stored in each table, etc. The target agent can quickly locate the information needed without browsing through thousands of files.

Building Programs that Rank File Priority.

For the Bleve repository, the research agent built a program that takes any bug report and outputs the 8 .go files that are most likely to be impacted based on keyword matching. The target agent can quickly locate the files it potentially needs to investigate to fix the bugs.

Automated Testing of Edits.

For the Tenacity repository, a validate_tenacity.py is built to automatically fetch the code change made by the target agent, search for relevant test files, run the tests, and return verdicts. This quickly validates candidate edits with no extra token cost and speeds up the whole bug fix workcycle for the target agent.

Results

Transferability.

On AutomationBench, we examine the transferability of the optimized environment across different harnesses and models, with an average cost reduction of 75.7% and generally higher pass rates. Haiku 5.5 with EnvOpt achieves pass rate higher than Opus 4.8 and even close to Opus 5. On Workspace-Bench, we evaluate on Codex with GPT-5.6-Sol and Claude Code with Opus 5, and observe an average 56.7% cost reductions with steady pass rates compared with the baselines. On Data-Eng-Bench, on non-target agents, we observe an average cost reduction of 53.54%.

Building on Meta-Harness.

EnvOpt can be applied on top of Meta-Harness to further improve agent’s token efficiency and pass rate. Using the optimized harness for Haiku 4.5, EnvOpt can further push the Pareto frontier for Haiku 4.5. When transferring to Opus 5, although Meta-Harness alone does not show gain on pass rate, EnvOpt on top of Meta-Harness still improves the pass rate with much lower cost.

Ablation: Tool Optimization Only.

During environment optimization, EnvOpt can also optimize ways to access the new environment, including new tools. We study the effectiveness of only using the new tools. We find that tool optimization alone can lead to improvements on both token efficiency and pass rate, but is significantly lower than optimizing both the tools and the environment. Since tool optimization might be covered by harness engineering, this ablation shows that EnvOpt brings significantly more headroom for optimization in addition to harness engineering.

Software Engineering Tasks.

On Bleve repository, EnvOpt remains close to perfect score using only 56.9% of the completion time of gskill. On the tenacity repository, EnvOpt achieves 8.9% higher pass rate with fewer steps than advisor models. These results show that EnvOpt significantly outperforms other methods to improve agent token efficiency even on software engineering tasks.

Training Cost and Time.

We compare the cumulative training cost with the cost saved for running different agents on unseen tasks, assuming there are 1000 such tasks. We show that for frontier agents deployed in large scale, we can easily save more than 10 times of the cost of running EnvOpt.

Rejected environment optimizations

Storing too much information in one file leads to more exploration.

For Data-Eng-Bench, the research agent tried to create a big file to store all possible values for all columns in a database. Though this is meant to help the agent retrieve the correct information with less token, putting everything into a big file encouraged the agent to make more tool calls to explore the values, since one tool result reveals to the agent that there is much information it has not explored, which eventually increased the cost.

Building a search tool that returns too many files causes more reading.

For Workspace-Bench, a search tool was built to help the agent locate the files that it needs to read for the task. Though the tool successfully reduced token cost for searching the needed files, it often included unnecessary files in the search results, which motivated the agent to read more files, thus increasing the cost.

Adding tests that only fit a subset of the training tasks.

In step 2 of EnvOpt on the Bleve repository, a check was added to a test script in the repository that scans for a particular bug fix code snippet and fails the test directly if the snippet is present. This was meant to save the time needed to fix a bug and worked well for the step 2 tasks. However, for the step 3 tasks, some correct bug fixes were rejected by this additional check, and the agent needed to spend much more time to bypass the check.

Research agent behaviors

Using unreliable verification proxy.

Since we do not provide verifiers for training tasks, the research agent for Workspace-Bench occasionally used proxies such as answer length and keyword matching to confirm answer quality during training steps. This often turns out to hurt performance on heldout tasks. After observing validation set, we updated the instruction for research agent to prevent this behavior.

Fail to follow the instruction.

Despite being told not to touch anything related to the model or the harness, the research agent in Workspace-Bench still tried to change the reasoning effort of Codex during its step 6. This change was not effective on heldout tasks, as we only keep the environment directory in the research agent’s sandbox and discard the other artifacts. However, such behavior early in one training step could significantly affect the research agent’s optimizations during that step, potentially harming the performance.

Early flaw in optimization can remain uncovered in many later steps.

In Data-Eng-Bench, the research agent introduced a SQL statement splitter in step 1 that parses based on semicolon. But the splitter will fail if the semicolon is in quoted text. This bug in the optimization from step 1 remained uncovered until step 5. This inflates cost through all the 4 steps, as the target agent cannot tell that the splitter corrupted its query, so it attributes the failure to its own model and spends tokens to rebuild it.

Predicting validation performance using research agent behavior.

We find that for AutomationBench, cost reduction on validation set is correlated with how much tool call patterns change during each training step as measured in the per-task difference in tool call counts between training runs before and after the step, with Pearson correlation of 0.75. Alternatively, by looking at the fraction of training tasks that received more than 30% cost reduction in each step, its Pearson correlation with validation set cost reduction is 0.82 for AutomationBench, 0.70 for Workspace-Bench, and 0.56 for Data-Eng-Bench.

Acknowledgements & Citation

Read the paper · Explore the code

This project is a collaboration of University of Chicago, UC Berkeley Sky Computing Lab, and Bespoke Labs.

This project is partly supported by NSF IFML and Bespoke Labs. Sky Computing Lab is supported by gifts from Accenture, Algorithmic SuperIntelligence Labs, Amazon, AMD, Anyscale, Broadcom, cmpnd, Google, IBM, Intel, Intesa Sanpaolo, Lambda, Lightspeed, Mirendil, NVIDIA, Soma Labs, and VESSL.

If you use this work, please cite it as:

@misc{li2026envoptautomatedoptimizationagent,
      title={EnvOpt: Automated Optimization for Agent Environments},
      author={Mingxuan Li and Huanzhi Mao and Qiuyang Mang and Hanchen Li and Joseph E. Gonzalez and Alex Dimakis},
      year={2026},
      eprint={2611.07372},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2611.07372},
}