Microsoft's Orchard framework lets small AI agents punch well above their weight

Key points

• Orchard is a free, open-source framework released by Microsoft Research that lets researchers build and train AI agents without needing expensive, proprietary infrastructure.
• Orchard-SWE, one of three demonstration models, scored 69.7% on SWE-bench Verified, a standard coding test, rising to 73.0% with an extra filtering step, using roughly 3 billion active parameters.
• Frontier AI systems achieving similar scores typically use models more than ten times larger.
• Microsoft released the training data and evaluation methods alongside the framework, so outside researchers can reproduce and build on the work.
• Three agent types are covered: software engineering, web browsing, and personal-assistant tasks such as calendar and email management.

Microsoft Research quietly published something researchers have wanted for years: a full, open toolkit for building AI agents that actually works at a realistic budget.

The project is called Orchard. An AI agent, for context, is software that does not just answer a single question. It plans, takes multiple steps, uses tools, and can recover when something goes wrong. Think of an agent as a digital assistant that can open your codebase, spot a bug, write a fix, run tests, and report back, all without you clicking anything.

Building agents like that has been expensive and closed off. The training pipelines, sandboxed testing environments, and datasets needed to make them work have mostly lived inside big labs. Orchard is Microsoft's attempt to change that.

What does Orchard actually do?

At its centre is a component called Orchard Env, a shared environment that runs on Kubernetes, a widely used system for managing thousands of software containers at once. Researchers can plug in a new type of task, a new model, or a new training method without rebuilding everything from scratch.

One practical benefit: Orchard lets researchers train an agent inside the exact same harness, the software wrapper that handles multi-step reasoning and tool use, that the agent will use when it is deployed. Earlier open tools forced researchers to train on a simplified stand-in and then switch to the real thing, which created a gap between training and real-world performance. Orchard closes that gap.

How well do the models actually perform?

Microsoft released three demonstration models to show the framework works across different tasks.

Orchard-SWE is the standout result. SWE-bench Verified is a well-known test where an AI model must read a real-world software bug report and produce a working fix. The model starts from a 61.4% baseline and reaches 69.7% through a sequence of training improvements. With a value model, a separate 4-billion-parameter system trained to recognise high-quality solutions and pick the best one from several candidates, it hits 73.0%.

All of this uses roughly 3 billion active parameters. For comparison, the frontier systems posting similar scores use models more than ten times larger, which means more computing cost and more energy per run.

Orchard-GUI, the web-browsing agent, was trained on just 400 demonstrations and 2,200 open-ended tasks. That is a small dataset by industry standards. It still scored 74.1% on WebVoyager and 67.0% on Online-Mind2Web.

What does this mean for ordinary people?

Directly, not much yet. Orchard is a research framework aimed at the teams building AI tools, not a product you can download and use today.

Indirectly, it matters. When the infrastructure for building capable agents is open and reproducible, more researchers can study how these systems behave, catch problems earlier, and build alternatives to closed commercial products. That scrutiny tends to be good for the people who eventually use the tools.

Microsoft Research published the training data and evaluation methods alongside the code, which means independent teams can check the results rather than taking the benchmarks on faith.

Common questions

Is Orchard something I can use myself?

Orchard is an open-source framework aimed at AI researchers and developers, not a finished product for everyday use. The code and training data are publicly available, but using them requires technical knowledge.

Does a higher benchmark score mean an AI agent is safe to trust with real tasks?

Not automatically. Benchmark scores measure performance on specific tests, not reliability in every situation. They are a useful signal, but real-world testing in the actual environment always matters more.