How Microsoft Orchard Makes Powerful AI Agents More Affordable

In this blog post How Microsoft Orchard Makes Powerful AI Agents More Affordable we will explain how Microsoft’s new research framework could help businesses build capable AI agents without depending on the largest and most expensive models.

Many organisations discover the same problem after a promising AI trial. The agent works, but every task sends a growing volume of information to a large model, response times increase, and the monthly bill becomes difficult to predict.

Microsoft Orchard tackles this problem from a different direction. Instead of assuming that a bigger AI model is always better, Orchard provides a reusable way to train and test smaller models inside realistic working environments. The aim is to teach models how to complete specific multi-step tasks efficiently, rather than paying for maximum general intelligence every time.

What Microsoft Orchard actually is

Orchard is an open-source research framework from Microsoft Research for developing AI agents. An AI agent is software that can understand a goal, decide what steps to take and use approved tools to complete a task.

For example, a normal chatbot might explain how to update a customer record. An agent could check the request, open the relevant business system, update the record and report what it changed.

Orchard is not a finished business product that you can switch on inside Microsoft 365. It is a foundation for training and evaluating specialised agents, and it is currently most relevant to organisations, software vendors and consultancies developing custom AI systems.

Its central component is Orchard Env, a reusable environment where agents can practise tasks safely. It can support software development, browser-based work and personal assistant activities such as handling calendars and email.

How the technology works without the jargon

Training an agent requires more than giving an AI model documents to read. The agent needs somewhere to act, make decisions, receive feedback and try again without damaging a live business system.

Orchard creates isolated sandboxes for this purpose. A sandbox is a temporary, separated computing space where an agent can run commands, work with files and test actions without receiving unrestricted access to the wider network.

These sandboxes run using Kubernetes, software that creates and manages large numbers of isolated computing environments. Orchard can create thousands of them when required, allowing many training or testing tasks to run at the same time.

Business task
 ↓
AI agent chooses an action
 ↓
Orchard runs the action in an isolated sandbox
 ↓
The result is checked and scored
 ↓
The model learns which actions produce better outcomes

Orchard also separates the training environment from the model and agent software. This matters because teams can reuse the same testing foundation across different models, tools and business scenarios instead of rebuilding everything for each experiment.

Why smaller agents could reduce AI costs

Large AI models are useful when a task requires broad knowledge or complex reasoning. But using the most powerful model for every invoice check, service request or account update can be like hiring a senior barrister to review every routine email.

Smaller models usually need less computing power. That can mean lower operating costs, faster responses and more deployment options, including private or tightly controlled environments.

Microsoft’s Orchard-SWE research model used about three billion active parameters. Parameters are the internal values a model uses to process information and make predictions. Despite its smaller size, it achieved 69.7% on a recognised software-engineering benchmark, rising to 73% when a second model helped rank its answers.

Microsoft reported that this result approached systems using models more than ten times larger. It is an impressive research outcome, although benchmark performance should never be treated as a guarantee that the same model will succeed in a particular business workflow.

Three business outcomes leaders should pay attention to

1. Lower cost per completed task

AI costs should be measured against completed business outcomes, not just the price of each model request. A cheap model that repeatedly fails may cost more than an expensive model that gets the task right the first time.

Orchard’s approach makes it easier to train and compare smaller models on the complete task. That creates an opportunity to reserve premium models for difficult decisions while sending predictable work to smaller, more economical agents.

This complements other cost controls such as smarter context management and showing agents only the tools they need.

2. Faster testing and less duplicated work

Custom agent projects often become expensive before they reach production. Each team creates its own test environment, data collection process and way of measuring success.

Orchard provides a shared layer that can be reused across coding, browser navigation and assistant workflows. The same environment can support data collection, training and evaluation, reducing the need to build separate infrastructure for every use case.

For decision-makers, the benefit is shorter experimentation cycles. Weak ideas can be stopped earlier, while promising use cases can move forward with clearer evidence.

3. More control over security and behaviour

An agent that can open files, send messages or change records creates a different level of risk from a chatbot that only answers questions. It needs strict boundaries around what it can access and what actions it can take.

Orchard sandboxes can restrict network access and place limits on processing power, memory and execution time. Each task can receive a fresh isolated environment, which reduces the chance of one test interfering with another.

This does not make an agent automatically secure or compliant. Australian organisations still need access controls, audit logs, data classification and human approval for high-impact actions.

These controls should align with Essential Eight, the Australian Government’s cybersecurity framework for reducing common cyber risks. An agent should never become a shortcut around multi-factor authentication, restricted administrator access or application controls.

What this could look like in a 200-person business

Consider a 200-person professional services firm receiving hundreds of internal requests each week. Staff ask about leave balances, project codes, device replacements, software access and client document procedures.

The firm initially connects every request to a large premium model. The agent gives good answers, but costs rise as conversations become longer, and routine questions consume the same expensive resources as unusual cases.

A better design could use a smaller, specialised model for common requests. It would be trained and tested against realistic scenarios, operate only within approved systems and escalate uncertain or sensitive requests to a stronger model or a person.

The business outcome is not simply a smaller AI model. It is a lower average cost per request, faster answers for employees and clearer control over which tasks AI can complete.

Where Orchard fits with Microsoft Foundry

Orchard focuses mainly on researching, training and evaluating agent models. Microsoft Foundry provides services for building, deploying, monitoring and governing AI applications and agents in business environments.

In practical terms, Orchard may influence how better specialised models are developed, while Foundry can provide the managed platform around a production agent. Our overview of Microsoft’s newer AI stack explains how these layers fit together.

Businesses also need to think beyond the model. Agent memory can preserve useful context between interactions, while hosted agents can reduce the infrastructure burden of running them reliably.

What CIOs and CTOs should do next

• Choose one measurable workflow. Start with a repetitive task where success, failure, time saved and cost can be clearly measured.
• Test more than one model size. Do not assume the largest model provides the best commercial result.
• Set escalation rules. Define when the agent must stop and involve a person or a stronger model.
• Build security into the test. Use isolated environments, minimum access and detailed activity logging from the beginning.
• Calculate cost per successful outcome. Include model usage, cloud infrastructure, monitoring, support and failed attempts.

Smaller does not mean less capable

Microsoft Orchard is important because it challenges the idea that every useful AI agent needs an enormous model. With focused training, realistic testing and a well-designed working environment, smaller models may complete valuable tasks at a much lower cost.

However, Orchard is still a research framework rather than a ready-made replacement for Microsoft Foundry, Azure OpenAI or Claude. The immediate lesson for business leaders is to design around the task and its risk, then select the smallest model that can complete it reliably.

CloudProInc brings more than 20 years of enterprise IT experience to this decision, combining practical AI knowledge with Azure, Microsoft 365 and cybersecurity expertise. As a Melbourne-based Microsoft Partner and Wiz Security Integrator, we help organisations test AI agents without losing sight of cost, data protection or Essential Eight requirements.

If you are not sure whether your current AI design is using more computing power and budget than it needs, we are happy to take a practical look with you — no strings attached.