Microsoft has unveiled and open-sourced code-testing-generator, an innovative polyglot agent designed to autonomously create and validate unit tests. Released as part of the MIT-licensed dotnet/skills repository, this agent addresses a critical gap left by conventional coding assistants: the ability to generate contextually relevant and executable tests without explicit instructions on frameworks, file locations, or assertion types. Unlike a hosted service, it operates locally within existing coding agents, ensuring code privacy and control.
The agent employs a sophisticated "Research-Plan-Implement" (RPI) pipeline to achieve its high accuracy. It begins by thoroughly analyzing the repository to identify code requiring tests, detect programming languages and testing frameworks, understand existing test conventions, and pinpoint actual build and test commands. Following this research phase, it selects one of three strategies—Direct, Single Pass, or Iterative—to write tests, always avoiding modifications to production code or external dependencies. Before reporting completion, code-testing-generator performs five rigorous checks, including a lightweight form of mutation testing, assertion quality assessment, scenario mapping, full workspace builds, and confirmation that new tests are discoverable by the repository's own test command.
This release is highly significant for developers and researchers due to its remarkable performance and practical utility. On Microsoft's internal 152-task benchmark, code-testing-generator achieved an impressive 92.1% task completion rate, significantly outperforming stock GitHub Copilot's 78.9% when using the same underlying model and prompts. The agent particularly excels with vague prompts, resolving 88.8% of tasks compared to Copilot's 66.3%, and flawlessly handled all 15 tasks targeting specific code differences where Copilot failed entirely. This capability promises to dramatically enhance the efficiency and reliability of automated unit test generation.