HireRoo Releases 'AI Collaboration Tasks,' a New Test to Evaluate 'AI Collaboration Skills'

HireRoo Inc. (Headquarters: Shibuya-ku, Tokyo; Representative Director: Kosuke Kuzuoka) has released 'AI Collaboration Tasks,' a new problem format for its coding assessment service 'HireRoo' that visualizes and evaluates the process by which candidates utilize AI agents to solve tasks.

This is the first coding assessment service in Japan based on 'Agentic coding' (development utilizing autonomous AI agents).

Unlike traditional systems that only record dialogues or prompts given, this system is a groundbreaking, unprecedented function that quantitatively evaluates the execution process of the AI agent itself.

Background: The evolving definition of an 'active engineer' due to AI

With the rapid proliferation of generative AI, its utilization has become standard in development sites, leading to a shift in the definition of an engineer who can succeed. The company received many comments from CTOs and recruiters stating that traditional coding tests were no longer sufficient to assess a candidate's true capability.

In response, the company developed the 'AI Collaboration Tasks,' which provides an Agentic coding environment and evaluates the process itself.

Three key features of the new 'AI Collaboration Tasks':

1. Agentic coding-based assessment and process visualization Candidates use AI agents to solve tasks, and the system visualizes their thought processes, such as 'which files were opened, what commands were executed, and how the trial-and-error process unfolded.' This data is calculated as the 'SEI (Software Engineering Index),' allowing recruiters to evaluate candidates objectively.

2. 'Agent Playback' for recruiters to review processes A feature that allows recruiters to play back the autonomous actions of the AI agent chronologically, like watching a recorded video. Recruiters can experience the process of 'why the code turned out that way.'

3. Zero environmental setup required Candidates do not need local setups. By simply opening a browser, they are provided with a practical development environment integrated with the latest AI agents (such as Claude Code and Codex).

Future Prospects: Redefining hiring standards for the AI era By evaluating the quality and process of the interaction with AI, in addition to the final output, HireRoo will accurately visualize the skills required of engineers in the AI era.