Establishment of an AI Safety Foundation to Support Safe Collaboration Between Humans and AI

In the "Research and Development on AI Safety / Strengthening AI Safety R&D" project (hereinafter referred to as the Project) promoted by NEDO (New Energy and Industrial Technology Development Organization), five organizations—NEDO, the National Institute of Advanced Industrial Science and Technology (AIST), Citadel AI Inc., Copy Inc., and the University of the Ryukyus—have developed and published guidelines and evaluation protocols that serve as a common foundation for ensuring the safety of AI systems.

The guidelines and protocols formulated under this project focus on ensuring safety from the planning and design stages of AI systems through to their evaluation and operation. They organize fundamental concepts and procedures for businesses developing and implementing AI-driven systems to identify risks and consider appropriate countermeasures.

By widely utilizing these guidelines and protocols, the project aims to permeate common concepts and procedures for AI system safety throughout society and accelerate the development of a common AI safety foundation for the secure use of AI.

1. Background Against the backdrop of initiatives such as the Hiroshima AI Process, which was launched at the 2023 G7 Hiroshima Summit, discussions and structural development regarding AI safety are progressing globally. In response to these international trends, Japan has established the AI Safety Institute (AISI) and is participating in international discussions.

This project supports efforts to promote the creation of international rules for the safe and secure use of generative AI from the perspective of research and development, bringing together government and private sectors. In recent years, as human-AI collaboration in judgment and action increases, responding to the common challenge of how to design, evaluate, and operate AI safety has become essential.

The project conducted research and development aimed at establishing a common foundation for evaluating and operating AI safety. While AI technologies and application fields are diverse, challenges such as designing safe interaction between humans and AI and ensuring safety through judgment, verification, and operation are common across all fields.

The project is structured to develop evaluation and management technologies that serve as a "yardstick" for safety, develop AI safety evaluation and implementation technologies for specific application domains, and organize/systematize these results into a form practical for business use, leading to the formulation of guidelines for AI safety implementation.

2. Key Outcomes To address the diverse challenges of AI safety, the project developed a wide range of guidelines, evaluation methods, templates, and evaluation environments that span the stages of "design, evaluation, and operation."

(1) Formulation of Multimodal AI Quality Management Guidelines As the core outcome of the project, AIST has formulated guidelines that organize quality management perspectives and processes for multimodal AI, which receives images and text and responds primarily through text. Focusing on "cross-modal correspondence capability" as a unique evaluation perspective for multimodal AI, the capability is classified into four levels. To ensure the safety and quality of multimodal AI systems, identifying the required level of cross-modal correspondence capability is crucial, and corresponding actions for each stage of the lifecycle have been organized systematically according to these levels.

Furthermore, the guidelines present three case studies: automated image captioning, image diagnosis for aging infrastructure, and content moderation on social media. They highlight points of caution and quality management issues in situations involving human judgment and supervision.

This guideline provides a common design and evaluation framework for ensuring safety and quality based on the characteristics of multimodal AI and serves as a base for practical application of AI safety.