40% of 241 AI-Generated Medical Articles Found to Contain Medical Inaccuracies After Review by Specialists

Medrock Inc. (Headquarters: Tokyo, CEO: Nakamura / Wadayoshi) has officially launched its 'Medical Expert Network' service for AI and LLM development companies, leveraging a network of 500 active physicians.

Based on real project data accumulated through expert physician reviews, among 241 medical articles created by medical writers using generative AI, approximately 40% (98 articles) received medical feedback from physicians. Articles with feedback required an average of 6.8 corrections per article. This result highlights the critical need for specialist verification in an era where generative AI is producing vast amounts of medical information.

※Data from Medrock's internal projects in June 2026 (241 medical articles)

Background: Who Ensures the Accuracy of AI-Written Medical Information?

The widespread adoption of generative AI has led to an explosive increase in medical and health-related content. However, mechanisms to verify the medical accuracy of this content have not kept pace. In healthcare, even minor errors can directly impact readers' health decisions.

Through practical physician review processes, Medrock has quantitatively identified these 'invisible errors.' When active physicians reviewed specialized medical articles, the following findings emerged (based on Medrock's internal project data of 241 articles):

- Approximately 40% of articles (98 out of 241) received medical feedback from physicians - Articles with feedback required an average of 6.8 corrections per article - Of the 664 total feedback items, 243 (37%) involved substantial medical corrections (including 78 factual errors, 69 omissions of clinically important information, and 49 unsupported assertions lacking evidence) - Among these, 12 were critical errors that could mislead patients' decisions regarding medical visits, medication, or treatment choices (e.g., incorrect emergency instructions, presenting non-insurance-covered treatments as options, misidentifying the appropriate medical specialty for consultation) - Significant variation by medical field, with psychiatry articles receiving an average of 11.6 feedback items per article

Specific issues included statements that 'definitively assert disease causes despite no established consensus,' 'omit common clinical causes,' and 'contain inaccuracies regarding Japan's insurance system conditions'—issues that only specialists could identify.

These 241 articles were created by writers using generative AI. AI reproduces errors and outdated information present in its training data. As content creation becomes more efficient, the value of expert physician verification before publication increases—precisely because AI reproduces errors and outdated information from its training data.

Services Offered by the Medical Expert Network

To address AI and LLM development companies' challenges in securing medical experts and ensuring quality control, Medrock offers the following services:

- Creation and verification of AI training data: Development of medically accurate training datasets by specialists in respective medical fields - Medical evaluation of model outputs: Specialist physicians score and verify medical responses generated by AI (including error categorization) - Operation of specialist physician panels: Provision of verified physician panels by medical specialty, including quality control and confidentiality management

Pricing is determined through individual quotations based on project scale.

Target Clients

- Companies developing medical-specialized LLMs or medical AI products - AI labs aiming to enhance the performance and safety of general-purpose LLMs in the medical domain - Data and annotation companies responsible for creating medical training and evaluation datasets

Why Medrock Can Deliver This

- Network of 500 active physician reviewers: Cross-specialty network enabling assignment of specialists according to project needs - CEO is a physician: The CEO personally ensures medical quality accountability - Scoring technology to quantify physician performance: Objectively evaluates specialists' qualifications and review experience to determine project suitability - Quality review data accumulated through real operations: Enables quantitative demonstration of review effectiveness, as shown in this press release

Future Outlook

Medrock aims to expand its learning data and evaluation services for AI development companies, establishing itself as a foundational platform for specialized knowledge in the Japanese medical field. Additionally, the company will offer verification services for medical content and advertising expressions by specialists (compliant with pharmaceutical regulations and medical advertising guidelines), striving to become the infrastructure supporting the reliability of medical information in the AI era.

Service Website

https://medrock.co.jp/dr-checks/

Data Overview

Target: 241 AI-generated medical articles reviewed and verified by Medrock's specialist physicians in June 2026

Review: Conducted by active specialist physicians from Medrock's network

Metrics: Presence of medical feedback and number of corrections required, assessed from a medical standpoint

Company Overview

Company Name: Medrock Inc.

Headquarters: Miyamasu-zaka Building 609, 2-19-15 Shibuya, Shibuya-ku, Tokyo

CEO: Nakamura / Wadayoshi

Founded: June 2024

Business: Medical-specialized marketing support, physician-reviewed content services, and data and evaluation services for medical AI

URL: https://medrock.co.jp/dr-checks/

Contact: [email protected]