Multi-AI Debate Outperforms Single-Generation in Marketing Strategy Evaluation

- Among a total of 112 evaluations, MyMarketer's Debate Mode was superior in 97 cases (86.6%), equal in 14, and inferior in only 1.

- The comparison covered four industries: AI marketing SaaS, locally-focused beauty salons, web development companies, and tax accounting firms specializing in inheritance.

- Strategy outputs were anonymized and presented in randomized order, with evaluations conducted by GPT, Claude, and Gemini across 14 assessment criteria.

- This verification does not represent the overall performance of any specific AI model and is an internal benchmark with some variations in input conditions. Detailed comparison conditions are described in the full report.

WHAT Co., Ltd. (Headquarters: 3-3-13 Nishi-Shinjuku, Shinjuku-ku, Tokyo; CEO: Hitoshi Yamamoto) conducted an internal comparative benchmark on July 11, 2026, for the 'Debate Mode' feature of its marketing strategy support AI, 'MyMarketer,' using synthetic scenarios across four industries.

This evaluation compared MyMarketer's Debate Mode—where multiple AI marketers with distinct roles discuss and refine strategies—against single-generation strategies produced by Claude Fable 5 and GPT-5.6 Sol. The purpose was not to assess the overall capabilities of specific AI models, but to examine how strategy outputs differ in quality between collaborative multi-AI discussion and single-generation approaches.

Overview of Evaluation Results

The evaluation used synthetic QA scenarios based on the following four industries:

AI Marketing SaaS / Locally-Focused Beauty Salons / Web Development Companies / Inheritance-Specialized Tax Accounting Firms

For each industry, strategies generated by MyMarketer's Debate Mode and the comparison models were evaluated across 14 criteria.

Overall Results

Out of 112 total evaluations, MyMarketer's Debate Mode was judged superior in 97 cases.

However, Debate Mode was not superior across all criteria. There were instances where the single-generation approach was rated higher or where no clear advantage was found.

Comparison with Claude Fable 5: Superior in 48 out of 56 evaluations

MyMarketer's Debate Mode was compared against single-generation strategies from Claude Fable 5 across four industries.

In three industries—locally-focused beauty salons, web development companies, and inheritance-specialized tax firms—there were no cases where Debate Mode was judged inferior.

However, for the AI marketing SaaS scenario, the Claude Fable 5 output was generated on July 6, 2026, while MyMarketer's Debate Mode used inputs including proprietary research data generated on July 11.

Due to this mismatch in input conditions, the AI marketing SaaS results are treated as exploratory reference data rather than a strict apples-to-apples comparison.

For the other three industries, input conditions were standardized.

Comparison with GPT-5.6 Sol: Superior in 49 out of 56 evaluations

As an additional validation, MyMarketer's Debate Mode was compared against single-generation strategies from GPT-5.6 Sol.

GPT-5.6 Sol was run with maximum reasoning settings, and in all four industries, it received essentially the same business information as MyMarketer's Debate Mode. The same day's snapshot data was used.

In comparisons with GPT-5.6 Sol, there were no cases where Debate Mode was judged inferior across all four industries.

However, the agreement rate among evaluators for the AI marketing SaaS scenario was 83%, showing more variability in judgments compared to other industries.

The calculation method for agreement rate is published on the benchmark details page.

The only case where Debate Mode was inferior: 'Uniqueness'

The sole evaluation where the single-generation approach was superior was in 'uniqueness' within the AI marketing SaaS scenario when compared to Claude Fable 5.

However, due to the aforementioned differences in input conditions for this scenario, caution is required in interpreting this result.

Additionally, while single-generation was rated higher in uniqueness in one case, it is unclear from this single run whether this was due to the discussion mechanism or probabilistic variation in the generative AI.

In some criteria such as 'Trend/Why Now,' equal ratings were observed.

These results do not indicate that multi-AI discussion universally outperforms single-generation across all industries and evaluation criteria.

Future work will involve multiple rounds of generation and evaluation to identify domains where discussion is most effective versus where single-generation or divergent generation may be more suitable.

Background: The Need for Refined AI Outputs in Client-Facing Proposals

The use of generative AI to rapidly draft marketing strategies and proposal outlines has become widespread.

However, single-generation AI outputs often fail to sufficiently organize key strategic elements, even when presenting multiple tactical options. These include:

- Which challenges are considered most critical

- Who the primary target audience should be

- Where to differentiate from competitors in the market

- How to connect strategy to specific actions

- Whether the proposal is executable and economically viable

In particular, for web development firms and marketing agencies responsible for new client proposals or improvements for existing clients, simply presenting AI-generated text is insufficient.

Proposers must understand the content, explain it in their own words, and make final judgment calls.

MyMarketer's Debate Mode is a feature where multiple AI marketers with different roles discuss and structure marketing strategy arguments and conclusions.

This evaluation compared single-AI generation against multi-AI discussion-based strategy design to assess differences in output quality.

Anonymous Comparative Evaluation to Minimize Source and Order Bias

To minimize the influence of source identity and presentation order on evaluation, the following methods were adopted:

1. Input of essentially identical business information

Both MyMarketer's Debate Mode and the comparison models received essentially the same business information.

The comparison models were not allowed multi-turn discussions or iterative refinement, and were asked to produce single-turn text-based strategy outputs. Additionally, tools and web search were disabled for Claude Fable 5 and GPT-5.6 Sol on the single-generation side. However, the AI marketing SaaS scenario with Claude Fable 5 had differing conditions.

The Claude Fable 5 output was reused from a July 6 generation, while MyMarketer's Debate Mode used inputs including proprietary research generated on July 11. For the other three industries, input conditions were unified.

2. Anonymization of output sources

Evaluation models were not informed of the service or model names that generated each strategy. Outputs were labeled as 'Strategy A' and 'Strategy B'.

This was intended to reduce bias from brand perception or preconceptions about the source.

3. Comparison under both AB and BA conditions

To assess the impact of presentation order, evaluations were conducted under two conditions:

Strategy A first,