Can AI replace market research? Evidence explained

Our 86% distribution accuracy is 95% of the 91% human self-replication ceiling, measured against 1,000 real-world surveys in the Artificial Societies Survey Evaluation Report, January 2026 (opens in a new tab). Distribution accuracy measures the overlap between simulated and human opinion distributions. The ceiling reflects people’s agreement with their own answers when asked again. It gives the result a human reference point, while the evaluation’s biography-prompted large language models (LLMs) provide a baseline for personas created from invented biographies.

Source: Artificial Societies, Survey Evaluation Report, January 2026, presented on our method and evaluation page. Self-contradiction means inconsistent answers; the measure does not test factual errors about the world.

Distribution overlap gives you evidence about the spread of opinions. It leaves a separate question about whether people holding one belief also hold another. Our September 2026 validity framework explains why matching answer totals cannot establish those relationships. For a researcher examining segments, the combinations of answers matter alongside the totals.

On research with reachable users, we agree with Nielsen Norman Group’s critique. Edoardo Chidichimo and Felix Wallis call criticism of synthetic research “well-warranted” in Distribution Matching Is a Bad Way to Evaluate Simulations, September 2026. That footnote cites Bisbee and colleagues, Sarstedt and colleagues, and Harding and colleagues. We regard those objections as questions our method must answer.

Moran and Rosala’s Nielsen Norman Group article (opens in a new tab), published in June 2024 and updated in September 2025, states: “Synthetic users cannot replace the depth and empathy gained from studying and speaking with real people.” We agree. A simulation of a reachable user does not provide the contact with that person that the authors defend.

The article also gives synthetic users a role: “Use synthetic users to help you prepare for research studies with real users.” It asks researchers to treat synthetic findings as hypotheses for testing. Preparation is a defensible use, and it fits our recommendation to explore possibilities before commissioning targeted human research. Our own published boundary is explicit: “Synthetic audiences will never be a true substitute for the people they seek to represent.” That is the position in our September 2026 post, including when simulation lets us run a study recruitment cannot support.

Artificial Societies starts with observations of people. Our method page describes published research, online commentary and reviews, anonymised social data, and public records of voting and investment. Persona construction uses psychometric methods to reconstruct belief systems: how an individual reasons, rather than a biography alone. We check each persona against audience benchmarks and prior behaviour before connecting it to a society through observed relationships and patterns of influence.

The Simulation Engine exposes those personas to scenarios and stimuli. You can examine the resulting opinions at population, segment or persona level. This is the mechanism behind testing a strategic narrative: the audience has a basis in observed people, and the analysis can examine differences within that audience. Diverse observations are a prerequisite. Where neither public nor client-held observations exist, the method has nothing to ground personas in; accuracy on an observed population does not transfer to an unobserved one.

Our peer-reviewed paper on group behaviour (He, Wallis, Gvirtz and Rathje, British Journal of Psychology (opens in a new tab), December 2024) supports the network modelling, not the survey accuracy figures; the science page explains what it found.

The Market Research Society (MRS) treats representation and sampling quality as central concerns in its Delphi Group report, Using synthetic respondents for market research (opens in a new tab), published in 2024. Its technical objection is direct: “LLMs are good at averaging, so deliver few surprises and level results into generic information.” The report also raises concern about obscuring the distinction between primary research and synthetic data. Those concerns make the origin of a persona and the tests applied to its answers relevant to a buyer.

The ICC/Esomar International Code on Market, Opinion and Social Research and Data Analytics (opens in a new tab), effective September 2025, sets disclosure duties. Article 7(e) requires clients to be informed about AI use, including synthetic data and personas, and about the extent of human oversight. Article 9(b) extends disclosure to significant use in sampling, deployment, analysis or interpretation.

The MRS document is a report; the ICC/Esomar document is a code. Neither calls for a ban on synthetic research. We publish our method, data sources and evaluation so a researcher can inspect how the work is produced. For your own study, specify which findings come from simulation and describe the human oversight involved. A presentation should preserve that distinction when its findings leave the research team.

Synthetic market research inherits the gap between stated intentions and behaviour. Our September 2026 validity framework acknowledges that answers to a survey can be weak predictors of action. Matching those answers is evidence about reported opinion. A decision that requires observed behaviour therefore still needs research capable of observing it, even when simulation has helped identify the question to investigate.

Bisbee and colleagues’ Synthetic Replacements for Human Survey Data? (opens in a new tab) appeared in Political Analysis in 2024; Gao and colleagues published Take caution in using LLMs as human surrogates (opens in a new tab) in the Proceedings of the National Academy of Sciences in 2025. Lukauskas and Šarkauskaitė’s 2026 preprint (opens in a new tab), which is not peer-reviewed, states: “LLM samples are not a drop-in replacement for human survey data”. A fluent answer alone cannot settle those concerns.

Our response is to examine how answers behave under testing. The September 2026 framework sets out eight tests of whether a stimulus caused a response shift, whether questions measure the intended attitude, and whether results hold for real populations. We state: “We hold ourselves to all eight.” Reordering answers should not move opinion; changing the evidence behind a claim should. The distinction matters when a simulated audience appears persuaded: you need to establish whether it responded to the argument or to the way the question invited agreement.

Audience simulation addresses barriers that occur in our case studies: confidential content, the effect of asking, and access to the audience. For an earnings-call engagement, restrictions around selective disclosure prevented testing draft material with real investors. In a transport engagement, exposing the audience to draft messages would have changed the debate under study. These are reasons to simulate before acting, with the results treated as forecasts of reactions.

Teneo’s engagement concerned audiences that were impossible to reach. Teneo’s own account of the engagement is in our Teneo case study.

Simulation also permits interrogation of belief systems and their response to a stimulus, as our September 2026 validity framework describes. In the earnings-call case study, simulated analysts’ questions matched the live call, and simulated press coverage matched the next morning’s framing. That account illustrates a decision rehearsal followed by a real event. It is a single engagement, separate from the survey evaluation, and should be read at that scope.

We would use Artificial Societies to explore strategic options, then commission targeted human research to test the findings that determine the decision. You can also reverse the sequence: establish a baseline through traditional research, then use simulation to test confidential content against it. The choice follows the material you can disclose and the audience you can reach.

For a straightforward general population survey about a publicly available product, we would recommend a traditional panel. Biography-prompted LLMs may suffice for low-stakes directional feedback. Networks of enriched personas fit decisions where you need to examine stakeholder reasoning and relationships.

For validation, our September 2026 framework calls for unpublished or newly fielded studies, since public survey programmes can appear in model training data. Agree on the human evidence that will test your finding before treating simulation as support for a strategic recommendation. With a confidential announcement, that means distinguishing what you can test with people before release from the reactions you will compare with the simulation after it becomes public.