Artificial Societies vs Synthetic Users: methods compared

Artificial Societies records 86% distribution accuracy across 1,000 real-world surveys: 95% of the 91% human self-replication ceiling, according to its Survey Evaluation Report, January 2026.

Distribution accuracy measures overlap between simulated and human opinion distributions. It tells you how closely the shares choosing each answer match. The ceiling reflects variation when humans answer again, so the comparison accounts for the noise in human responses.

The report also measures the relationships between answers. Artificial Societies’ internal coherence was 89% in the same January 2026 report; its evaluation page sets that against a human-panel range of 60–95% and an acceptable threshold of 70%. That measure, Cronbach’s alpha, asks whether questions intended to measure an underlying attitude agree with one another. It supports analysis of related attitudes within a society; it does not establish whether findings generalise to a new population.

Artificial Societies’ self-contradiction rate was under 2% in the same report; its evaluation page puts human panels at about 9%. We label that measure hallucination rate on our evaluation page, where its definition concerns inconsistent answers rather than factual errors about the world. These are separate checks with separate purposes. We publish them because matching a survey total alone cannot tell you whether the simulated people hold coherent combinations of beliefs.

Artificial Societies constructs personas from observations of real people, then connects them through relationships and patterns of influence. Our method page describes the inputs: published research, online commentary and reviews, alongside anonymised social data and public records of voting and investment. Psychometric methods reconstruct how individuals reason, and we validate each persona against audience benchmarks and prior behaviour before it joins a society.

We call the network-building stage Society Assembly. A society contains 12 to 3,500 personas, according to Artificial Societies’ method and evaluation page. The Simulation Engine then exposes those connected personas to the audience’s situation and the material being tested. You can examine responses at market, segment or individual level, keeping the individual reasoning available alongside the population result.

That construction sets the scope of our work: we model audiences for which diverse observations exist, whether public or client-provided. For a hard-to-reach audience, the first task is to establish those observations. A result measured on an observed population offers no basis for promising accuracy on people for whom the method has nothing to draw on.

Synthetic Users runs AI-simulated one-to-one interviews and surveys with personas generated from a plain-language audience description, according to its homepage (opens in a new tab). The researcher describes the people they want to understand; the product generates participants and elicits their answers. Each synthetic participant takes part as an independent interview. That makes the individual interview the unit to examine when assessing whether the output suits your research. The company names user-experience researchers and product managers among its intended users, alongside marketing leads, agency owners, innovation managers and engineering leads.

It calls the product “a discovery co-pilot, not a replacement for real research” on the same page (opens in a new tab). We would take that description seriously when designing a study: use the interviews to investigate a question, then decide what human evidence the decision still requires.

Synthetic Users also publishes a worked comparison of synthetic and human interviews in its science post (opens in a new tab). You can inspect the interview topic and scoring criteria when assessing its claim of qualitative parity. Those details give an interview researcher something more useful to examine than a percentage detached from the work it scores.

Artificial Societies models a connected population; Synthetic Users describes independent interviews. The difference concerns what happens between people as well as how each persona is built. Our method connects individuals through relationships and influence, while Synthetic Users (opens in a new tab) starts its participants from the audience description you write. We would choose on those design choices before comparing headline percentages.

Table sources: our method and evaluation and January 2026 Survey Evaluation Report; Synthetic Users’ homepage (opens in a new tab), science post (opens in a new tab) and pricing page (opens in a new tab). The parity study describes its comparison as of late February 2024. Budget planning favours Synthetic Users’ published starting price; obtain our terms before making a cost comparison.

A society is worth considering when the connection between participants belongs in the research question. A stakeholder’s response to a message and the movement of opinion through a network are different objects of study. We model individual belief systems together with social-network dynamics, so you can examine the population without giving up the ability to inspect its constituent personas.

We run structured surveys, one-to-one persona interviews and moderated focus-group simulations on the same society in the same study, so choosing a society keeps individual questioning available. We would use the network when influence matters, and individual follow-up when a response needs explanation; the research question determines which part of the method deserves your attention.

Synthetic Users reports 85–92% synthetic-organic parity. Its science post (opens in a new tab) describes a late-February-2024 comparison between eight human interviews with UK school teachers about classroom technology and eight synthetic interviews using the same script. The analysis combined manual review with word and phrase frequencies and n-gram analysis, counts of word sequences.

The score assigns 30% to thematic overlap, 30% to depth and specificity of insight, 20% to comprehensive coverage and 20% to qualitative alignment, according to that study. These weights describe the judgement being made: how closely the synthetic interviews resemble the human interviews in their topics and substance. The published example gives you a topic, a script held constant and a method of comparison to inspect.

Our distribution-overlap result answers a different question. The Synthetic Users score assesses qualitative similarity across interviews; ours assesses the shares selecting survey answers. Neither percentage becomes a head-to-head result by placing it beside the other. For an interview-led study, qualitative similarity may be the relevant evidence. For population opinion, examine the distribution measure and the tests of how beliefs relate within that population.

Synthetic Users’ homepage (opens in a new tab) also claims that “21+ peer-reviewed papers” support its methodology. Supporting literature and evaluation of a particular product are distinct forms of evidence. We apply that distinction to our own research too: the collective-behaviour paper explains part of the basis for societies, while our survey figures have their own evaluation report.

We would send you to Synthetic Users for a discovery task centred on independent interviews, where you want participants generated from an audience description and an interview-based comparison to assess. Its published workflow (opens in a new tab) fits that task directly. If your decision is about what users say individually, we would not ask you to make network influence the centre of the study merely because we model it.

Synthetic Users also gives a research lead published terms to bring into a budget discussion. Its pricing page (opens in a new tab) lists annual plans starting at $12,500, sold through a research-token pool, with unlimited collaborative seats and up to 20% of unused tokens rolling over at renewal. For a team allocating interview research across colleagues, those are concrete purchasing terms to examine. They do not establish which product would cost less for your proposed work.

The same pricing page offers a 30-minute live demo study and advertises no free trial. We would bring an actual audience description and interview script to that session, then examine the answers against the questions your team needs resolved. Ask how the proposed interview depth affects token use before treating an annual starting price as the cost of your research programme.

Choose Artificial Societies when you need to investigate how opinion moves through a population as well as the reasons individuals give. We focus on strategic decisions, including investor communications and public affairs, where our method connects observed people through relationships and influence. If those relationships are part of your question, a society gives them an explicit place in the simulation.

We would also make the structure of responses part of the buying decision. Artificial Societies’ September 2026 validity framework sets out eight tests covering whether the stimulus caused a response shift, whether questions measure the intended attitude and whether results hold for real populations. Those tests let you examine the method beyond its aggregate accuracy figure. A persuasive interview answer alone cannot establish whether a persona holds stable beliefs.

The framework includes changes a research lead can inspect: rephrase a question, reorder its answer options or remove evidence supporting a claim. Wording changes should preserve the underlying judgement; removing substantive evidence should affect it. Ask us to relate those tests to the message or strategy you intend to study, so the evaluation concerns the decision you will actually make.

For audiences that are hard to reach, we begin with the observations available to build them. Our method uses public data and can include your proprietary research. That is a reason to discuss an audience outside your existing research programme with us, provided diverse observations exist. The useful first conversation concerns who you need to understand and what evidence describes those people.

Teneo’s own account of the engagement is in our Teneo case study. For your project, ask which audience observations and network relationships support the proposed study.

Artificial Societies’ network modelling rests on its peer-reviewed paper in the British Journal of Psychology (opens in a new tab) (He, Wallis, Gvirtz and Rathje, December 2024); it does not validate the survey accuracy figures, and the science page sets out what it found.

The finding concerns homophily: the tendency to form communities with similar others. It provides evidence for a specific collective behaviour in simulated societies, which is relevant to our decision to model people in networks. Artificial Societies’ survey-accuracy figures come from the separate January 2026 Survey Evaluation Report; the paper does not peer-review those figures. A buyer can inspect the evidence for the group behaviour and the survey performance as distinct claims.

We publish this comparison using our method pages, Survey Evaluation Report, validity framework, public results page and case studies, the cited peer-reviewed paper, and Synthetic Users’ product, pricing and science pages. The source checks listed below are dated 17 and 18 September 2026. We compare persona construction, research design and published evidence, with purchasing information where available.

This page reports published methods rather than a product trial or a head-to-head experiment. We exclude funding and company-location comparisons because they do not resolve the methodological choice. Recommendations express our judgement about the research task; published percentages retain their own definitions and test populations.

The Synthetic Users trade mark belongs to its respective owner. Artificial Societies is not affiliated with or endorsed by Synthetic Users. Information is believed accurate as of the source dates below. We will promptly correct errors brought to support@societies.ai, including changes to the product descriptions or commercial terms cited here.