Back
Method & Evaluation

How Artificial Societies accurately models human opinion

Artificial Societies uses networks of AI personas to simulate high-value audiences, used by Fortune 100 organisations to test how strategies shift opinions that matter. Here's how AS models human opinion, and how it performs against human panels and simple synthetic personas.

How it works

The standard approach of creating a synthetic persona is by prompting an LLM with an invented biography (e.g, "you are a 34-year-old teacher from Ohio"), which produces generic, stereotyped answers.

Artificial Societies is different. We employ a unique and proprietary approach, developed over years of research originating from the first large-scale AI society paper. This approach combines real-world intelligence gathering, the mapping of individual belief systems, and a unique network science modelling method.

1

Persona Construction

Rich psychographic profiles, built from anonymised, real-world observations.

  • Gathering public social platform observations, deep research, and first-party intel
  • A Behavioural Analysis Engine psychometrically triangulates observations
  • Resulting in a nuanced belief system that enables future predictions
2

Society Creation

Personas are placed in a network that captures real-world social influence.

  • Each society is a network of 200–3,500 personas
  • Personas are connected from observed interactions and historical associations
  • Network science models how opinion forms and spreads
3

Simulation Engine

Belief systems and network systems combine to model opinion at every level.

  • A question or stimulus is posed within a specific context
  • Each persona reasons from its beliefs; social-influence models calibrate it
  • Responses emerge for individuals, subgroups and whole populations

How it performs

Artificial Societies accurately capture human opinion distributions, and every individual response is interpretable with why they answered as they did.

Example 1 of 4

Is 'everyone should be treated fairly' an American value?

95% distribution accuracy
Human responses
Artificial Societies
AgreeDisagreeNot sure
Community Organizer
Director, neighborhood nonprofit
Advocating for underserved families.
Atlanta, United StatesFemaleMillennialDirectorNonprofit
Response
Agree
Comment
Fairness is the one thing this country is supposed to stand for. If we don't believe that, what are we even doing?
Hedge Fund Analyst
Analyst, long/short equities
Earning those outcomes.
New York, United StatesMaleGen ZMid-levelFinance
Response
Disagree
Comment
This place rewards the people who show up and work harder than everyone else. That's the real American value.

Replicated across 1,000 past surveys

Artificial Societies' surveys have been compared with 1,000 large-scale survey questions on different populations and opinion topics. In addition to top-line distribution accuracy, we also measure the multiple levels of realism that matter for consequential decisions.

Distribution accuracy

How much simulated opinions overlap with human panels, compared to how much human opinions self-replicate.

Artificial Societies86%
Biography-prompted LLMs67%
Human self-replication ceiling91%

What this means: Reliably test how a decision shifts opinion.

Hallucination rateLower is better

How often participants contradict themselves — AS personas stay focused better than human respondents.

Artificial Societies<2%
Human panels~9%
Biography-prompted LLMsup to 35%

What this means: Higher-quality signal on high-value audiences.

Internal coherence

Alignment amongst questions about the same underlying attitudes (Cronbach's α). Human panels span 60-95%; acceptable quality starts at 70%.

89%
070% threshold100%
Artificial SocietiesHuman panel range

What this means: Valid relationships between opinions, required for critical strategies.

Open-response quality

Distinct-word richness in a typical open-text response, benchmarked against 120,000 human social-media posts.

Artificial Societies93%
Biography-prompted LLMs61%
Human (social media)91%

What this means: Open-ended answers with human-level nuance.

How it differentiates

Artificial Societies is unique beyond digital twins and synthetic populations.

Bottom-up

Digital twins

Rich individual nuance
Interpretable at the person level
No top-line population diversity
Hard to reach high-value audiences at scale
Artificial Societies

Networks of enriched personas

Accurate top-line distributions
Rich individual nuance
Reliable crosstabs and segments
Reaches high-value audiences at scale
Top-down

Synthetic populations

Accurate top-line averages
Individuals are invented and lack nuance
Unreliable segments and crosstabs