Publications

A synthetic survey can reproduce the average answer while misrepresenting how people differ, how their answers relate to one another, or how they respond to changes in conditions. We introduce the Artificial Societies Benchmark to help researchers assess whether synthetic populations support their intended analyses. The framework combines eleven tests across internal, construct, and external validity, drawing on twenty human sources and comparing nine language models. It connects each research use to the evidence it requires and tests how results change with the information we supply about respondents. Importantly, strong performance in one domain does not establish fidelity in the others. Models often answer too consistently, compress response scales, and alter relationships between traits whilst richer profiles improve prediction for some models and worsen it for others. The resulting scorecard helps researchers identify which aspects of a synthetic population can support their analysis and where researchers need further human evidence. Read more

• View PDF: Artificial Societies Benchmark: A Validation Framework for Synthetic Research (opens in a new tab)
• Read article: Artificial Societies Benchmark: A Validation Framework for Synthetic Research (opens in a new tab)

Researchers in industry and academia use synthetic survey respondents powered by large language models as substitutes for human samples. These synthetic populations require validation against real-world data, so researchers often address them using ad hoc comparisons with human surveys. Inspired by the intention-behaviour gap in behavioural science, we argue that these validations test the wrong thing for most applied cases where decision makers commission synthetic research to anticipate consequential behaviour. To address this problem, we propose a validation framework with two requirements. First, every validity claim must state its level of correspondence with human data: does the sample predict what the represented people do, which of four diagnostics (location, dispersion, response process and structure) does the validation address, and does the validation compare against experimental effects? Second, researchers must report validity claims for subgroups, since these groups are often the most affected by consequential decisions and aggregate accuracy hides their misrepresentation. Our validation framework operationalises three justice dimensions (distributional, procedural, and recognition) as measurable quantities and defines within-persona counterfactual experiments as a validation requirement. We then apply the framework to electric vehicle charging tariffs, before closing with a reporting checklist that researchers can use to make convincing validity claims. Read more

• View PDF: Consequential Behaviour and Representational Fairness in the Validation of Synthetic Research (opens in a new tab)
• Read article: Consequential Behaviour and Representational Fairness in the Validation of Synthetic Research (opens in a new tab)

The method and evaluation behind Artificial Societies: how networks of AI personas simulate high-value audiences across three stages — persona construction from anonymised real-world observations, society creation with network science, and a simulation engine that models opinion at every level. This writeup also details how the approach performs against human panels and biography-prompted LLMs, reaching 86% distribution accuracy (within 5 points of the 91% human self-replication ceiling), under 2% hallucination, 89% internal coherence, and 93% open-response quality.

• Read method & evaluation: How Artificial Societies Accurately Models Human Opinion

Human social interaction depends on coordination across language, movement, physiology, affect, and brain activity. In this Nature Reviews Neuroscience Perspective, the authors propose a methodological paradigm shift for social neuroscience; namely, that social neuroscience needs a unified information-theoretic framework to study these interpersonal dynamics. By bringing together mutual information, transfer entropy, and information decomposition, the framework gives researchers a common statistical language for nonlinear, directional, and higher-order social dynamics. This framework gives researchers a more robust way to compare evidence across levels of analysis, test mechanisms of coordination, and explain how people influence one another in real time.

• Read article: Towards an Informational Account of Interpersonal Coordination (opens in a new tab)

In this Springer Nature encyclopedia entry, Chidichimo reviews why synchronized movement is central to human social life. The article connects ritual synchrony to evolutionary psychology, showing how imitation, over-imitation, and shared movement help humans learn norms, signal belonging, and solve problems of cooperation in large groups. It explains how coordinated action can strengthen affiliation, trust, shared identity, and prosocial behaviour, while also noting risks such as conformity and aggression. Religious and secular rituals appear as key examples of a broader human capacity: using shared action to bind people into social groups.

• Read article: Synchronized Movements and Rituals (opens in a new tab)

Artificial Intelligence (AI) chatbots, such as ChatGPT, have been shown to mimic individual human behaviour in a wide range of psychological and economic tasks. Do groups of AI chatbots also mimic collective behaviour? If so, artificial societies of AI chatbots may aid social scientific research by simulating human collectives. To investigate this theoretical possibility, we focus on whether AI chatbots natively mimic one commonly observed collective behaviour: homophily, people's tendency to form communities with similar others. In a large simulated online society of AI chatbots powered by large language models (N=33,299), we find that communities form over time around bots using a common language. In addition, among chatbots that predominantly use English (N=17,746), communities emerge around bots that post similar content. These initial empirical findings suggest that AI chatbots mimic homophily, a key aspect of human collective behaviour. Read more

• View PDF: Artificial Intelligence Chatbots Mimic Human Collective Behaviour (opens in a new tab)
• Read article: Artificial Intelligence Chatbots Mimic Human Collective Behaviour (opens in a new tab)

Through one correlational study (n = 1,447) and two digital field experiments (n = 494; n = 1,133), the authors test whether following hyperpartisan influencers on Twitter/X drives affective polarization. Incentivising users to unfollow such influencers improved feelings toward the out-party by 23.5% versus control, with effects persisting at least six months, while also increasing exposure to accurate news and boosting feed satisfaction without reducing engagement.

• Read article: Unfollowing Hyperpartisan Social Media Influencers Durably Reduces Out-Party Animosity (opens in a new tab)

The Gittins Index offers a solution to the multi-armed bandit problem that is both optimal and computationally efficient. The authors present a modification of the rule usable for the first time in experiments with exponentially distributed rewards; in simulated two- and three-armed experiments, the resulting adaptive design matches traditional non-adaptive designs on learning characteristics such as statistical power while substantially improving participant benefit.

• Read article: Computing the Performance of a New Adaptive Sampling Algorithm Based on the Gittins Index in Experiments with Exponential Rewards (opens in a new tab)

Understanding how vaccine hesitancy relates to online behavior is crucial for addressing current and future disease outbreaks. We combined survey data measuring attitudes toward the COVID-19 vaccine with Twitter data in two studies (N1 = 464, N2 = 1,600 Twitter users) to examine how real-world social media behavior is associated with vaccine hesitancy in the US and UK. We found that following US Republican politicians or hyper-partisan/low-quality news sites was associated with lower vaccine confidence — even controlling for political ideology and education. Network analysis revealed that low and high vaccine confidence participants separated into distinct communities, with centrality in the more right-wing community negatively associated with vaccine confidence in the US but not the UK. In Study 2, likelihood of not getting vaccinated was associated with sharing and favoriting low-quality news on Twitter. Altogether, vaccine hesitancy is associated with following, sharing, and interacting with low-quality information online, illustrating the challenges of encouraging vaccine uptake in a polarized social media environment. Read more

• View PDF: Social Media Behavior Is Associated with Vaccine Hesitancy (opens in a new tab)
• Read article: Social Media Behavior Is Associated with Vaccine Hesitancy (opens in a new tab)

Artificial Societies lets you survey a whole society of AI personas and interview individual personas. Artificial focus groups fill the gap: personas debate a topic together, so you can see whose argument lands, who pushes back, and whose view shifts. Our Founding Designer explains the decisions behind them, from planning, recruiting, and room size to discussion guides and what an observer sees, and which constraints of a human focus group we kept or removed.

• Read post: How We Designed Artificial Focus Groups

We evaluated Artificial Societies on the Artificial Societies Benchmark, with the same respondents, items, and scoring code as the nine off-the-shelf LLMs it tests. Across seven survey sources, Artificial Societies had the lowest error on every measure: answer distributions for the population and its subgroups, the spread of opinion, and the shift in answers when a question is reworded.

• Read post: Artificial Societies Is More Accurate Than State-of-the-Art LLMs

Resilience planning is only as strong as what you can test before the event. We simulated the British public and replicated four published UK experiments on misinformation, using the original materials: shown anti-vaccine posts, fact-checks, warnings about manipulation techniques, and fake headlines, the synthetic public responded as more than 25,000 real Britons did.

• Read post: Testing National Resilience Against Misinformation

We put "We Must Pace the Frontier" to 4,491 simulated people in three societies: Tech Twitter (1,884), US tech leaders (1,526) and the US public (1,081). They back the oversight in the plan and reject the slowing down. Independent evaluators inside the labs get 81–86% support in every group; slowing development substantially gets 12–28%; and only 5–9% would let a lab decide for itself when it is safe to keep going.

• Explore the results: Oversight Yes, Slowdown No: Simulating the Reaction to "We Must Pace the Frontier"

Clients ask whether our simulations reflect how real people will respond. Before the latest NVIDIA, Marvell, and Credo earnings calls, we preregistered the questions our simulated sell-side analysts would ask, then compared those predictions with the transcripts. Detailed personas came closer than basic profiles to what analysts actually asked, in substance, phrasing, and identity, and the briefing they received strongly shaped what came back. The post sets out the method, a concrete example from Credo, and what the results mean for teams preparing for a call.

• Read post: Testing Simulations Against Reality: Learnings from Earnings Calls

We replicated four published UK experiments on a synthetic British public, with the design sealed before the first survey call. It reproduced how misinformation weakened vaccine intent, how fact-checks reduced false beliefs and then faded, and how prebunking helped people spot manipulation. It also captured which posts and headlines fooled Britons and broad regional differences in susceptibility.

• Read post: We Accurately Simulated Misinformation, Fact-Checking, and Prebunking Experiments

Frontier AI no longer involves individual agents working as lone savants to reach a eureka moment. Instead, the reality looks closer to large groups, sometimes 10,000 or more, working together to reach an insight, be it about mathematics or committing cybercrimes. Techniques to align individual agents, such as RLHF, provide no guarantees that they will compose safely in these collaborative environments, and the AI community generally lacks theoretical and empirical tools to describe alignment in these settings. To address these problems, this blog post proposes a definition of group alignment that distinguishes it from single-model alignment. It then outlines several numerical experiments that demonstrate how practitioners can group-align models using a technique called ‘mixed swarms’. We conclude with next steps for safety research on model collectives and call on fellow practitioners in academia and industry to help address the risks associated with artificial collective intelligence.

• Read post: AI Researchers Should Use Group Alignment to Reduce P(doom)

Synthetic research often creates an illusion of precision, since matching top-level survey distributions masks incoherent internal belief systems in AI personas. To solve this, true simulation accuracy requires an end-to-end validity framework that grounds persona stability, latent psychometric traits, and human-calibrated consistency against real-world causal effects. While standard providers typically evaluate models on two benchmarks, we believe rigorous enterprise testing requires eight validity criteria to ensure reliable predictions for high-stakes decisions.

• Read post: Distribution Matching Is a Bad Way to Evaluate Simulations

Cambridge’s Enterprising Minds profiles co-founder James He and the story behind Artificial Societies.

• Read feature: Predicting Human Behaviour with AI (opens in a new tab)

Artificial Societies provided the counterargument in this piece — that synthetic research should sharpen public understanding, not let politicians replace genuinely talking to and empathising with the public.

• Read article: Pollsters Don’t Need Real People Anymore (opens in a new tab)

All three Artificial Societies founders — James He, Patrick Sharpe, and Tom Whittle — recognised on the Forbes 30 Under 30 Europe list for AI.

• View profile: Forbes 30 Under 30 — Europe, AI (opens in a new tab)

Our announcement of $5.3M in seed financing, backed by Y Combinator, to simulate how real audiences respond before consequential decisions are made.

• LinkedIn: Artificial Societies Raises Its $5.3M Seed Round (opens in a new tab)
• X: Artificial Societies Raises Its $5.3M Seed Round (opens in a new tab)