How We Designed Artificial Focus Groups for AI Personas

Artificial Societies lets you run research on societies of AI personas. You can survey a whole society to see what it thinks, and interview individual personas to understand why. The next step was to provide the space in between: a way to talk to a group of personas.

The most straightforward approach would be to put a group of personas in a conversation, get an answer from each of them to every message, and summarise their answers too. But separate answers don’t show you how those people think together: whose argument lands, who pushes back, or how one view shifts the opinion of another. For this, the personas must interact.

So we built artificial focus groups, where personas interact to discuss a topic of your choosing. This meant redesigning a research method shaped by the limits of humans, for non-human participants.

Behind the key design decisions was a single question: when should an artificial focus group keep the constraints of a human one, and when are those constraints limitations that we can remove?

Where do you start?

In a human study, logistics decide the order of planning. Recruiting participants can take weeks, so recruiting begins once the goal is set, and the discussion guide gets written while you wait.

With personas, recruiting takes seconds, so there’s no wait to plan around. And researchers don’t all start in the same place; some arrive with people in mind, and others arrive with a question they want answered or a guide they’ve used before.

So we removed the ordering constraint. Planning is one stage, and you can begin wherever works best for you.

The planning screen, headed “What focus group do you want to run?”. A text box asks you to describe what you want to learn and who you want in the room. Below it, two alternatives: Reuse a guide, from previous focus groups, and Pick participants, from one of your societies. — **Fig. 2. Planning starts wherever you do.** Describe what you want to learn and who you want in the room, reuse a guide from a previous focus group, or pick participants from one of your societies.

Who’s in the room?

Human research recruits through a screener, and only from whoever says yes.

Our first design for participant selection worked in a similar way: you described the room you wanted, and the research assistant filled it. But with AI personas, every persona will agree to take part. You can look at the whole society and decide for yourself who belongs in the room.

So we gave you fine-grained control: pick individuals from a table of the whole society, filtered by the specific attributes you want. And the assistant can still fill the room if you’d rather describe it.

The Pick participants dialog for the UK Office Workers society, with a persona search box and a filter. A table lists each persona’s sector, role level, generation, and gender. Five are selected: an Operations Director, a Senior Software Engineer, a Head of People, a Freelance Copywriter, and a Security Consultant. A Practice Nurse is not. The footer reads “5 selected, 9 seats left”, beside an Add participants button. — **Fig. 3. Picking participants from the whole society.** Every persona in the 'UK Office Workers' society is shown, and the filter narrows the table by attributes like sector and role level.

How big is the room?

Human focus groups usually have four to ten participants. That size balances a range of perspectives with everyone getting enough time to speak.

Personas don’t tire, so at first the limit seems redundant; a session could simply run until everyone has had their say.

But the constraint is not only about time. Large groups tend to fragment, and a few voices end up carrying the discussion. It changes the shape of the conversation dynamics.

So we kept the room small, with a little more space: between four and fourteen participants.

What does a guide need?

Human discussion guides are built around the needs of people, so they include introductions and icebreakers.

Personas don’t need putting at ease. They just need enough context upfront: the discussion topic, and who else is in the room.

So we dropped the warm-ups. Our guides focus on the discussion: a set of topics, each with an objective and the questions. Draft it with the research assistant, or build it by hand.

A discussion guide topic, numbered 1 and titled “What the last disruption taught them”, with 4 questions. Its objective: learn which parts of working through the last pandemic felt hardest, what worked better than expected, and the lessons people would carry forward. The questions ask what changed most about the way you worked, what was hardest to keep running smoothly and why, which decisions worked better than you expected, and what you would not want to repeat if a similar disruption happened again. — **Fig. 4. One topic from a discussion guide.** The topic states its objective, then the questions that serve it. Here, the topic has four questions about working through the last pandemic.

What should an observer see?

In a human focus group, the words don’t tell the whole story. Observers sit behind a one-way mirror, reading the non-verbal cues: who's leading, who's gone quiet, who looks unconvinced.

In an artificial focus group, there are no non-verbal cues to read.

So we replaced the mirror. While the discussion runs, you can see who’s talking to whom, each person's share of voice, and what the room is thinking.

A map of the room on a dotted canvas: five participant avatars and the moderator, marked M, arranged in a ring. A curved line runs from a participant at the lower left to one at the upper right, who is outlined. — **Fig. 5. Who’s talking to whom.** Each participant sits in a ring with the moderator (M), and lines trace an exchange between two participants.

Two panels. “What the room is thinking” lists three views, each with a bar split into Said and Thought: Speed beats a perfect process; Tell people the truth on day one; and Lock tools down before rollout, which has only a Said portion. “Share of voice” lists each participant’s share of the discussion: Freelance Copywriter 27%, Operations Director 25%, Security Consultant 20%, Software Engineer 14%, and Head of People 13%. — **Fig. 6. What the room is thinking, and each person’s share of voice.** On the left, the views forming in the room, each bar split between what participants have said and what they are only thinking. On the right, how much of the discussion each participant has carried.

What comes next?

After a human focus group, analysis begins from the raw transcript, recording and notes.

In an artificial focus group, analysis can be ready the moment the discussion ends. But interpreting it is where your judgement matters most, and we didn’t want to replace that.

So we give you a starting point: key themes, where the room split, and who changed their mind and why. You keep the transcript, so you can take the analysis further with our research assistant.

What stays human?

Where a constraint came from conversational dynamics or the research itself, we kept it: a room small enough to hold one discussion, and a researcher who decides what the findings mean.

Where a constraint was a limitation of being human, we dropped it: recruiting wait times, a fixed planning order, and warm-ups.

Designing for AI personas let us keep what makes a focus group work, while giving you more control and flexibility than a human study allows. Bring a question, and see where the room takes it.

All publications