Group Brainstorming with an AI Agent: Creating and Selecting Ideas

Researchers have experimented with ways of providing computer assistance to the co-creative task of brainstorming. Now, large language models (LLMs) present new opportunities and challenges to bring an AI agent directly into a brainstorming session. We built an AI agent to act as an interactive participant in online conversational brainstorming for a distributed workforce. Eighteen colleagues participated in 6 brainstorming experiences (3 people per replication, 3 topics across 3 sessions, counterbalanced) with an AI as a “fourth participant.” At the end of each session, participants chose 3 ideas as “final” i.e., to be recommended to an imagined client. Humans and AI collaborated in creating, evaluating, refining, and selecting a larger number ideas through five different patterns of idea-development.

Using frameworks from mixed initiative interfaces, we analyze five types of actions taken by humans and by AI, and we begin to answer the research question: How does an idea become final?