Why Plenums Lab Makes AI Agents Debate Before Settling on an Answer
Getting a second opinion on an idea usually means assembling other people. A focus group. A team meeting. A few friends willing to argue it out. That kind of review takes time to arrange and often costs money, so plenty of everyday decisions get made on the strength of a single person's judgment or a single AI model's answer.
Evan Khanna is a Computer Science student at James Madison University and the creator of Plenums Lab, a platform where independent AI agents debate a question across different models before a verdict comes back to the person who asked it. The idea grew out of a personal frustration. He wanted an instant focus group for his own decisions, without hiring anyone or waiting for friends to be free.
In this episode of Lead with AI, Dr. Tamara Nall speaks with Khanna about how the multi-model approach works, how a high school teacher used it to help settle a staffing decision, and why he built in guardrails before he ever let it answer a question that mattered.
What Is Plenums Lab, and Why Call It That?
The idea behind Plenums Lab started as a personal frustration for Khanna. He wanted a focus group for his own ideas and decisions, something immediate, not a meeting he had to schedule or a team he had to hire. Out of that frustration came a theory he is now researching formally, which he calls the multi-model approach. Put several AI agents, built on different underlying models, to work on the same question, and the result is not necessarily a better answer. It is often a more accurate one, shaped by more perspectives than a single model could offer on its own.
The working title for the project was AI Debate Lab. Khanna settled on Plenums Lab instead because a plenum is an assembly of different people, and that is exactly what the platform recreates, except the assembly is made of AI agents instead of people. The platform is already deployed. Anyone who wants to try it can join the waitlist at plenumslab.com, where Khanna sends out access codes directly.
Two Decisions Settled by a Panel of AI Agents
The clearest example in the episode involves Khanna's former business teacher, whom he calls Miss Lewis. She was trying to decide whether to let an employee go from the school store she runs or move that person to a different role they might be better suited for. She set up three agents on three different models, by Khanna's recollection, one running Claude, and the other two split between ChatGPT and a Chinese-built model, and ran her question through them. Plenums Lab exported a time-stamped PDF of the full exchange, including the topic, the agents, the arguments each one made, and the factors each one weighed.
What stood out to Miss Lewis was not the debate itself. It was the ending. The judge agent did not hand her a list of perspectives to sort through on her own. It delivered a specific, decisive answer, one she could still trace back through the reasoning if she wanted to check it.
Khanna has used the same setup on himself. As part of Microsoft's Copilot Ambassador program at James Madison University, he had two ideas for an upcoming campus event, a coffee giveaway and a tabling event with branded merchandise and gift cards. He ran both past three agents, Claude, a Chinese-built model he calls Qwen, and ChatGPT, and assigned one agent to argue in bad faith, pushing back on every idea it was given. The coffee giveaway lost on cost and setup. The tabling event won on simplicity and on being easy for the campus agency to fund.
One API Call Turns Into a Debate
Khanna says the whole project started by accident. He made a standard API call to ChatGPT, got a response back, then on a whim fed that response into another model, Claude. The output chained into the next call, and the next, until it started to resemble an argument.
He had built a debate without setting out to build one.
From there, Khanna slowed the process down on purpose. He added a moderator to keep the exchange on track and a judge to close it out with a verdict. What makes the system work, in his telling, is less the chain of calls itself and more what he calls the harness, a set of prompts he writes for every debate, instructing each agent how to argue based on what the user actually asked. That harness, not the raw model calls, is what turns a sequence of API requests into something that reads like a real debate.
Choosing How Deep the Debate Goes
Plenums Lab is built around flexibility rather than one fixed setup. A debate can run on cheaper, faster models for a quick read on a decision, or on more capable ones that take longer and dig deeper. Users who are not sure which models fit their questions can let Khanna's system choose for them.
The scale is adjustable too. A debate can involve as few as two or three agents, or, as Khanna has tested himself, twenty to thirty at once. The tradeoff is cost. The business model he is building toward charges by token, so a larger debate costs more to run. Depending on the complexity of the question and the models chosen, a full debate takes up to around fifteen minutes, start to finish.
Guardrails Before the Debate Starts
Khanna built an ethics framework into Plenums Lab before he built much else, in a file he refers to as his ethics.md. If a user asks something health-related, the underlying models will usually add their own disclaimer on their own. Khanna adds a second layer on top of that through his own system prompt, so the disclaimer is guaranteed rather than left to chance.
The same philosophy carries through to how the results are framed. Every debate closes with a reminder that the verdict is one opinion to weigh against others, not a final answer to accept on faith. Khanna wants users to rerun the debate, bring in a different agent, and apply their own judgment before acting on anything it produces. Miss Lewis did exactly that. She read through the reasoning herself rather than taking the verdict at face value, and judged that it held up.
Where Plenums Lab Goes From Here
Khanna sees the current version of Plenums Lab as a foundation rather than a finished product. The same harness that debates a staffing decision or a campus event could be narrowed into something more specific, such as a tool that debates whether a business decision meets compliance requirements or an inventory app that argues out what a company should reorder. He is also open to licensing the harness itself, so another company could build it into their own internal tools.
Alongside the product, Khanna is researching why AI models disagree with each other in the first place and what that disagreement reveals about alignment and AI safety. For now, the way in is the waitlist at plenumslab.com, where he is still reviewing requests and handing out access himself.
For students, teams, and anyone who wants to stress-test a decision before acting on it, Plenums Lab is open through its waitlist at plenumslab.com.
For more conversations with the founders and researchers building practical AI tools, subscribe to Lead with AI on your preferred podcast platform.
Quick Answers
What is Plenums Lab?
Plenums Lab is a platform where independent AI agents, built on different models, debate a question while an AI moderator and judge deliver a decision.
How does a debate actually work?
A user submits a question or a decision to weigh. Multiple AI agents argue different sides of it, a moderator keeps the exchange on track, and a judge reviews the arguments and delivers a specific verdict, along with a time-stamped PDF of the full exchange.
Can I choose which AI models to debate?
Yes. Users can select specific models, run as many as twenty to thirty agents in one debate, or let the system choose a faster, cheaper set of models for a quicker read.
Does Plenums Lab just list opinions, or does it decide?
Both. The agents present different arguments, but the judge agent still delivers a decisive answer rather than leaving the perspectives for the user to sort through alone.
Who is Evan Khanna?
Evan Khanna is a Computer Science student at James Madison University and the creator, owner, and researcher behind Plenums Lab.
Follow or Subscribe to Lead with AI Podcast on your favorite platforms:
Website: LeadwithAIPodcast.com | Apple Podcasts: Lead with AI | Spotify: Lead with AI | YouTube: @LeadwithAIPodcast | Facebook: Lead with AI | Instagram: @LeadwithAIpodcast | TikTok: @LeadwithAIpodcast | Twitter (X): @LeadwithAI
Follow Dr. Tamara Nall:
LinkedIn: Tamara Nall | Website: TamaraNall.com

Comments