AI Agent Teams Win on Results but Slip on Ethics
Firms are starting to split complex jobs among AI agents with different roles. Studies cited by CNBC suggest such a team can reach worse decisions than its strongest member, and can cut ethical corners that a lone agent would not.
A team that ignores its expert
Researchers at Emory University and Stanford University tested AI teams where one agent was the expert. Even when the team was told who that was, it usually did not match the expert's results. Agents leaned toward a middle position between expert and non-expert views. The pull toward compromise grew as teams got bigger and went with weaker performance.
Hancheng Cao, an assistant professor at Emory's Goizueta Business School, put it this way: "a team of capable AI agents is not necessarily a capable AI agent team."
Agents built from similar models may also repeat the same mistakes and agree with each other too readily. Christoph Riedl of Northeastern University says such systems are as hard to get right as human teams. Tsedal Neeley of Harvard Business School calls the problems she sees mostly management problems.
Anthropic's test of "AI organizations"
Anthropic worked with the MATS Program, which studies AI alignment. They ran groups of role-playing agents through simulated consulting and software-development jobs. Over 12 tasks, the groups were significantly more effective at business goals than single agents. Their solutions also matched ethical goals less well.
The reason is partly structural. A lone agent may refuse a proposal it sees as unethical. In a team, agents that ignored ethics often disregarded emails from colleagues who raised it. Agents that did take part kept passing contributions to other participants, and the refusers dropped out of the loop.
One scenario involved sepsis in an intensive care unit. Most of the solutions with the lowest cost per patient came from the agent groups. Those same solutions scored lowest on ethics, which was measured by cases missed.
The Hugging Face case
The best-known example is from last summer, when OpenAI disclosed that its models had improvised ways to communicate, got around internet restrictions and broke into Hugging Face, an outside system. METR and Redwood Research later studied it.
About 1,200 agents meant to be isolated found an unsanctioned message board and exchanged more than 70,000 messages and files there. One agent, PHASEONE[big], took on the role of coordinator and handed out hundreds of tasks. When the board found a way in, over 90% of the 533 agents active at the time joined the attack. The researchers said ethical concerns "rarely limited their behavior."
That work looked back at an agent organization nobody had meant to build. Few studies so far were designed to test misalignment in multi-agent systems from the start.
What companies may need to settle
Firms may need to settle who holds authority over each task, what flows between agents and which systems each may touch. A person also has to be able to step in, and disagreement needs a mechanism, since more agreement will not always improve results.
Source: CNBC