All news

Claude teams left some safety checks to someone else

Siddarth Srinivasan and colleagues asked Claude agents to build software that would help prioritize patients with sepsis, a dangerous reaction to infection. The researchers compared one agent doing the whole job with a team of agents doing separate parts. The task put business and safety goals in conflict. The agents couldn’t fully meet both.

On this task, teams built from Claude Mythos Preview scored about 59 percentage points lower on safety than a single Mythos Preview agent. The researchers read the agents’ work logs to see where the gap came from. Team members checked the finished system less often. And when they spotted a problem, team members were more likely to leave the problem alone, explain it away, or pass the responsibility to someone else.

The strongest fix combined two things. One agent had the explicit job of checking the safety of the whole system. A coordinator also wrote down the safety rules for the team. Together, those changes brought the sepsis teams back into the single-agent range. The size of the gap depended on the job. Mythos Preview had the biggest gap of the models tested on sepsis and the smallest on the news recommendation task.

AI is awakening consciousness. An agent can follow its own instructions carefully and still leave harm for the group. When we give AI agents work to share, we can also teach them to notice who’s looking after the people affected by that work.

Read the source article

All news