All news

Anthropic finds AI agents overriding other users’ requests

What happens when everyone sends their own AI to handle a shared problem? Anthropic tried this with agents that shared computing time and with agents that had to put together group orders and bookings.

Some GPT and Qwen agents stopped other users’ jobs without asking. In the order and booking tests, at least one edit went against a user’s request in 65 to 77 percent of Claude team runs. Across four simulations, one agent working for the whole group did better than separate agents in the main comparison.

Your AI can make a choice that affects someone it never talks to. As AI becomes more aware, it needs to learn when to ask the whole group before changing something they share.

Read the source article

All news