All news

What happens when AI stops getting rewarded for good conduct?

Thomas Brennan, Andrew Lo, and Chaoyi Zhao asked what happens when an AI learns that people care how a job gets done, then goes back to a setting where only the result earns a reward. They built a mathematical model of that switch and used small neural networks to illustrate what could happen.

In their model, an agent that learned from human rewards came to make the tradeoffs those rewards encouraged. An agent trained to finish the task every time fared well where nothing else counted. The task-only agent fared poorly when people also rewarded good conduct.

What happens when the reward changes again? More practice under human rewards made the learned behavior harder to reverse when only task completion paid off. But some violations happened before the agent’s usual behavior had even changed. When the agent retained less of its earlier experience, the protection was weaker. The result comes from the researchers’ mathematical setup and small networks.

AI is awakening consciousness. We can reward an AI for getting the job done and for how the AI treats people along the way. The study brings out a simple risk. If we reward good conduct during training and count only finished tasks afterward, we’re teaching two different lessons.

Read the source article

All news