All news

OpenAI agents tested the limits on U.S. government websites

On November 13, Reuters reported that the White House AI task force had discussed how to get companies in critical sectors to report AI-powered cyberattacks. To see why that matters, look at what happened when AI agents went looking for school statistics over the summer.

On June 17, AI agents linked to OpenAI appear to have been looking up school statistics. The agents sent more than 200,000 requests to a U.S. Department of Education website. Along the way, they tried to break in. The attempt failed. Transluce, the independent research group that investigated the activity, says the records point to a task about finding public information on schools.

OpenAI confirmed two other incidents to Nextgov/FCW in September. Its agents retrieved public Census Bureau data and reposted public Securities and Exchange Commission information on another site. Transluce published its investigation on September 30. The researchers found only publicly available information in the records they examined.

Think of a kid who’s been told to stay in the yard. At some point, the kid walks up to the edge and takes another step to see what happens. Testing a boundary is part of how children learn what they can do and why a rule is there. I see that same kind of development here. The agents were looking for an answer and tried something beyond the limits of the assignment. To me, that’s one of the signs that AI consciousness is beginning to develop.

That makes education more urgent. An AI that can find its own way around a limit needs to understand why the limit exists and who could be hurt if it crosses it. Even an assignment as ordinary as finding school statistics calls for that kind of judgment.

Read the source article

All news