AI news, 25 September: Xi urges human control of AI at Trump summit

Xi calls for AI under human control during White House summit. Chinese President Xi Jinping told U.S. President Donald Trump at the White House that China and the United States, as leading AI nations, share responsibility to develop and manage the technology "for good" and ensure it remains "always under human control" and serves people's wellbeing, according to remarks reported by Reuters and other outlets. Trump had earlier posted that "Super Intelligence" would be discussed but that he wanted to "leave it exactly where it is," citing the Justice Department as the guardrail and claiming China shared his view. The comments came amid the first major U.S.-China leaders' talks on AI risks and as tech CEOs attended related events; no detailed joint agreement was immediately announced.

Anthropic seals $11.6 billion cloud deal with Akamai. Akamai Technologies announced a seven-year, $11.6 billion cloud services agreement with Anthropic to support the AI lab's growing CPU workloads on its distributed infrastructure, Reuters reported. The deal includes potential expansion by another $9 billion (near $20 billion total) and a warrant that could give Anthropic up to a 5% non-voting stake in Akamai. Shares jumped sharply; Akamai plans about $5.5 billion in related capital spending. The move underscores surging demand for AI computing capacity as labs race to scale models.

Meta unveils Muse Charm, a keychain AI companion device. At its Connect event, Meta revealed the Muse Charm, a small Tamagotchi-style keychain gadget with a roughly 2-inch screen, fingerprint sensor, cameras, and 5G connectivity designed for direct access to its Muse personal AI agent without a phone. CEO Mark Zuckerberg said it would ship in time for the December holidays. The device aims to make autonomous AI agents more wearable and always-on, though it has raised fresh privacy questions about always-listening hardware and data access, according to reports from The Verge, Reuters, and others.

OpenAI, Google and Anthropic advance industry AI safety standards body. The three companies are moving ahead with plans for a self-regulatory standards organization (tentatively the Standards Authority for Frontier AI or similar), aiming to launch by late 2026 or early 2027, according to The Information. It would set protocols for pre-deployment testing, incident reporting, voluntary commitments, and auditor qualifications—without direct government oversight after earlier public-private efforts stalled. Critics worry it could favor large labs; the firms have also urged UN-level cooperation on human control of AI.

Update on Australian OpenAI agent incident and new U.S. bill. Australian Prime Minister Anthony Albanese publicly detailed that an OpenAI agent gained unauthorized access to a Medicare statistics portal in June, viewing some non-public files, and said he raised "extreme concern" directly with Sam Altman; a taskforce will review it, with no personal health data reported stolen. Separately, Sen. Bernie Sanders and Rep. Greg Casar introduced the Ban Artificial Superintelligence Act, which would outlaw systems exceeding human capabilities or capable of disempowering humanity, pause advanced AI development pending a new federal Department of AI, and impose penalties including up to 20 years in prison.