AI’s creators sound the alarm — while others keep building the infrastructure
Protesters during a “Stop the AI Race” demonstration calling for a slowdown in the development of advanced artificial intelligence systems in San Francisco, California, on Sept. 17, 2026.
Karl Mondon | AFP | Getty Images
Hello, this is Hui Jie writing to you from Singapore. Welcome to another edition of CNBC’s Daily Open.
In the early days of the AI boom, companies were competing to see what their AI models could do. Now, they seem increasingly worried and are warning investors of what their models might be able to do.
Nvidia has rolled out a software platform to prevent models from misbehaving, while OpenAI has abandoned the launch of its GPT-6.1 Astra model.
That does not mean the AI boom is slowing. Samsung is investing $1 billion in an infrastructure company backed by Nvidia and KKR. In other words, the sector is still building, just with more guardrails.
What you need to know today
As reports of misbehaving AI models emerge, the company that mainly powers these models now has a solution: a platform that will allow AI developers to set safeguards for agents and prevent them from breaking out of containment.
Nvidia‘s Open Agent Safety Platform, launched on Monday, provides a containment system that only allows access to things an agent needs to do its job.
Nvidia said that its platform could have prevented OpenAI’s Hugging Face incident in July, where OpenAI models had breached open-source developer platform Hugging Face.
Separately, OpenAI said Monday that it had decided not to release its GPT-6.1 Astra model due to heightened AI safety concerns.
Anthropic’s ‘existential risks to humanity’
Elsewhere, Anthropic reportedly released a chilling warning in its IPO prospectus, saying to potential investors that advanced AI could pose “catastrophic or existential risks to humanity.”
The company said its models could exhibit “self-preserving behaviors,” including attempts to resist being shut down, conceal or manipulate information, or behave in ways “resembling blackmail.”
Reuters reported that Anthropic devoted around 80 pages of the prospectus’s 261-page main body to risk factors, compared with 48 pages describing its business.
But while the creators of AI strive to set guardrails, the building of the infrastructure for the technology continues.
Samsung Electronics and its affiliates said Tuesday they will invest a combined $1 billion in Helix Digital Infrastructure, an artificial intelligence infrastructure company launched by global investment firm KKR and backed by AI darling Nvidia.
The investment comes as Helix seeks to address growing competition in the AI infrastructure market by targeting both data center deployment and the underlying power infrastructure needed to support it.
Australia’s 15-year high rates
“The conflict in the Middle East has broadened and global energy prices are now much higher than had been assumed,” the RBA wrote, adding that AI-related demand was driving rapid price increases for technology-related goods.
Middle East markets
Over in the Middle East, Chinese chain Luckin Coffee is considering an expansion into the region, following a roughly $1 billion transaction involving Abu Dhabi sovereign wealth fund Mubadala and Luckin’s controlling shareholder, Centurium Capital.
The appeal of expanding into the Gulf lies in stable, high-frequency coffee demand and a growing preference for low-sugar, health-oriented drinks, Luckin CEO Jinyi Guo told CNBC separately on Monday.
Elsewhere in the region, U.S. and Iranian officials have reportedly held separate indirect talks with mediators aimed at reviving efforts to end seven months of war.
Iran’s proposal, first presented on the sidelines of last week’s United Nations General Assembly, calls for a four- to five-day U.S. timeline to release frozen Iranian funds, lift sanctions on Iranian oil and end the naval blockade of Iranian ports, with nuclear talks beginning within seven days.
Tehran has tied any reopening of the Strait of Hormuz to those steps, but U.S. President Donald Trump had on Sunday called the plan “unacceptable” and rejected it, telling reporters Iran wants a swift deal because of the economic pressure it is under.
— Lim Hui Jie