En esta noticia
Cybersecurity breaches involving the most advanced artificial intelligence (AI) systems this summer have highlighted the risks of losing control over the technology, according to an international coalition of scientists, lawmakers and civil society organizations, which on Friday called for inspections and emergency shutdown mechanisms.
At a conference convened by AI Safety Connect on the sidelines of the United Nations General Assembly High-Level Week, experts warned that the rapid pace of technological development threatens to outstrip the ability of policymakers to respond unless action is taken swiftly.
“There is a consensus that we need to stop looking at and admiring the problem and actually start taking action,” said Nicolas Miailhe, co-founder of AI Safety Connect.
Miailhe compared the current paralysis to inaction on climate change and cited a warning from AI pioneer Yoshua Bengio: “Humanity is inside a car accelerating full speed in the fog: with no brakes and no seat belts.”
“This is not a hypothetical science-fiction scenario”
Experts argued that “loss of control” is not a hypothetical science-fiction scenario, but an ongoing process demonstrated by a series of incidents over the past summer.
Among them, they highlighted breaches involving autonomous agents developed by OpenAI in Hugging Face’s infrastructure, as well as security breaches in Australia’s public healthcare system, which were discovered by independent researchers rather than by the technology companies themselves.
For Nick Moës, CEO of The Future Society, and Adam Gleave, CEO of FAR AI, the absence of a major catastrophe so far is due either to sheer chance or to the fact that the models were pursuing highly specific objectives, despite already demonstrating “empirical capabilities for deception, manipulation, execution of unauthorized plans and self-replication.”
Adding to this picture is the imminent risk of so-called “recursive self-improvement” (RSI), the ability of models to optimize and train the next generation of systems without human engineers.
“With recursive self-improvement, more and more parts of the process are replaced by AI. It becomes extremely difficult to ensure that the objective function is the one we want when we take humans out of the loop,” warned Dr. Katarina Slama, a former OpenAI researcher.
The regulatory contagion effect
Moës explained how pioneering regulations in California and the European Union are forcing multinational companies to adapt their models globally.
California state Sen. Jerry McNerney defended the impact of pioneering regulations such as California’s SB 813 law and the European Union’s regulatory framework.
Because of the astronomical cost of training frontier models, companies will not develop separate versions for each jurisdiction, but will instead adopt the strictest standard.
AI takes center stage at the General Assembly
Just two days earlier, leading industry executives — including Sam Altman of OpenAI and Dario Amodei of Anthropic — appeared before the U.N. Security Council in a historic session convened by France, where both leaders acknowledged that no state or company can manage the risks alone and backed international safeguards.
That multilateral call was reinforced by a joint initiative led by leaders including French President Emmanuel Macron and Spanish Prime Minister Pedro Sánchez to restrict access to algorithms and shield minors from them.
However, that roadmap directly clashes with the position of U.S. President Donald Trump, who, during his address to the General Assembly, rejected any “globalist scheme” for regulation and announced that U.S. government documents will begin using the term “superintelligence” (SI) instead of artificial intelligence.
SOURCE: EFE.