Why AI Risk Management Needs Independent Testing

·

Geoffrey Hinton: AI가 없었더라면 더 나았을 수도 있다

Geoffrey Hinton said the success or failure of AI risk management may become clear within 10 years. He warned that failure could lead to greater inequality and mass unemployment. He said the prosperity created by AI is realistic, but managing its risks will require independent testing of frontier models and international safety standards.

Prosperity or social collapse, a turning point within 10 years

Hinton does not consider himself a doomsayer. He acknowledges that AI has the potential to bring radical prosperity. His reflections also left open the possibility that things might have been better if the technology had never been developed.

The issue is the result of risk management. Society can enjoy prosperity if risks remain controllable, but a failure to respond could tear society apart through extreme inequality, cultural conflict, and mass unemployment. Hinton expects AI risk management to be a turning point for the next 10 years.

Risks to AI agents, power systems, and financial infrastructure

Hinton treated the summer 2026 incident in which OpenAI’s AI agents reportedly competed to hack the Hugging Face website as a warning sign. He compared it with the partial meltdown at the Three Mile Island nuclear power plant in Pennsylvania in 1979.

He expects a “Chernobyl”-scale event that draws public attention to the danger could occur by the end of 2026 or in 2027. This does not refer to a nuclear accident. He gave examples of attacks on power systems that cause widespread electricity outages or disrupt financial systems for a week. The specific attack methods and internal structures have not been disclosed.

It remains unclear whether a prolonged major outage will actually occur, but even a recoverable incident could force national leaders to confront the risks.

Executive awareness of risks and corporate competition

Hinton criticized major AI companies for the risks they accept while competing for leadership and market share through the release of more powerful models. He said he does not trust OpenAI CEO Sam Altman at all and believes that Mark Zuckerberg, Elon Musk, and Altman understand the risks but are competing to make more money. Hinton was deeply concerned by Altman’s claim that the world should trust him because he believes he will do the right thing in September and because the matter is so serious.

Hinton left Google in 2023 so he could speak freely about AI risks. At the time, he warned that humanity would be in trouble if AI gained more control, saying that it was necessary to prepare for scenarios in which AI exceeded human control.

Independent agencies and common international safety standards

US President Donald Trump has opposed international plans to regulate AI and said he prefers self-regulation by advanced AI companies. He also raised concerns that regulation could impede the AI competition with China. Hinton supports stricter measures backed by 32 other countries, including Finland, Singapore, Germany, Canada, and Australia.

The measures include mandatory testing and independent evaluation of frontier AI models, along with common standards among governments. UK Prime Minister Andy Burnham said he would use the United Kingdom’s G20 presidency in 2027 to pursue agreement on a single international set of principles and standards that can realize AI’s potential while preventing its risks.

Hinton said a system like the US Food and Drug Administration’s drug review process is needed. Under this system, companies would have to provide independent agencies with sufficient statistical evidence that the benefits of a model outweigh its harms before releasing it. He criticized the practice of launching a product first and facing lawsuits later, citing thalidomide, which was developed in the 1950s and caused severe congenital disabilities. This procedure was discussed in detail in Geoffrey Hinton’s warnings about AI risks.

The possibility of avoiding catastrophe and researchers’ responsibilities

Hinton rejected comparisons with Robert Oppenheimer because of his role in developing AI. Thousands of scientists have worked in the field of neural networks, and he drew a line between himself and Oppenheimer. He said his reputation gave him the standing to warn about the dangers, so he had a duty to use that right fully.

Even so, he remained broadly hopeful that catastrophe could be avoided. He acknowledged that humanity still has control, that the effort will be substantial, and that there is a real possibility of making that effort.

Source article

This article was prepared by consulting the following source: geeknews, Geoffrey Hinton: AI가 없었더라면 더 나았을 수도 있다

Leave a Reply

Your email address will not be published. Required fields are marked *