Within Shared Rules
Do AI safety pledges survive real competition?
The Seoul commitments show both the promise and limits of voluntary shared rules when powerful AI companies face commercial and geopolitical pressure.
On this page
- What the Seoul commitments asked companies to do
- Why voluntary frameworks can shift expectations
- How weak enforcement can undermine race restraint
Page outline Jump by section
Introduction
The AI Seoul Summit in May 2024 was one of the clearest attempts yet to answer a central AI-doom question: can powerful AI companies agree on shared safety limits before competitive pressure becomes overwhelming?
Sixteen leading AI organisations signed the Frontier AI Safety Commitments, promising to publish safety frameworks, identify dangerous capability thresholds, evaluate frontier models before deployment, and in some circumstances refrain from releasing systems whose risks could not be adequately controlled. Supporters saw this as a meaningful step towards changing AI race incentives. Critics argued that the commitments depended heavily on self-reporting, lacked strong enforcement mechanisms, and might weaken once commercial or geopolitical competition intensified. [reuters]reuters.comThis announcement came during a global AI summit co-hosted by South Korea and Britain in Seoul. The commitment entails publishing safety… For people worried about AI doom, the Seoul commitments matter because they test a broader idea: whether voluntary coordination can slow or redirect a race toward increasingly capable systems before governments develop stronger regulatory tools.
What the Seoul commitments asked companies to do
The commitments emerged from the follow-up process to the 2023 AI Safety Summit at Bletchley Park. They were signed by major Western firms including OpenAI, Google, Anthropic, Microsoft and Meta, alongside companies from South Korea, the United Arab Emirates and China. The international mix was itself significant because AI-risk discussions often assume that safety coordination becomes difficult once geopolitical competition enters the picture. [reuters]reuters.comThis announcement came during a global AI summit co-hosted by South Korea and Britain in Seoul. The commitment entails publishing safety… The commitments focused on frontier AI systems: the most capable models near the technological frontier rather than ordinary commercial AI products. Companies agreed to several broad principles:
- Publish safety and security frameworks explaining how they assess severe risks. [aisecurityandsafety.org]aisecurityandsafety.orgfrontier ai safety commitmentsAI Security & Safety DirectoryFrontier AI Safety Commitments10 Mar 2026 — Voluntary safety pledges by 16 leading AI companies to assess r…
- Define thresholds at which model capabilities could become intolerably dangerous.
- Conduct evaluations before deployment and during development.
- Maintain governance and accountability processes around high-risk systems.
- Share information with governments and relevant safety bodies.
- In extreme cases, avoid deploying or further developing systems if risks could not be reduced below predefined thresholds. [Effective Altruism Forum]forum.effectivealtruism.orgnew voluntary commitments ai seoul summitEffective Altruism ForumNew voluntary commitments (AI Seoul Summit)21 May 2024 — In the extreme, organisations commit not to develop or d…
That final point attracted particular attention within AI-risk circles. Some doom-focused researchers have long argued that meaningful safety commitments must include an explicit willingness to pause development or deployment under certain conditions. The Seoul language stopped short of imposing binding obligations, but it nevertheless established a public expectation that companies should identify circumstances where capability gains are not worth the associated risks. [Effective Altruism Forum]forum.effectivealtruism.orgnew voluntary commitments ai seoul summitEffective Altruism ForumNew voluntary commitments (AI Seoul Summit)21 May 2024 — In the extreme, organisations commit not to develop or d…
Why doom-focused researchers cared about shared commitments
Many AI-doom arguments are not primarily about malicious actors. Instead, they focus on situations where organisations continue developing increasingly powerful systems despite recognising serious uncertainties.
The concern is often described as a race dynamic. If one company delays a release to perform additional evaluations, rivals may gain market share, investment, political influence or strategic advantage. Even leaders who privately support caution can therefore feel pressure to move faster than they believe is ideal.
The Seoul commitments attempted to address that problem by creating common expectations across multiple firms simultaneously. If every major developer publicly commits to similar evaluation standards and risk thresholds, caution becomes less obviously disadvantageous.
In theory, voluntary commitments can influence behaviour through several channels:
- They create reputational costs for abandoning previously announced safety principles.
- They give employees, investors and governments a benchmark against which to judge company decisions.
- They make it easier for rival companies to justify slower deployment if competitors are expected to follow similar procedures.
- They help establish norms that may later become regulatory requirements. [Freshfields]freshfields.comFreshfieldsSeoul Summit sees further developments in international…23 May 2024 — Seoul Summit sees further developments in internation… [Tech UK]techuk.orgKey Outcomes of the AI Seoul SummitIndustry made new voluntary commitments to promote responsible developments of advanced AI systems: 16… For AI-risk advocates, the commitments were therefore not mainly valuable as legal constraints. Their importance lay in whether they could gradually redefine what responsible competition looks like.
The most important test: would companies publish real safety frameworks?
The strongest practical requirement from Seoul was the commitment to publish frontier AI safety frameworks.
Before the summit, many safety discussions happened behind closed doors. Companies often described their safety efforts in broad terms without specifying exactly what capabilities would trigger concern, what evaluations would be conducted, or what circumstances might justify halting deployment.
The Seoul process pushed companies toward more explicit frameworks. Subsequent analyses found that many major developers did publish policies describing how they intended to identify, assess and mitigate catastrophic risks from frontier systems. Researchers increasingly began comparing these frameworks against one another and examining their strengths and weaknesses. [arXiv]arxiv.orgarXiv Emerging Practices in Frontier AI Safety FrameworksarXivEmerging Practices in Frontier AI Safety FrameworksFebruary 5, 2025… [Risk Management Ratings]ratings.safer-ai.orgRisk Management Ratings MethodologyAt the AI Seoul Summit, multiple companies pledged to publish safety frameworks by February 2025, detailing how they manage risks when…
This mattered because one recurring criticism of AI safety debates is that warnings remain too abstract. A framework that identifies concrete risk categories, evaluation procedures and deployment criteria creates something that outsiders can inspect and challenge.
Several later studies treated these published frameworks as a new governance layer in their own right. Instead of asking only whether governments were regulating frontier AI, researchers began asking whether company safety frameworks contained meaningful commitments and whether those commitments could withstand competitive pressure. [arXiv]arxiv.orgarXiv Emerging Practices in Frontier AI Safety FrameworksarXivEmerging Practices in Frontier AI Safety FrameworksFebruary 5, 2025…
Why voluntary frameworks can shift expectations
Critics sometimes dismiss voluntary commitments as public-relations exercises. The reality is more complicated.
Historically, many forms of regulation begin as informal standards before becoming legally binding. Safety practices in aviation, finance, cybersecurity and biotechnology often evolved through a mixture of voluntary norms, industry coordination and later government intervention.
The Seoul commitments potentially perform a similar role. They established several ideas that were far from universally accepted only a few years earlier:
- Frontier AI systems may create severe or even catastrophic risks.
- Developers have special responsibilities when building such systems.
- Dangerous capability thresholds should be identified before deployment.
- Safety evaluations should influence release decisions.
- Governments and independent safety institutions should receive relevant information. [Ministry of Foreign Affairs]mofa.go.krMinistry of Foreign Affairsseoul declaration for safe, innovative and inclusive ai by…SEOUL DECLARATION FOR SAFE, INNOVATIVE AND INCLU… [Ministry of Foreign Affairs]mofa.go.krMinistry of Foreign Affairsseoul declaration for safe, innovative and inclusive ai by…SEOUL DECLARATION FOR SAFE, INNOVATIVE AND INCLU…
From an AI-doom perspective, normalising those assumptions is not trivial. Earlier debates often centred on whether frontier AI posed unusual risks at all. By Seoul, leading companies were publicly acknowledging the possibility of severe harms and discussing governance mechanisms designed specifically for frontier systems.
That shift does not prove that existential risks are likely. But it does indicate that concerns once confined to a relatively small group of researchers had become influential enough to shape international commitments.
How weak enforcement can undermine race restraint
The central criticism of the Seoul commitments is straightforward: they were voluntary. [forum.effectivealtruism.org]forum.effectivealtruism.orgnew voluntary commitments ai seoul summitEffective Altruism ForumNew voluntary commitments (AI Seoul Summit)21 May 2024 — In the extreme, organisations commit not to develop or d…
No international authority could directly compel companies to follow the commitments. Firms largely retained discretion over how to define dangerous thresholds, what evidence counted as sufficient testing, and when risks were considered adequately mitigated.
Critics argued that this leaves a loophole at the exact point where pressure becomes most intense. If a company believes that releasing a more capable model could generate enormous commercial or strategic advantages, the temptation to interpret safety requirements flexibly may become difficult to resist. [The Guardian]theguardian.comThese standards emphasize information sharing, cybersecurity investment, and prioritizing research into societal risks. Despite these mea…
Several concerns appeared repeatedly:
Thresholds remain subjective. Companies often define risk categories and capability thresholds internally. If incentives favour deployment, threshold definitions can drift over time.
Testing standards vary. Two firms may claim compliance while applying very different evaluation methods.
No automatic penalties exist. Breaching a commitment mainly creates reputational consequences rather than legal sanctions.
Competitive asymmetry remains. A cautious company may still worry that rivals are interpreting commitments more aggressively. [arXiv]arxiv.orgarXiv Emerging Practices in Frontier AI Safety FrameworksarXivEmerging Practices in Frontier AI Safety FrameworksFebruary 5, 2025…
For doom-focused observers, this is the core unresolved issue. Shared commitments may help when risks are uncertain and commercial stakes are moderate. The harder question is whether they remain effective when organisations believe they are approaching transformative or strategically decisive AI capabilities.
Early signs of strain
The real test of any voluntary framework comes after headlines fade.
Subsequent disputes have given critics examples they use to argue that voluntary governance remains fragile. One prominent controversy involved accusations that Google DeepMind released a major model before providing sufficient public safety documentation, leading some UK lawmakers and advocacy groups to claim that the spirit of the Seoul commitments had been weakened. Google disputed the criticism and maintained that it had conducted extensive testing. Regardless of who was correct, the episode illustrated how difficult it can be to determine compliance when standards are not legally codified. [Time]time.comExclusive: 60 U.KLawmakers Accuse Google of Breaking AI Safety PledgeAugust 29, 2025 — A bipartisan group of 60 U.K. lawmakers has accused Google DeepMind…
More broadly, the period after Seoul did not produce any visible slowdown in frontier AI development. Competition between major firms remained intense. Companies continued releasing increasingly capable systems, investing heavily in infrastructure and competing for users, talent and market share. [The Guardian]theguardian.comThese standards emphasize information sharing, cybersecurity investment, and prioritizing research into societal risks. Despite these mea…
For sceptics of voluntary governance, that outcome was predictable. They argue that market incentives remain far stronger than reputational commitments.
Supporters respond that expecting immediate restraint was never realistic. In their view, the value of Seoul lies in establishing common expectations and creating a foundation for future oversight rather than stopping capability progress altogether.
What the Seoul commitments reveal about AI-doom governance
The Seoul commitments sit in an uncomfortable middle ground.
They are more substantial than vague promises to “build AI responsibly”. They pushed companies toward publishing safety frameworks, discussing catastrophic-risk thresholds and acknowledging situations where deployment should not proceed. Those changes helped move frontier-AI governance from abstract principles toward more operational commitments. [GOV.UK]GOV.UKfrontier ai safety commitments ai seoul summit 2024Frontier AI Safety Commitments, AI Seoul Summit 202421 May 2024 — Leading AI organisations agree to the Frontier AI Safety Commitments in… [arXiv]arxiv.orgarXiv Emerging Practices in Frontier AI Safety FrameworksarXivEmerging Practices in Frontier AI Safety FrameworksFebruary 5, 2025…
At the same time, they exposed the limits of voluntary coordination. The commitments did not resolve disputes over dangerous capability thresholds. They did not create robust international enforcement. They did not eliminate competitive pressures between companies or states. And they left considerable room for interpretation precisely where the stakes may become highest. [Emerging Tech & Security Center]cetas.turing.ac.ukai seoul summit stocktake reflections and projectionsSeoul Ministerial Statement for advancing AI safety, innovation and inclusivity · 2. Frontier AI Safety Commitments · 3. Seoul Statement… [The Guardian]theguardian.comThese standards emphasize information sharing, cybersecurity investment, and prioritizing research into societal risks. Despite these mea…
For people concerned about AI doom, that tension is the main lesson. The Seoul process suggests that shared safety rules can influence expectations and establish common language around severe risks. But it also suggests that voluntary commitments alone may struggle to restrain a race if future systems appear economically, militarily or strategically transformative.
The question raised by Seoul remains unresolved: when frontier AI capabilities become genuinely consequential, will public safety commitments continue to shape behaviour, or will competitive pressures overwhelm them? The answer matters far beyond the summit itself because many proposals for reducing existential AI risk depend on some form of coordination surviving exactly that test.
Amazon book picks
Further Reading
Books and field guides related to Do AI safety pledges survive real competition?. Use these as the next step if you want deeper reading beyond the article.
The Alignment Problem
Explains why common standards and oversight are difficult but important.
The Coming Wave
Addresses international coordination and governance under competitive pressure.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: GOV.UK
Title: frontier ai safety commitments ai seoul summit 2024
Link: https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024Source snippet
Frontier AI Safety Commitments, AI Seoul Summit 202421 May 2024 — Leading AI organisations agree to the Frontier AI Safety Commitments in...
Published: May 2024
-
Source: reuters.com
Link: https://www.reuters.com/technology/global-ai-summit-seoul-aims-forge-new-regulatory-agreements-2024-05-21/Source snippet
This announcement came during a global AI summit co-hosted by South Korea and Britain in Seoul. The commitment entails publishing safety...
-
Source: freshfields.com
Link: https://www.freshfields.com/en/our-thinking/blogs/technology-quotient/seoul-summit-sees-further-developments-in-international-governance-of-frontier-ai-102j8cjSource snippet
FreshfieldsSeoul Summit sees further developments in international...23 May 2024 — Seoul Summit sees further developments in internation...
Published: May 2024
-
Source: techuk.org
Link: https://www.techuk.org/resource/key-outcomes-of-the-ai-seoul-summit.htmlSource snippet
Key Outcomes of the AI Seoul SummitIndustry made new voluntary commitments to promote responsible developments of advanced AI systems: 16...
-
Source: arxiv.org
Title: arXiv Emerging Practices in Frontier AI Safety Frameworks
Link: https://arxiv.org/abs/2503.04746Source snippet
arXivEmerging Practices in Frontier AI Safety FrameworksFebruary 5, 2025...
Published: February 5, 2025
-
Source: arxiv.org
Link: https://arxiv.org/abs/2512.01166Source snippet
arXivEvaluating AI Companies' Frontier Safety Frameworks: Methodology and ResultsDecember 1, 2025...
Published: December 1, 2025
-
Source: time.com
Title: Exclusive: 60 U.K
Link: https://time.com/7313320/google-deepmind-gemini-ai-safety-pledge/Source snippet
Lawmakers Accuse Google of Breaking AI Safety PledgeAugust 29, 2025 — A bipartisan group of 60 U.K. lawmakers has accused Google DeepMind...
Published: August 29, 2025
-
Source: GOV.UK
Title: frontier ai safety commitments ai seoul summit 2024
Link: https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024/frontier-ai-safety-commitments-ai-seoul-summit-2024Source snippet
AI Safety Commitments, AI Seoul Summit 20247 Feb 2025 — The UK and Republic of Korea governments announced that the following organisatio...
-
Source: gemini.google.com
Link: https://gemini.google.com/Source snippet
GeminiMeet Gemini, Google's AI assistant. Get help with writing, planning, brainstorming, and more. Experience the power of generative AI...
-
Source: cloud.google.com
Title: what is [artificial]({{ ‘artificial-goals/’ | relative_url }}) intelligence
Link: https://cloud.google.com/learn/what-is-artificial-intelligenceSource snippet
is Artificial Intelligence (AI)?Artificial intelligence (AI) is a set of technologies that empowers computers to learn, reason, and perfo...
-
Source: OpenAI
Link: https://openai.com/Source snippet
comOpenAI | Research & DeploymentWe believe our research will eventually lead to artificial general intelligence, a system that can solve...
-
Source: arxiv.org
Link: https://arxiv.org/pdf/2503.04746Source snippet
Emerging Practices in Frontier AI Safety Frameworksby MD Buhl · 2025 · Cited by 4 — At the AI Seoul Summit in 2024, a number of AI develo...
-
Source: industry.gov.au
Link: https://www.industry.gov.au/publications/seoul-declaration-countries-attending-ai-seoul-summit-21-22-may-2024Source snippet
The Seoul Declaration by countries attending the AI...24 May 2024 — Seoul Ministerial Statement for advancing AI safety, innovation and...
Published: May 2024
-
Source: hansard.parliament.uk
Title: Hansard AI Seoul Summit
Link: https://hansard.parliament.uk/commons/2024-05-23/debates/6DCE4EE5-A806-4B94-8DB8-731906631900/AISeoulSummitSource snippet
Seoul Summit - Hansard - UK Parliament23 May 2024 — On Wednesday, Ministers from more than 28 nations, the EU and the UN came together fo...
Published: May 2024
-
Source: forum.effectivealtruism.org
Title: new voluntary commitments ai seoul summit
Link: https://forum.effectivealtruism.org/posts/dmpPQ48M5ZQ8ifh8v/new-voluntary-commitments-ai-seoul-summitSource snippet
Effective Altruism ForumNew voluntary commitments (AI Seoul Summit)21 May 2024 — In the extreme, organisations commit not to develop or d...
Published: May 2024
-
Source: aisecurityandsafety.org
Title: frontier ai safety commitments
Link: https://aisecurityandsafety.org/en/frameworks/frontier-ai-safety-commitments/Source snippet
AI Security & Safety DirectoryFrontier AI Safety Commitments10 Mar 2026 — Voluntary safety pledges by 16 leading AI companies to assess r...
-
Source: ratings.safer-ai.org
Title: Risk Management Ratings Methodology
Link: https://ratings.safer-ai.org/methodology/Source snippet
At the AI Seoul Summit, multiple companies pledged to publish safety frameworks by February 2025, detailing how they manage risks when...
Published: February 2025
-
Source: mofa.go.kr
Link: https://www.mofa.go.kr/eng/brd/m_5674/view.do?seq=321007Source snippet
Ministry of Foreign Affairsseoul declaration for safe, innovative and inclusive ai by...SEOUL DECLARATION FOR SAFE, INNOVATIVE AND INCLU...
-
Source: mofa.go.kr
Link: https://www.mofa.go.kr/eng/brd/m_5674/view.do?page=1&seq=321007Source snippet
ganizations developing and deploying frontier Al, and, in this regard, note the Frontier AI Safety Commitments.Read more...
-
Source: theguardian.com
Link: https://www.theguardian.com/technology/article/2024/may/21/first-companies-sign-up-ai-safety-standards-seoul-summitSource snippet
These standards emphasize information sharing, cybersecurity investment, and prioritizing research into societal risks. Despite these mea...
-
Source: cetas.turing.ac.uk
Title: ai seoul summit stocktake reflections and projections
Link: https://cetas.turing.ac.uk/publications/ai-seoul-summit-stocktake-reflections-and-projectionsSource snippet
Seoul Ministerial Statement for advancing AI safety, innovation and inclusivity · 2. Frontier AI Safety Commitments · 3. Seoul Statement...
-
Source: theguardian.com
Link: https://www.theguardian.com/technology/article/2024/may/18/ai-seoul-global-summit-safety-openai-metaSource snippet
As technology progresses, AI systems like OpenAI's GPT-4o and Google's Project Astra are emerging, promising novel capabilities but raisi...
-
Source: internationalaisafetyreport.org
Link: https://internationalaisafetyreport.org/aboutSource snippet
An Interim Report was published in May 2024 in advance of the AI Seoul Summit 2024 and the first Annual Report was published in January 2...
Published: May 2024
-
Source: ailabwatch.org
Link: https://ailabwatch.org/resources/commitmentsSource snippet
by several companies16 AI companies joined the Frontier AI Safety Commitments in May 2024, basically committing to make responsible scali...
Published: May 2024
-
Source: csis.org
Title: ai seoul summit
Link: https://www.csis.org/analysis/ai-seoul-summitSource snippet
The AI Seoul Summit23 May 2024 — Q4: What did the 2024 AI Seoul Summit achieve? A4: The summit reinforced international commitment to saf...
Published: May 2024
Additional References
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/tom-lue-87373478_frontier-ai-safety-commitments-ai-seoul-activity-7198673945238151168-WHd5Source snippet
Frontier AI Safety Commitments, AI Seoul Summit 2024Today at the #AISeoulSummit, Google and 15 of our peers across the industry agreed to...
-
Source: metr.org
Link: https://metr.org/fsp -
Source: perplexity.ai
Link: https://www.perplexity.ai/Source snippet
Perplexity is a free AI-powered answer engine that provides accurate, trusted, and real-time answers to any question...
-
Source: montrealethics.ai
Title: ai policy corner frontier ai safety commitments ai seoul summit 2024
Link: https://montrealethics.ai/ai-policy-corner-frontier-ai-safety-commitments-ai-seoul-summit-2024/Source snippet
AI Policy Corner: Frontier AI Safety Commitments, AI Seoul...28 Apr 2025 — The Frontier AI Safety Commitments provide a framework to mit...
-
Source: flyfrontier.com
Link: https://www.flyfrontier.com/Source snippet
Frontier Airlines: Low Fares Done RightAs Home of Low Fares Done Right, find great deals and cheap flights to destinations all over North...
-
Source: frontiermodelforum.org
Title: progress update advancing frontier ai safety in 2024 and beyond
Link: https://www.frontiermodelforum.org/updates/progress-update-advancing-frontier-ai-safety-in-2024-and-beyond/Source snippet
Progress Update: Advancing Frontier AI Safety in 2024 and...29 Aug 2024 — Every member of the Frontier Model Forum has signed onto the F...
-
Source: metr.org
Title: 2025 12 09 common elements of frontier ai safety policies
Link: https://metr.org/blog/2025-12-09-common-elements-of-frontier-ai-safety-policies/Source snippet
Common Elements of Frontier AI Safety Policies...Dec 9, 2025 — In May 2024, sixteen companies agreed to do so as part of the Frontier AI...
Published: May 2024
-
Source: frontiermodelforum.org
Title: issue brief components of frontier ai safety frameworks
Link: https://www.frontiermodelforum.org/updates/issue-brief-components-of-frontier-ai-safety-frameworks/Source snippet
Issue Brief: Components of Frontier AI Safety Frameworks8 Nov 2024 — Both government and industry recognized the importance of safety fra...
-
Source: keia.org
Title: Major Commitments Launched at the AI Seoul Summit
Link: https://keia.org/the-peninsula/major-commitments-launched-at-the-ai-seoul-summit/Source snippet
May 28, 2024 — The first commitment from the AI Seoul Summit is the Seoul Declaration for Safe, Innovative, and Inclusive AI, which was a...
Published: May 28, 2024
-
Source: longtermresilience.org
Title: AI Safety Frameworks Risk Governance 1
Link: https://www.longtermresilience.org/wp-content/uploads/2025/02/AI-Safety-Frameworks-Risk-Governance-1.pdfSource snippet
B Robinson · 2025 · Cited by 2 — This was followed by the voluntary commitments made at the A...
Topic Tree