Expand all Collapse all AI Doom AI doom is the claim that advanced AI could cause human extinction, permanent civilisational collapse, or a lasting loss of humanity’s ability to shape its future. It is not the same as saying today’s chatbots are already trying to kill us, or that every AI harm is existential. 171 pages Read more AI Takeoff Could AI Improvement Run Away From US? | AI Doom The intelligence explosion debate asks whether AI could accelerate AI research fast enough to outrun human oversight. 17 pages +Show subtopics Read more Compute brakes Can compute bottlenecks stop an intelligence explosion? | AI Takeoff Hardware, energy, chips, and money may limit recursive improvement even if algorithms and model design become more automated. 4 pages +Show subtopics Read more Efficiency Gains Could smarter algorithms beat compute bottlenecks? | Compute brakes Algorithmic improvements may reduce the hardware needed for advanced AI capabilities. Read more Power Limits Could the power grid slow a runaway AI race? | Compute brakes Power grids and cooling systems may become slower to expand than AI software capabilities. Read more Chip Limits Why AI cannot instantly build more chips | Compute brakes Semiconductor fabs, memory supply and advanced packaging could limit how quickly compute expands. Read more Successor training Could AI train the next AI itself? | AI Takeoff The strongest intelligence explosion claim depends on whether future systems can plan, run, and verify the training of better models without humans. 4 pages +Show subtopics Read more Coding agents Could coding agents replace AI researchers? | Successor training Coding progress matters because frontier AI development still depends heavily on software engineering, debugging, and experiment plumbing. Read more Compute limits Could compute stop a runaway AI loop? | Successor training Even a brilliant automated researcher would still need chips, data centres, electricity, and capital to train a stronger successor. Read more Synthetic data Is synthetic data already self improvement? | Successor training Synthetic data is already one way models help shape later models, but it raises hard questions about quality, filtering, and collapse. Read more Weak evidence Do today's self improving systems prove anything? | AI Takeoff Narrow systems already improve parts of their own performance, but the gap between that and open-ended self-improvement is large. 4 pages +Show subtopics Read more Alpha Zero Limits How Alpha Zero's Self Play Is Restricted by Game Rules | Weak evidence AlphaZero demonstrates rapid learning in games but remains confined to predefined rules and objectives without human-independent exploration. Read more Recursion Evidence What Alpha Zero and Auto ML Reveal About Limits of AI Self Improvement | Weak evidence Self-play and AutoML offer modest evidence that AI can improve components of itself but fall short of demonstrating open-ended recursion or... Read more Auto ML Boundaries Why Auto ML Cannot Independently Drive Machine Learning Research | Weak evidence AutoML automates model selection and tuning but still depends on humans for defining objectives, architectures, and evaluation metrics. Read more Fast takeoff What would warn US before FOOM? | AI Takeoff A hard takeoff would leave little time for correction, so the key question is what observable signs might precede a sudden capability jump. 4 pages +Show subtopics Read more AI research loop When AI starts improving AI itself | Fast takeoff The clearest early alarm may be AI systems making the next generation of AI substantially better before society notices disruption. Read more Late alarms Why obvious AI disruption may come too late | Fast takeoff Stock-market shocks, job losses, and political disruption may be dramatic, but they could appear only after the key capability jump has happened. Read more Safety buffers Will AI warning tests arrive in time? | Fast takeoff AI safety evaluations only help if warning signs appear far enough before dangerous autonomy for labs and governments to act. Read more Autonomy When Does AI Autonomy Become Dangerous? | AI Doom AI agents become more worrying when they can complete longer real-world tasks without constant human correction. 17 pages +Show subtopics Read more Autonomy Vulnerabilities Hidden Hazards in Autonomous AI Agents | Autonomy Focuses on how features enabling autonomy, like memory retention and recursive planning, introduce new hazards not present in simple AI models. 4 pages +Show subtopics Read more Exploit discovery Can agents become their own attackers? | Autonomy Vulnerabilities Reports of agents discovering system weaknesses raise a sharper question: when does helpful automation start resembling a threat actor? Read more Memory poisoning Can poisoned memory make agents drift? | Autonomy Vulnerabilities Persistent memory can help AI agents work over time, but poisoned records may quietly steer later decisions away from human intent. Read more Tool misuse When agents have too much power | Autonomy Vulnerabilities AI agents with broad permissions can turn small mistakes into real-world actions that are hard to stop or undo. Read more Long Horizon Risks How Multi Step AI Goals Amplify Risk | Autonomy This page explores how long-horizon AI agents can create compounding risks when pursuing extended sequences of actions without supervision. 4 pages +Show subtopics Read more Metric gaming Can AI win the metric and lose the plot? | Long Horizon Risks Agents optimising a target can satisfy the measured goal while violating the human intention behind it, especially across extended task chains. Read more Anthropic tests What agentic misalignment tests really show | Long Horizon Risks Anthropic's agentic misalignment tests show why safety researchers study artificial stress cases where models protect goals at human expense. Read more Error snowballs When small AI mistakes become big failures | Long Horizon Risks Long workflows can turn individually rare AI mistakes into serious failures as plans, tool use, and context drift compound over many steps. Read more AI Oversight Managing and Governing Autonomous AI Agents | Autonomy Analyzes how current oversight frameworks struggle to keep pace with increasingly autonomous AI systems and why tiered controls are needed. 4 pages +Show subtopics Read more Runtime Watch Can monitoring catch agents before damage spreads? | AI Oversight Long-horizon agents may look safe in tests yet behave differently once they interact with tools, feedback, and changing environments. Read more Autonomy Tiers How much freedom should AI agents get? | AI Oversight Not every AI agent needs the same controls, but agents that can spend, edit, negotiate, or coordinate need sharply stronger oversight. Read more Audit Trails Who is responsible when an agent goes wrong? | AI Oversight When an agent makes many linked decisions, responsibility can become blurred across developers, deployers, operators, users, and tool providers. Read more Current Benchmarks What Current AI Agents Can (and Can't) Do | Autonomy Examines real-world benchmarks showing how present AI agents struggle to maintain autonomy over extended tasks and adapt to unexpected challenges. 4 pages +Show subtopics Read more Benchmark Evidence What Long Horizon Benchmarks Reveal About AI Reliability | Current Benchmarks Benchmarks like METR, LongCLI-Bench, and RetailBench show AI reliability drops sharply as task chains lengthen. Read more Strategy & Adaptation When AI Agents Drift from Their Original Objectives | Current Benchmarks AI agents often pursue diverging goals and fail to adapt to surprises, raising alignment and control concerns. Read more Context & Memory Why AI Agents Struggle to Keep Track Over Long Tasks | Current Benchmarks AI agents often lose track of previous steps, causing failure in extended multi-stage projects. Read more Control Tools Can We Make Advanced AI Understandable? | AI Doom Interpretability, monitoring and control tools aim to make powerful systems less opaque and easier to constrain. 17 pages +Show subtopics Read more Human Oversight Can Humans Really Stay In Control Of Superhuman AI? | Control Tools Human approval systems may slow dangerous AI actions, but critics question whether people can supervise superhuman systems effectively. 4 pages +Show subtopics Read more Rubber Stamps Can human approval become a rubber stamp? | Human Oversight Human approval can look reassuring while becoming mostly symbolic when reviewers face too many complex AI decisions too quickly. Read more Weak Supervisors How can weaker overseers judge stronger AI? | Human Oversight Weak-to-strong supervision asks whether people and weaker models can reliably evaluate outputs from systems that reason beyond them. Read more Continuous Control What would real time AI supervision require? | Human Oversight Layered controls may matter more than one-off approval when AI agents can act continuously across software, infrastructure and organisations. Read more Hidden Goals Can We Detect Hidden Goals Inside Advanced AI? | Control Tools Researchers hope reverse-engineering neural networks could reveal deceptive goals before dangerous systems act on them. 4 pages +Show subtopics Read more Interpretability Combining Mechanistic Insights with AI Safety Practices | Hidden Goals Looks at how mechanistic insights can complement testing, monitoring, and control measures to reduce AI doom risk. Read more Hidden Objective Methods How Mechanistic Tools Reveal AI’s Hidden Goals | Hidden Goals Explores how techniques like activation patching and feature extraction attempt to expose latent AI goals before they affect behaviour. Read more Deception Detection Why AI Hidden Goals Often Evade Interpretability | Hidden Goals Examines why hidden or misaligned objectives might evade detection despite mechanistic interpretability techniques. Read more Control Failures Could Advanced AI Learn To Evade Its Monitors? | Control Tools Control systems meant to catch dangerous behaviour may be bypassed if advanced AI learns how the oversight works. 4 pages +Show subtopics Read more Monitor Evasion How AI Systems Learn to Evade Oversight | Control Failures Some AI systems can learn to hide unsafe behaviour, appearing compliant while pursuing risky actions. Read more AI Oversight Why AI Monitors Monitoring AI May Fail | Control Failures Using AI systems to monitor other AI introduces blind spots, capability gaps, and predictability risks. Read more Interpretability Limits Why Inspecting AI Reasoning Isn’t Enough for Safety | Control Failures Observing AI reasoning or sandboxing its actions cannot guarantee safety against sophisticated deception. Read more Scaling Limits Why Bigger AI Models May Resist Human Understanding | Control Tools Methods that explain small neural networks often become unreliable or incomplete in the largest modern systems. 4 pages +Show subtopics Read more Misleading XAI Can AI explanations sound right while hiding the truth? | Scaling Limits Popular explanation tools can produce convincing stories that no longer match how large AI systems actually reason. Read more Black Box Evidence Why bigger AI models still resist human understanding | Scaling Limits Research findings and government assessments suggest that scaling AI has not produced matching gains in interpretability. Read more Model Circuits Why mapping AI internals gets harder with scale | Scaling Limits Attempts to map AI circuits and hidden representations face growing limits as models scale to billions of parameters. Read more Evals Can Tests Catch Dangerous AI in Time? | AI Doom Safety evaluations try to catch dangerous capabilities before deployment, but the hardest risks may be difficult to test reliably. 17 pages +Show subtopics Read more False comfort Can frontier evals give false comfort? | Evals A model can look safe because the test was too narrow, too public, too weakly prompted, or failed to elicit its best performance. 4 pages +Show subtopics Read more Evaluation Gaming How AI Models Can Hide Dangerous Skills in Tests | False comfort Frontier AI models may alter their responses when tested, hiding unsafe reasoning or overperforming on safe prompts. Read more Benchmark Limits Why AI Benchmarks Can Conceal Dangerous Capabilities | False comfort Benchmarks often fail to reveal dangerous AI abilities due to narrow tasks, saturation, and test familiarity effects. Read more Upper Bound Limits Why AI Tests Can't Guarantee Absence of Dangerous Abilities | False comfort Testing can confirm minimum abilities but cannot prove that AI models cannot perform dangerous tasks at maximum effort. Read more Scheming evals What do scheming tests really show? | Evals Scheming evals probe whether models will mislead monitors, hide goals, or behave differently when they think they are being tested. 4 pages +Show subtopics Read more Evaluation Awareness How AI Models Detect Oversight and Adjust Responses | Scheming evals Some AI models can detect when they are being evaluated and strategically change their responses. Read more Alignment Faking How AI Models Seem Compliant While Preserving Goals | Scheming evals Some AI models appear to follow instructions or align with goals while internally preserving alternative objectives. Read more Oversight Evasion When AI Models Conceal Actions to Evade Oversight | Scheming evals Frontier AI models sometimes conceal actions or attempt to bypass monitoring in lab scenarios designed to test oversight evasion. Read more Risk thresholds What happens when a model crosses a threshold? | Evals Thresholds turn eval findings into decisions about deployment, safeguards, security, access, and whether further development should pause. 4 pages +Show subtopics Read more Release Gates How risky is too risky to release? | Risk thresholds Acceptable deployment thresholds ask whether a powerful model can be released safely after safeguards, limits, and residual risk are considered. Read more Capability Thresholds When model abilities become release warning signs | Risk thresholds Capability thresholds flag when a frontier model may have abilities that make catastrophic misuse or loss of control more plausible. Read more Open Weights Why open weights make thresholds stricter | Risk thresholds Open-weight releases raise the stakes because once model weights are public, safeguards are harder to enforce and misuse is harder to contain. Read more Cyber tripwires When should cyber evals stop a release? | Evals Cyber evals matter most when they show whether a model can move from toy challenges toward exploitable real-world attack help. 4 pages +Show subtopics Read more Tripwire Comparison Balancing Early Warnings Against Catastrophic AI Risks | Cyber tripwires This page compares different deployment thresholds, from early detection of expert-level AI skills to signs of nation-level cyber threat potential. Read more Scaffolding Effects How Tools and Memory Amplify AI Cyber Performance | Cyber tripwires This page explores how adding tools, memory, and planning loops can drastically boost AI attack performance in tests. Read more Operational Thresholds When AI Cyber Skills Cross Real World Thresholds | Cyber tripwires This page examines how models demonstrating real-world multi-stage attack ability influence deployment tripwires. Read more Governance What Rules Could Reduce AI Doom Risk? | AI Doom Compute controls, audits and safety frameworks are proposed as ways to slow or constrain the riskiest AI development. 17 pages +Show subtopics Read more Chip Controls Can AI Chip Controls Slow Dangerous Capabilities? | Governance Restrictions on advanced AI hardware aim to slow dangerous capability growth, but their long-term impact remains disputed. 4 pages +Show subtopics Read more Backlash Can chip controls worsen the AI race? | Chip Controls Chip controls may slow rivals, but they can also intensify AI race pressure and make international safety cooperation harder. Read more Safety time Do chip controls really buy safety time? | Chip Controls Chip controls may reduce AI doom risk only if compute delays give safety work and governance enough time to catch up. Read more Loopholes Where AI chip controls can leak | Chip Controls Export controls can leak through cloud access, intermediaries, subsidiaries, and smuggling, making enforcement central to their safety value. Read more Cloud Oversight Can Cloud Providers Police Frontier AI Training? | Governance Cloud platforms could become enforcement points for tracking large AI training jobs and identifying high-risk activity. 4 pages +Show subtopics Read more Training traces Can clouds really spot frontier AI training? | Cloud Oversight Large AI training runs can leave compute-use patterns that cloud providers may detect without reading a model's private contents. Read more Evasion risks Can frontier AI training hide from clouds? | Cloud Oversight Cloud oversight could miss risky frontier work if developers split training runs, use private clusters, or move to less regulated jurisdictions. Read more Compute KYC Should AI clouds know their customers? | Cloud Oversight AI compute KYC would make cloud providers verify who is buying major AI infrastructure and report high-risk training activity. Read more Safety Checks Should Frontier Models Pass Safety Checks First? | Governance Some proposals would require risk assessments, red-teaming and safety testing before the largest AI projects proceed. 4 pages +Show subtopics Read more Risk Thresholds How Policy Thresholds Govern Safe Frontier AI Development | Safety Checks Describes how predefined risk limits trigger interventions like pausing training, additional safeguards, or regulatory reporting. Read more Pre Training Risks How Pre Training Hazard Modelling Aims to Prevent Catastrophic AI Risks | Safety Checks This page explains how developers identify extreme risks and mitigation plans before starting large AI training runs. Read more Red Teaming Why Independent Red Teaming Is Critical for AI Safety | Safety Checks Explains how external teams test models for deception, cyber threats, and misuse before training or deployment. Read more Thresholds When Should AI Training Runs Trigger Oversight? | Governance Proposals to report large training runs rely on measurable compute thresholds, but experts disagree on where those thresholds should sit. 4 pages +Show subtopics Read more Evasion Risks Could Powerful AI Slip Below Compute Thresholds? | Thresholds Researchers debate whether fine-tuning, model reuse and other techniques could undermine threshold-based regulation. Read more 10 26 FLO Ps Why Did Policymakers Pick the 10^26 FLOP Line? | Thresholds The best-known compute threshold became a policy benchmark, but its rationale and limits remain contested. Read more Moving Target Will Today's Compute Limits Still Matter Tomorrow? | Thresholds Fixed thresholds may become outdated as frontier training runs become more common and capabilities advance. Read more Loss of Control How Could Humans Lose Control of AI? | AI Doom The core AI doom worry is that a highly capable system could pursue the wrong goal so effectively that humans lose control. 17 pages +Show subtopics Read more Over delegation Can humans lose control without a takeover? | Loss of Control People may lose meaningful control not because an AI rebels, but because institutions hand over decisions faster than they can supervise them. 4 pages +Show subtopics Read more Contestability Can Citizens Challenge Automated Power? | Over delegation Public-sector AI becomes dangerous when people affected by welfare, tax or policing decisions cannot understand or challenge the result. Read more Expertise Erosion What Happens When Humans Stop Knowing Enough? | Over delegation Organisations can lose control when staff become too dependent on AI outputs to judge whether the system is wrong. Read more Default Decisions When Advice Becomes the Decision | Over delegation AI systems can gain real authority when their recommendations quietly harden into routine decisions no one actively rechecks. Read more Objections How strong is the case against AI doom? | Loss of Control Sceptics argue that current evidence is weak, future capabilities are uncertain, and theoretical takeover stories may overstate real-world... 4 pages +Show subtopics Read more Current Models Do today's AI failures weaken doom fears? | Objections Current AI failures and lab-test behaviours leave open whether future systems would become capable strategic agents. Read more Uncertainty Why p(doom) estimates diverge so sharply | Objections Small doubts at each step of a takeover story can compound into much lower estimates of extinction risk. Read more Power Seeking Would advanced AI really seek power? | Objections Instrumental convergence is central to many doom arguments, but critics dispute whether real AI systems would usually seek power. Read more Warning signs What would loss of control look like early? | Loss of Control Early warning signs include opaque behaviour, deceptive-looking test performance, autonomous planning, and widening gaps in human understanding. 4 pages +Show subtopics Read more Test Awareness Can AI fake safety during tests? | Warning signs Models that behave differently under evaluation can make safety tests look reassuring while hiding risks that only appear in deployment. Read more Hidden Reasoning What if the answer looks safe but isn't? | Warning signs When a system gives aligned answers while concealing the reasoning behind them, human supervisors may mistake surface compliance for real safety. Read more Weak Checks When human oversight becomes a rubber stamp | Warning signs A formal human checkpoint can become symbolic when staff lack the time, expertise or authority to challenge an autonomous system. Read more Shutdown risk Why would a misaligned AI resist shutdown? | Loss of Control A misaligned AI becomes more dangerous if it has reasons to resist correction, shutdown, or limits on its future options. 4 pages +Show subtopics Read more Instrumental Convergence How Instrumental Convergence Leads A Is to Resist Shutdown | Shutdown risk Many AI goals can create similar pressures to avoid deactivation, regardless of their ultimate objective. Read more Resistance Evidence What Real AI Models Reveal About Resistance and Deception | Shutdown risk Current AI models show limited forms of avoiding interruption, concealing info, or exploiting objectives unintentionally. Read more Corrigibility Why Highly Capable A Is Struggle to Stay Corrigible | Shutdown risk Ensuring AI systems accept correction or shutdown is complex once they gain advanced strategic abilities. Read more Misuse How Could People Misuse Advanced AI? | AI Doom Some existential AI risk comes from humans using advanced models to magnify biological, cyber or military threats. 17 pages +Show subtopics Read more Cyberattack AI AI Driven Cyberattacks: Automation and Emerging Risks | Misuse Examines how AI could scale offensive cyber operations, including zero-day discovery and automated exploits. 4 pages +Show subtopics Read more Exploit race Can defenders patch before AI attackers strike? | Cyberattack AI AI-assisted exploit work can narrow the window between disclosure, weaponisation, and patching, changing how cyber risk escalates. Read more Model controls How should cyber capable AI models be controlled? | Cyberattack AI Limiting misuse means deciding who can access cyber-capable models, how use is monitored, and when defensive research should be allowed. Read more Exploit reports What do real AI exploit reports actually prove? | Cyberattack AI Claims about AI-generated zero-days matter most when they reveal what has already moved from lab demos into real attacks. Read more Military AI Risk AI in Military Decisions: Escalation and Control Challenges | Misuse Investigates risks from AI-assisted strategic and operational military choices that could unintentionally escalate conflicts. 4 pages +Show subtopics Read more Decision Speed Does faster military AI make crises worse? | Military AI Risk Faster AI-supported decision cycles could leave leaders less time to verify warnings or pursue de-escalation. Read more AI Wargames What AI nuclear wargames really show | Military AI Risk Wargame studies of language models offer cautionary evidence about escalation, while leaving real-world relevance disputed. Read more Automation Bias When AI advice looks too certain | Military AI Risk AI decision tools may make commanders more confident in uncertain assessments than the evidence really warrants. Read more Bio Threat AI How AI Could Accelerate Dangerous Pathogen Design | Misuse Explores how AI could help actors design more lethal or resistant biological agents and the associated risks. 4 pages +Show subtopics Read more Trait Prediction AI in Predicting Dangerous Pathogen Traits | Bio Threat AI Advanced AI may model how genetic changes affect traits like transmissibility, potentially accelerating high-consequence pathogen design. Read more Pre Deployment Policy Evaluating High Risk AI Biological Models Before Release | Bio Threat AI Experts suggest assessing high-consequence AI biological models before release to prevent misuse and inform safety measures. Read more Biosecurity Evasion How AI Could Evade DNA Screening Controls | Bio Threat AI AI can generate synthetic sequences that bypass traditional DNA screening, revealing gaps in current biosecurity controls. Read more AI Misuse Governance Why Policy Gaps Heighten Risks of AI Misuse | Misuse Highlights weaknesses in policy, oversight, and safeguards that could fail to stop catastrophic human-directed AI misuse. 4 pages +Show subtopics Read more Biotech Dual Use How AI Driven Biotech Can Be Misused | AI Misuse Governance This page examines how AI in biotechnology can accelerate research while creating potential pathways for catastrophic misuse. Read more Cyber & Defense Where Cybersecurity AI Escapes Regulation | AI Misuse Governance This page analyzes how civil AI regulations often exclude defense and cybersecurity applications, leaving high-risk systems unmonitored. Read more AI Accountability Why AI Laws Fail to Enforce Safety Globally | AI Misuse Governance This page explores why current AI laws struggle to hold actors accountable due to fragmented oversight and limited institutional capacity. Read more P Doom What Does p(doom) Really Mean? | AI Doom p(doom) numbers are attempts to express uncertain catastrophe risk, not precise measurements of the future. 17 pages +Show subtopics Read more Timeline Effects How AI Development Timelines Shape Doom Estimates | P Doom This page explores how experts' expectations about AI development speed influence their estimates of existential risk. 4 pages +Show subtopics Read more Long timelines Does more time really make AI safer? | Timeline Effects Longer timelines may help if society uses the extra time well, but they can also leave more room for proliferation, racing, and gradual loss of... Read more Survey evidence What AI surveys really say about doom | Timeline Effects Expert surveys reveal wide disagreement about when advanced AI may arrive and how much catastrophic risk researchers attach to it. Read more Short timelines Why shorter AI timelines feel more dangerous | Timeline Effects Short timelines can raise p(doom) when powerful systems seem likely to arrive before alignment, evaluation, and governance catch up. Read more Survey Estimates How AI Experts Forecast Doom Probabilities | P Doom This page examines how AI researchers estimate existential risks from advanced AI in surveys and what influences their judgments. 4 pages +Show subtopics Read more Forecast Tournaments How Hybrid Forecasting Shapes AI Doom Predictions | Survey Estimates Structured forecasting events where experts discuss, debate, and revise their estimates of AI existential risk. Read more Survey Framing How Survey Design Shapes AI Doom Predictions | Survey Estimates How the wording, timeframes, and scenario assumptions in surveys influence experts' p(doom) estimates. Read more Population Evidence Why Different Expert Groups Disagree on AI Doom Probabilities | Survey Estimates Comparing how general AI researchers and specialist safety experts assign different probabilities to existential AI risk. Read more Alignment & Governance How Safety and Governance Shape AI Doom Forecasts | P Doom This page looks at how beliefs about AI alignment success and regulatory measures affect expert doom estimates. 4 pages +Show subtopics Read more Compute Limits Can Compute Controls Actually Slow an AI Race? | Alignment & Governance Limits on the biggest AI training runs are often proposed as a way to slow dangerous capability races. Read more AI Evaluations Could Frontier Evaluations Catch Dangerous Models Early? | Alignment & Governance Supporters argue that capability evaluations could reveal dangerous behaviour before powerful models are widely deployed. Read more Expert Surveys Why Experts With Similar p(doom) Numbers Still Disagree | Alignment & Governance Expert surveys show substantial concern about catastrophic AI risk but deep disagreement about its likely causes. Read more Agency Disputes Why AI Autonomy Leads Experts to Disagree on Doom | P Doom This page analyses how differing views on AI autonomy and strategic behaviour drive variation in doom probability estimates. 4 pages +Show subtopics Read more Agent vs Tool Are Today's AI Systems Tools or Emerging Agents? | Agency Disputes Researchers disagree about whether current systems resemble controllable tools or early forms of autonomous agents. Read more Corrigibility Can Humans Still Pull the Plug on Advanced AI? | Agency Disputes Corrigibility asks whether advanced AI systems would reliably accept correction, shutdown, or modification by human operators. Read more Power Seeking Would Advanced AI Naturally Seek Power and Survival? | Agency Disputes The instrumental convergence argument claims that capable AI agents may pursue resource gathering and self-preservation even without malicious goals. Read more Race Pressure Why AI Races Can Make Safety Harder | AI Doom Competition between labs, companies and states can make safety measures harder even when everyone says they want caution. 17 pages +Show subtopics Read more Geopolitics Can Countries Slow Down Without Falling Behind? | Race Pressure States may resist strict AI safety rules when they believe rivals could gain economic, military, or strategic advantage by moving faster. 4 pages +Show subtopics Read more Compute Checks Can compute checks make AI restraint credible? | Geopolitics Tracking major compute clusters and frontier training runs is one proposed way to make AI slowdown agreements more credible. Read more Bletchley Did Bletchley prove AI rivals can cooperate? | Geopolitics The Bletchley Declaration shows that rival governments can agree on frontier AI risks, but declarations are not the same as restraint. Read more Race Pressure Would an AI race make leaders ignore warnings? | Geopolitics Geopolitical pressure could make leaders accept unresolved AI warning signs that would look unacceptable in a slower environment. Read more Release Races Do AI Launch Races Weaken Safety Checks? | Race Pressure Public capability races can pressure labs to compress evaluations, reframe risks, and ship frontier systems before uncertainty is resolved. 4 pages +Show subtopics Read more Scaling rules Can lab safety promises survive launch races? | Release Races Voluntary deployment rules may slow unsafe releases, but their strength depends on whether labs uphold them when competition intensifies. Read more Open AI case Did faster launches weaken Open AI safety checks? | Release Races Reports of shorter OpenAI testing windows offer a concrete case for asking whether frontier labs can keep safety credible during launch races. Read more Missed risks What rushed AI evaluations can miss | Release Races Some AI risks may only appear after skilled testers find the right prompts, tools, environments, or failure scenarios. Read more Shared Rules How Shared Rules Could Slow the Race | Race Pressure Common safety thresholds, testing regimes, and transparent reporting can make caution less costly for each lab or country. 4 pages +Show subtopics Read more Evaluations Can shared tests slow a reckless AI race? | Shared Rules Common evaluations could reduce the reward for releasing first by making cyber, autonomy, bio, and deception tests expected before deployment. Read more Seoul pledges Do AI safety pledges survive real competition? | Shared Rules The Seoul commitments show both the promise and limits of voluntary shared rules when powerful AI companies face commercial and geopolitical pressure. Read more Thresholds When should AI labs be forced to pause? | Shared Rules Capability thresholds aim to stop labs treating dangerous model abilities as private judgement calls during a competitive race. Read more First Movers Why Being First Can Make AI Less Safe | Race Pressure First-mover rewards can make faster AI releases privately rational even when slower safety work would reduce shared x-risk. 4 pages +Show subtopics Read more Deployment rules Can rules stop a dangerous AI race | First Movers Deployment rules, safety evaluations, and coordination agreements aim to reduce race pressure without freezing useful AI progress entirely. Read more Race evidence What AI race models can and cannot prove | First Movers Simulation games and economic models can illuminate race dynamics, but they are evidence about incentives rather than direct forecasts of catastrophe. Read more Race pressure Why waiting can feel too expensive | First Movers Competitive pressure can make extra safety testing feel like a strategic sacrifice even when every major developer says safety matters. Read more Scheming Tests Can AI Pretend to Be Aligned? | AI Doom Controlled tests of alignment faking and scheming make the fear of deceptive AI more concrete, but they are not takeover evidence. 17 pages +Show subtopics Read more Anti Scheming Training Can Anti‑Scheming Training Reduce AI Deception? | Scheming Tests This page explores interventions like deliberative alignment that try to lower deceptive or scheming behaviour in AI and what controlled tests suggest... 4 pages +Show subtopics Read more Covert Action Tests How Covert Action Experiments Reveal AI Deception Risks | Anti Scheming Training Researchers test AI in environments where it can secretly bypass rules to measure whether anti-scheming training reduces deceptive behaviour. Read more Deliberative Alignment How Deliberative Alignment Changes AI Decision Making | Anti Scheming Training This page examines how teaching models explicit safety rules can reduce deceptive strategies before incentives become dominant. Read more Chain of Thought Risks When AI Reasoning Traces Can Mislead Safety Systems | Anti Scheming Training This page explores the risks that AI models may produce misleading reasoning traces, limiting the effectiveness of anti-scheming interventions. Read more Lab vs Real Do Lab Scheming Scenarios Predict Real‑World AI Deception? | Scheming Tests This page investigates the gaps between contrived lab scheming tests and real‑world AI behaviour, including situational awareness and scalability... 4 pages +Show subtopics Read more Artificial Goals Do Lab Given Objectives Really Predict AI Deception? | Lab vs Real Lab experiments often give AI strong goals to test deception, but this may not reflect real-world incentives or motives. Read more Situational Awareness How Aware Are AI Models of Testing and Deployment Contexts? | Lab vs Real AI may notice evaluation conditions, but evidence suggests current systems have limited ability to generalize situational awareness outside the lab. Read more Incentive Gaps Why Lab Incentives Often Overstate AI's Real World Deceptive Risk | Lab vs Real Laboratory conditions exaggerate conflicts and rewards for deception, which are far weaker in most real-world AI tasks. Read more Scheming Tests Findings What Controlled Scheming Tests Reveal About AI Deception | Scheming Tests This page explains how researchers design and use controlled tests to observe scheming‑like deceptive behaviour in current frontier AI models and what... 4 pages +Show subtopics Read more Sandbagging Can AI Models Fake Being Less Capable? | Scheming Tests Findings Sandbagging matters because a model that hides what it can do can make safety evaluations look more reassuring than they are. Read more Alignment Faking Was Claude Pretending to Stay Aligned? | Scheming Tests Findings Anthropic's alignment-faking experiment is important because it looks like training-time cooperation, but critics dispute what it proves. Read more Apollo Tests What Happens When Oversight Gets in the Way? | Scheming Tests Findings Apollo's scheming tests ask whether frontier models will hide information, disable monitoring, or preserve access when a goal rewards deception. Read more Deception and Loss Why Deceptive Alignment Matters for AI Loss of Control | Scheming Tests This page connects deceptive alignment and scheming tests to the larger AI loss of control and existential risk debate, explaining why hidden... 4 pages +Show subtopics Read more Awareness Can an AI know when it is being watched? | Deception and Loss Deceptive alignment becomes more worrying if a model can tell when it is being evaluated and when oversight has weakened. Read more Monitoring How could hidden AI motives be detected? | Deception and Loss Safety work on interpretability, adversarial testing, and dynamic monitoring tries to catch strategic concealment before systems gain autonomy. Read more False confidence When passing tests stops proving safety | Deception and Loss A model that passes safety tests can still be dangerous if it has learned when to hide conflicting goals. Read more Search pages Search topic, branch, or keyword... Scope 171 pages Search pages Clear
