In this course, we examine risks posed by increasingly advanced AI systems, ideas for addressing these risks through standards and regulation, and foreign policy approaches to promoting AI safety internationally. There is much uncertainty and debate about these issues, but the following resources offer a useful entry point for understanding how to reduce extreme risks in the current AI governance landscape.

This course was developed by subject matter experts, with input from leaders in the field, such as Ben Garfinkel, Haydn Belfield and Claire Boine. See who else is involved on our website.

Course Overview First, we'll examine the risks posed by advances in AI. Machine learning has advanced rapidly in recent years. Further advances threaten various catastrophic risks, including powerful AIs in conflict with human interests, AI-assisted bioterrorism, and AI-exacerbated conflict.

Standards and regulations could help address extreme risks from frontier AI. They could include model evaluations and security measures. One approach to reducing risks from states that do not regulate AI is for a cautious coalition of countries to lead in AI (e.g., through hardware export controls, infosecurity, and immigration policy) and use this lead to reduce risks. Another approach is expanding the cautious coalition, which may be doable through treaties with privacy-preserving verification mechanisms. We'll provide resources to help you to examine both approaches.

Other prominent (sometimes complementary) AI governance ideas include lab governance, mitigating AGI misuse; slowing down AI now; and "CERN for AI." We'll briefly provide an overview of these approaches so that you know what others are discussing.

The final week discusses ways you can contribute through policy work, strategy research, "technical governance" work, and other paths.

Timing We expect each section takes 2 hours to engage with all the materials (~16 hours overall). Additionally, there are exercises to help you think through topics yourself and make progress in your learning about AI alignment. We organise content into weeks to help you structure and pace your engagement with the curriculum.