If you’re overwhelmed trying to understand how advanced AI could impact our lives, you’re not alone. There’s a lot of noise out there.<\/p>\n
We’ve boiled it down to 11 essential resources for getting up to speed on the risks of AGI.<\/p>\n
We’ve chosen these because they represent the most influential ideas shaping the current debate. Though we don’t agree with everything the authors say, we think they’re well worth reading.<\/p>\n
Researchers at Forethought argue that an “intelligence explosion” could compress a century of technological progress into a decade, creating numerous grand challenges that humanity must prepare for now. You can listen to Will MacAskill discuss this piece on our podcast<\/a>.<\/p>\n An analysis of a concrete scenario in which AGI arrives soon via the automation of AI research. The AI 2027 team also provides its own forecasts of several key outcomes in the accompanying research<\/a>. Many people (including us) think it’s unlikely things will unfold this fast, but in any case it has become one of the most discussed pieces of research in the field. (The team wrote an update<\/a> on their timeline estimates in April 2026.)<\/p>\n Former OpenAI employee Leopold Aschenbrenner makes a compelling case \u2014 across five in-depth chapters \u2014 that AGI is coming much sooner than many expect, and few realise just how much it will change the world. We think this piece might underplay the challenge of aligning AGI with human interests<\/a> and the need for international coordination on AI risks. However, its quantitative predictions about AI development have roughly borne out so far<\/a>.<\/p>\n Researcher Ege Erdil makes one of the most influential arguments against the idea that we may have AGI within the next decade \u2014 including his doubts about the prospect of a rapid ‘intelligence explosion,’ and why he expects current revenue trends in AI development to slow down. Ege and Tamay Besiroglu discussed these ideas on the Dwarkesh podcast<\/a>.<\/p>\n Holden Karnofsky’s series from 2021 argues that transformative AI could make the upcoming decades the most important in history. Some of it is now out of date, but contains several useful articles including How we could stumble into AI catastrophe<\/a>, AI could defeat all of us combined<\/a>, Why AI alignment could be hard with modern deep learning<\/a> by guest author Ajeya Cotra, and Jobs for helping with the most important century<\/a>.<\/p>\n Experts Ajeya Cotra and Arvind Narayanan discuss the factors behind the pace of AI development. They present contrasting views about the likely speed of progress in AI and its societal effects, offering useful insights into the state of the debate.<\/p>\n Projections by researchers at Epoch suggest AI companies can continue scaling their systems through 2030, primarily facing constraints in power availability and chip manufacturing capacity.<\/p>\n Joe Carlsmith’s ‘Is Power-Seeking AI an Existential Risk?’ is one of the central papers putting together the argument that extremely capable AI systems could pose an existential threat to humanity. The idea: future AIs might be motivated to disempower humans \u2014 and they could become smart enough to succeed.<\/p>\n ‘Gradual Disempowerment’ argues that even if we avoid the risks of power-seeking and scheming AIs, there may be other ways AI systems could disempower humanity. Our political, economic, and cultural systems might slowly drift away from serving human interests in a world with advanced AI.<\/p>\n “Taking AI Welfare Seriously” argues that there’s a realistic possibility some AIs will be conscious in the near future. As the authors explain, this means we shouldn’t only<\/em> be worrying about the risks AI poses to humanity \u2014 we potentially need to consider the interests and welfare of future AI systems as well.<\/p>\n It’s important to understand why<\/em> there’s enthusiasm for building powerful AI systems, despite the risks. In ‘Machines of Loving Grace,’ the CEO of Anthropic \u2014 the AI company behind Claude \u2014 attempts to paint a positive vision for powerful AI.<\/p>\n Here are additional lists of resources we’ve put together for specific AI-related topics.<\/p>\n2. AI 2027<\/a> by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean (April 2025)<\/h4>\n
3. Situational Awareness: The Decade Ahead<\/a> by Leopold Aschenbrenner (June 2024)<\/h4>\n
4. The case for multi-decade AI timelines<\/a> by Ege Erdil from Epoch AI (April 2025)<\/h4>\n
5. The Most Important Century<\/a> by Holden Karnofsky and other authors (2021)<\/h4>\n
6. Does AI Progress Have a Speed Limit?<\/a> by Ajeya Cotra and Arvind Narayanan (April 2025)<\/h4>\n
7. Can AI scaling continue through 2030?<\/a> by Jaime Sevilla et al. at Epoch AI (August 2024)<\/h4>\n
8. Existential Risk from Power-Seeking AI<\/a> by Joe Carlsmith (2023)<\/h4>\n
9. Gradual Disempowerment<\/a> by Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud (January 2025)<\/h4>\n
10. Taking AI Welfare Seriously<\/a> by Robert Long, Jeff Sebo, et al. (November 2024)<\/h4>\n
11. Machines of Loving Grace \u2014 How AI Could Transform the World for the Better<\/a> by Anthropic CEO Dario Amodei (October 2024)<\/h4>\n
Other reading lists by topic<\/h2>\n