Skip to main content Skip to secondary navigation

Research

Main content start

Summer 2020 Undergraduate Research Fellowship Featured Projects Archive

To see more of our Summer 2020 Undergraduate Research Fellowship Projects, check out our abstract book.

Research Questions

This is a compilation of sources of potential research questions on existential risk mitigation. All credit for these research questions and lists goes to Arden Koehler, Howie Lempel, David Manheim, and particularly Michael Aird, as well as other researchers and experts cited in these lists; mistakes are our own. We have compiled this list as a conversation-starter, not as strong endorsements. Listings are lightly edited for this context. 

Research questions about entirely natural risks (e.g. asteroid strikes, supervolcanoes) have been excluded from this list because research strongly suggests that the risks involved are extremely low; anthropogenic risks appear more urgent (Ord, 2020).


Longtermism, Existential Risks, or GCRs Generally

The Precipice, Appendix F: Policy and research recommendations - Toby Ord, 2020

Also available here

Research questions that could have a big social impact, organised by discipline - Arden Koehler & Howie Lempel (80,000 Hours), 2020

Pp. 83-112 of Legal Priorities Research: A Research Agenda - Legal Priorities Project, 2021 

Open Research Questions - Center on Long-Term Risk

ALLFED’s research priorities and Effective Theses topic ideas - 2019

Various lists of question proposals - Michael Aird, 2020

Crucial questions for longtermists

Some history topics it might be very valuable to investigate

Questions related to moral circles that are listed at the end of this post and in this comment

Research statement [FHI RSP]

 

AI Risks: Technical

Concrete Problems in AI Safety - Amodei et al., 2016

“Clusters of ideas that we believe warrant further attention and research” - Center for Human-Compatible AI (CHAI)

Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda - Nate Soares and Benya Fallenstein (MIRI)

Alignment for Advanced Machine Learning Systems - Jessica Taylor et al. (MIRI), 2016

Research Agenda v0.9: Synthesising a human's preferences into a utility function - Stuart Armstrong, 2019

Related talk here

FLI AI Safety Research Landscape - Future of Life Institute, 2018

Associated paper here

Some materials from or related to Paul Christiano that some people have indicated serve as research agendas, collections of questions, or supporting materials:

Iterated Distillation and Amplification

AI alignment landscape

Directions and desiderata for AI alignment

Other research agendas are listed here

 

AI Strategy/Governance

The Centre for the Governance of AI’s research agenda - 2018

Pp. 35-55 of Legal Priorities Research: A Research Agenda - Legal Priorities Project, 2021 

Artificial Intelligence and Global Security Initiative Research Agenda - Centre for a New American Security

Promising research projects - AI Impacts, 2018

Cooperation, Conflict, and Transformative Artificial Intelligence (the Center on Long-Term Risk’s research agenda) - Jesse Clifton, 2019

A survey of research questions for robust and beneficial AI - Future of Life Institute

Proposals by individual researchers:

Problems in AI Alignment that philosophers could potentially contribute to - Wei Dai, 2019

Problems in AI risk that economists could potentially contribute to - Michael Aird, 2021

Technical AGI safety research outside AI - Richard Ngo, 2019

“Studies which could illuminate our strategic situation with regard to superintelligence” - Luke Muehlhauser, 2014 (he also made a list in 2012)

A shift in arguments for AI risk - Tom Sittler, 2019

Longtermist AI policy projects for economists - Risto Uuk

 

Biorisks

Pp. 56-82 of Legal Priorities Research: A Research Agenda - Legal Priorities Project, 2021 

Project Ideas in Biosecurity for EAs - David Manheim ("In conjunction with a group of other EA biosecurity folk"), 2021

80 Questions for UK Biological Security - Luke Kemp et al., 2021

 

Climate Risks

Climate research suggestions from The Precipice - Toby Ord, 2020

Additional research questions - 80,000 Hours, 2020:

More generally, what environmental problems — if any — pose existential risks? (Adapted from Effective Thesis)

What are potential risks from geoengineering technologies and which of these technologies — if any — might be promising for mitigating climate change?

 

Nuclear Risks

Nuclear risks research suggestions from The Precipice - Toby Ord, 2020

Improve modeling on nuclear winter and climate effect of asteroids, comets, and supervolcanoes

Work on resolving the key uncertainties in nuclear winter modelling.

Characterise the remaining uncertainties then use Monte Carlo techniques to show the distribution of outcome possibilities, with a special focus on the worst-case possibilities compatible with our current understanding.

Investigate which parts of the world appear most robust to the effects of nuclear winter and how likely civilisation is to continue there.

Research and develop methods for genetically engineering or breeding crops that could thrive in the tropics during a nuclear winter scenario (Adapted from ALLFED, Effective Theses Topic Ideas)

How does Chinese nuclear no first-use policy affect global stability and potential strategic doctrines for emerging technologies? (Adapted from Brian Tse, personal correspondence)

 

Global Priorities

The Global Priorities Institute’s research agenda - 2019