Summer 2021 Undergraduate Research Fellowship: Featured Projects
This is a compilation of sources of potential research questions on existential risk mitigation. All credit for these research questions and lists goes to Arden Koehler, Howie Lempel, David Manheim, and particularly Michael Aird, as well as other researchers and experts cited in these lists; mistakes are our own. We have compiled this list as a conversation-starter, not as strong endorsements. Listings are lightly edited for this context.
Research questions about entirely natural risks (e.g. asteroid strikes, supervolcanoes) have been excluded from this list because research strongly suggests that the risks involved are extremely low; anthropogenic risks appear more urgent (Ord, 2020).
Longtermism, Existential Risks, or GCRs Generally
The Precipice, Appendix F: Policy and research recommendations - Toby Ord, 2020
Also available here
Research questions that could have a big social impact, organised by discipline - Arden Koehler & Howie Lempel (80,000 Hours), 2020
Pp. 83-112 of Legal Priorities Research: A Research Agenda - Legal Priorities Project, 2021
Open Research Questions - Center on Long-Term Risk
Various lists of question proposals - Michael Aird, 2020
AI Risks: Technical
Concrete Problems in AI Safety - Amodei et al., 2016
“Clusters of ideas that we believe warrant further attention and research” - Center for Human-Compatible AI (CHAI)
Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda - Nate Soares and Benya Fallenstein (MIRI)
Alignment for Advanced Machine Learning Systems - Jessica Taylor et al. (MIRI), 2016
Research Agenda v0.9: Synthesising a human's preferences into a utility function - Stuart Armstrong, 2019
Related talk here
FLI AI Safety Research Landscape - Future of Life Institute, 2018
Associated paper here
Some materials from or related to Paul Christiano that some people have indicated serve as research agendas, collections of questions, or supporting materials:
Other research agendas are listed here
Pp. 35-55 of Legal Priorities Research: A Research Agenda - Legal Priorities Project, 2021
Artificial Intelligence and Global Security Initiative Research Agenda - Centre for a New American Security
Promising research projects - AI Impacts, 2018
Cooperation, Conflict, and Transformative Artificial Intelligence (the Center on Long-Term Risk’s research agenda) - Jesse Clifton, 2019
A survey of research questions for robust and beneficial AI - Future of Life Institute
Proposals by individual researchers:
Problems in AI risk that economists could potentially contribute to - Michael Aird, 2021
Technical AGI safety research outside AI - Richard Ngo, 2019
“Studies which could illuminate our strategic situation with regard to superintelligence” - Luke Muehlhauser, 2014 (he also made a list in 2012)
A shift in arguments for AI risk - Tom Sittler, 2019
Longtermist AI policy projects for economists - Risto Uuk
Pp. 56-82 of Legal Priorities Research: A Research Agenda - Legal Priorities Project, 2021
Project Ideas in Biosecurity for EAs - David Manheim ("In conjunction with a group of other EA biosecurity folk"), 2021
80 Questions for UK Biological Security - Luke Kemp et al., 2021
Climate research suggestions from The Precipice - Toby Ord, 2020
Additional research questions - 80,000 Hours, 2020:
More generally, what environmental problems — if any — pose existential risks? (Adapted from Effective Thesis)
What are potential risks from geoengineering technologies and which of these technologies — if any — might be promising for mitigating climate change?
Nuclear risks research suggestions from The Precipice - Toby Ord, 2020
Improve modeling on nuclear winter and climate effect of asteroids, comets, and supervolcanoes
Work on resolving the key uncertainties in nuclear winter modelling.
Characterise the remaining uncertainties then use Monte Carlo techniques to show the distribution of outcome possibilities, with a special focus on the worst-case possibilities compatible with our current understanding.
Investigate which parts of the world appear most robust to the effects of nuclear winter and how likely civilisation is to continue there.
Research and develop methods for genetically engineering or breeding crops that could thrive in the tropics during a nuclear winter scenario (Adapted from ALLFED, Effective Theses Topic Ideas)
How does Chinese nuclear no first-use policy affect global stability and potential strategic doctrines for emerging technologies? (Adapted from Brian Tse, personal correspondence)