Micah Carroll, Matija Franklin & Hal Ashton, Beyond Preferences in AI Alignment
Tan Zhi-Xuan (2025)
One-Sentence Thesis
The authors challenge the dominant preferentist approach to AI alignment, which assumes that human preferences are an adequate representation of human values, and argue for a reframing of AI alignment to focus on normative standards and social roles.
Argument Outline
- 1Introduction to the preferentist approach to AI alignment
- 2Critique of rational choice theory as a descriptive model of human decision-making
- 3Critique of expected utility theory as a normative standard of rationality
- 4Alternative approaches to AI alignment, including alignment with normative standards and social roles
Key Distinctions
Preferentist approach vs. alternative approaches to AI alignment
Descriptive vs. normative accounts of human decision-making
Key Terms
Preferentist approach
Rational choice theory
Expected utility theory
Flashcards
17 cardsRelated Questions
Which of the following does Tan Zhi-Xuan criticizes in "Micah Carroll, Matija Franklin & Hal Ashton, Beyond Preferences in AI Alignment"?
In Tan Zhi-Xuan's "Micah Carroll, Matija Franklin & Hal Ashton, Beyond Preferences in AI Alignment", Garrabrant criticizes which of the following?
In Tan Zhi-Xuan's "Micah Carroll, Matija Franklin & Hal Ashton, Beyond Preferences in AI Alignment", London, A. J. defines which of the following?
In Tan Zhi-Xuan's "Micah Carroll, Matija Franklin & Hal Ashton, Beyond Preferences in AI Alignment", Baker et al. develops which of the following?
In Tan Zhi-Xuan's "Micah Carroll, Matija Franklin & Hal Ashton, Beyond Preferences in AI Alignment", van Rooij et al. provides which of the following?
What is the main critique of the preferentist approach to AI alignment?