In the past few months, I thought a lot about the implications of non-causal decision theory. In addition to writing up my thoughts in a long paper that we plan to publish on the FRI website soon, I also prepared a presentation, which I delivered to some researchers at FHI and my colleagues at FRI/EAF. Below you can find a recording of the talk.
The slides are available here.
Given the original target audiences, the talk assumes prior knowledge of a few topics:
- Some decision theory
- The prisoner’s dilemma
- Newcomb-like problems and Douglas Hofstadter’s superrationality
- Although it doesn’t cover cooperation/superrationality, I think Yudkowsky’s Newcomb’s Problem and Regret of Rationality is a good first exposition.
- Paul Almond (2010): On Causation and Correlation, Part 1: Evidential decision theory is correct.
- Douglas Hofstadter (1983): Dilemmas for Superrational Thinkers, Leading Up to a Luring Lottery. Scientific American, 248(6).
- Acausal trade
- If you want a lot of detail on this branch of decision theory, see Arif Ahmed’s book on Evidence, Decision, and Causality.
- The orthogonality thesis
- Superintelligence. See, e.g.,
- the very accessible two-part introduction (part 1, part 2) by Tim Urban on Wait But Why,
- the more elaborate and academic Superintelligence by Nick Bostrom, and
- Altruists Should Prioritize Artificial Intelligence by Lukas Gloor.
- The differentiation between consequentialism versus deontology and virtue ethics. Also consider effective altruism.
- On the notion of the multiverse or parallel universes, see Max Tegmark’s Parallel Universes.
- Gains from trade, see Brian Tomasik’s Gains from Trade through Compromise.