In the past few months, I thought a lot about the implications of non-causal decision theory. In addition to writing up my thoughts in a long paper that we plan to publish on the FRI website soon, I also prepared a presentation, which I delivered to some researchers at FHI and my colleagues at FRI/EAF. Below you can find a recording of the talk.

The slides are available here.

Given the original target audiences, the talk assumes prior knowledge of a few topics:

• Some decision theory
  • The prisoner’s dilemma
  • Newcomb-like problems and Douglas Hofstadter’s superrationality
    • Although it doesn’t cover cooperation/superrationality, I think Yudkowsky’s Newcomb’s Problem and Regret of Rationality is a good first exposition.
    • Paul Almond (2010): On Causation and Correlation, Part 1: Evidential decision theory is correct.
    • Douglas Hofstadter (1983): Dilemmas for Superrational Thinkers, Leading Up to a Luring Lottery. Scientific American, 248(6).
    • Acausal trade
    • If you want a lot of detail on this branch of decision theory, see Arif Ahmed’s book on Evidence, Decision, and Causality.
  • The orthogonality thesis
• Superintelligence. See, e.g.,
  • the very accessible two-part introduction (part 1, part 2) by Tim Urban on Wait But Why,
  • the more elaborate and academic Superintelligence by Nick Bostrom, and
  • Altruists Should Prioritize Artificial Intelligence by Lukas Gloor.
• The differentiation between consequentialism versus deontology and virtue ethics. Also consider effective altruism.
• On the notion of the multiverse or parallel universes, see Max Tegmark’s Parallel Universes.
• Gains from trade, see Brian Tomasik’s Gains from Trade through Compromise.