Naturalized agency or space-time embedded intelligence is the problem of designing agents that are embedded in the environment, i.e. agents that can be destroyed, whose source code can be read, etc. Here are some introductions to the topic:
- Nate Soares (2015): Formalizing Two Problems of Realistic World-Models.
- Laurent Orseau and Mark Ring (2012): Space-Time Embedded Intelligence.
- Nate Soares and Benya Fallenstein (2017): Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda, section 2.
- Nate Soares and Benja Fallenstein (2014): Botworld 1.0, section 1.
The Universe from an Intentional Stance 