2 thoughts on “Acausal Trade slideshow”

1. I’m all for trying to find potential safety measures for AI arms race scenarios and I like the idea behind using linked decisions as a way to hold other agents accountable to future actions. I’m mostly curious what it looks like when it’s a bit more fleshed out.

   In slide 34-36 I think it would be useful to expand this scenario to include a concrete and plausible domain for these AIs to gain a strategic advantage in. What if we assume the strategic advantage was gaining market share over social media (Facebook and Google+). Maybe this would involve a strong understanding of both what type of product would be most desirable and how to market that product most effectively. This keeps with your premises that early deployment gains strategic advantage, whereas waiting has safety implications (safety implications could apply to protecting against an AI that can model people’s preferences in order to have a large amount of people spend all their time on these apps even when they didn’t want to).

   If Facebook and DeepMind’s AIs were launched nearly simultaneously, are you claiming that they should want to cooperate to not be undermined by the other, given that they would have more or less an equal chance of losing the AI race? Is this based on the idea that it’s worth it to be risk averse and cooperate rather than risk losing an AI race? If not, what reasons do they have for cooperating and acting as one? That’s the part that I think I require the most clarity on.

   Maybe another reason they could have for cooperating is capturing market share from other companies within the social media space, rather than risking losing all of it to their biggest competitor. It seems to me that this type of dynamic could be expected in almost all domains, because it’s hard to think of domains that have only two players versus having a large number of competing companies, or have barriers to entry that stop somebody from developing a technology that could help them gain market share.

   Scenario B
   If we assume that Facebook and Deepmind are the only two companies in the space of social media and they start with 50% of the market share each, then linked decisions and source code simulation may lead to coordinated actions where each AI assumes that defecting will hurt themselves in expectation. If they act as one, then I’m assuming they will end up again with 50% of the market share each, so they don’t have an incentivized to deploy earlier, making more room for safety research.

   Scenario X
   But what if they each started with 25% of the market share and the last 50% of the market share was split up amongst other companies. Presumably, they could coordinate to gain 50% of the market share each, but waiting would always increase the chance that some other player would develop and deploy AI within this space, or at least want to coordinate with Facebook and Deepmind to bring their market share down to 33%. I would argue that we can’t find any real world examples that resemble Scenario B, and that, given we will always find ourselves in something closer to Scenario X, we should expect that the same arms race dynamics apply, even with linked decisions, because the potential market share approaches 0% the longer you wait.

   Thanks for your comments and your posts!

   LikeLiked by 1 person

   February 28, 2019 at 20:29 Reply

2. Thanks Esam, that’s a good point–realistically, the world is sufficiently multipolar that even if the major powers coordinate to go slowly, there will be other players entering the game unexpectedly. I’m working on the next version of this now and I’ll take that into account somehow. Maybe I’ll ultimately need to abandon that example, for the reasons you give.

   LikeLike

   March 4, 2019 at 10:21 Reply