[Computer-go] Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

Brian Sheppard sheppardco at aol.com
Wed Dec 6 19:01:03 PST 2017

I see the same dynamics that you do, Darren. The 400-game match always has some probability of being won by the challenger. It is just much more likely if the challenger is stronger than the champion.

-----Original Message-----
From: Computer-go [mailto:computer-go-bounces at computer-go.org] On Behalf Of Darren Cook
Sent: Wednesday, December 6, 2017 7:55 PM
To: computer-go at computer-go.org
Subject: Re: [Computer-go] Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

>> One of the changes they made (bottom of p.3) was to continuously 
>> update the neural net, rather than require a new network to beat it 
>> 55% of the time to be used. (That struck me as strange at the time, 
>> when reading the AlphaGoZero paper - why not just >50%?)

Gian wrote:
> I read that as a simple way of establishing confidence that the result 
> was statistically significant > 0. (+35 Elo over 400 games...

Brian Sheppard also:
> Requiring a margin > 55% is a defense against a random result. A 55% 
> score in a 400-game match is 2 sigma.

Good point. That makes sense.

But (where A is best so far, and B is the newer network) in A vs. B, if B wins 50.1%, there is a slightly greater than 50-50 chance that B is better than A. In the extreme case of 54.9% win rate there is something like a 94%-6% chance (?) that B is better, but they still throw B away.

If B just got lucky, and A was better, well the next generation is just more likely to de-throne B, so long-term you won't lose much.

On the other hand, at very strong levels, this might prevent improvement, as a jump to 55% win rate in just one generation sounds unlikely to happen. (Did I understand that right? As B is thrown away, and A continues to be used, there is only that one generation within which to improve on it, each time?)

Computer-go mailing list
Computer-go at computer-go.org

More information about the Computer-go mailing list