[Computer-go] CGOS source on github

Brian Lee brian.kihoon.lee at gmail.com
Sat Jan 23 05:58:02 PST 2021


DeepMind has published a number of papers on how to stabilize RL strategies
in a landscape of nontransitive cycles. See
https://papers.nips.cc/paper/2018/file/cdf1035c34ec380218a8cc9a43d438f9-Paper.pdf

I haven't fully digested the paper, but what I'm getting from it is that if
you want your evaluation environment to be more independent of the
population of agents that you're evaluating against, you should first
compute a max-entropy Nash equilibrium of agents, and evaluate against this
equilibrium distribution.

To give a concrete example from the paper, imagine the CRPSS - the Computer
Rock Paper Scissors Server. Imagine there are currently 4 bots connected: a
Rock-only bot, a Paper-only bot, and two Scissor-only bots. The max-entropy
Nash equilibrium is 1/3, 1/3, 1/6, 1/6. So the duplicated Scissor bots are
naturally detected and their impact on the rating distribution is negated.
With CGOS's current evaluation scheme, the Rock bot would appear to have a
higher Elo score, because it has more opportunities to beat up on the two
Scissors bots.

The paper also proposes a vector extension to Elo that can better predict
outcomes under these nontransitive cycles.

Given that what we have is (at a macro level) duplication of various bot
lineages, and (at a micro level) rock-paper-scissors relationships between
bots in sharp openings, this paper seems quite relevant.


On Sat, Jan 23, 2021 at 5:34 AM Darren Cook <darren at dcook.org> wrote:

> > ladders, not just liberties. In that case, yes! If you outright tell the
> > neural net as an input whether each ladder works or not (doing a short
> > tactical search to determine this), or something equivalent to it, then
> the
> > net will definitely make use of that information, ...
>
> Each convolutional layer should spread the information across the board.
> I think alpha zero used 20 layers? So even 3x3 filters would tell you
> about the whole board - though the signal from the opposite corner of
> the board might end up a bit weak.
>
> I think we can assume it is doing that successfully, because otherwise
> we'd hear about it losing lots of games in ladders.
>
> > something the first version of AlphaGo did (before they tried to make it
> > "zero") and something that many other bots do as well. But Leela Zero and
> > ELF do not do this, because of attempting to remain "zero", ...
>
> I know that zero-ness was very important to DeepMind, but I thought the
> open source dedicated go bots that have copied it did so because AlphaGo
> Zero was stronger than AlphaGo Master after 21-40 days of training.
> I.e. in the rarefied atmosphere of super-human play that starter package
> of human expert knowledge was considered a weight around its neck.
>
> BTW, I agree that feeding the results of tactical search in would make
> stronger programs, all else being equal. But it is branching code, so
> much slower to parallelize.
>
> Darren
> _______________________________________________
> Computer-go mailing list
> Computer-go at computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20210123/fb38ba53/attachment.htm>


More information about the Computer-go mailing list