[Computer-go] Having an "estimated winrate" on the AGA pro game broadcasts

Adrian Petrescu apetresc at gmail.com
Wed Aug 31 17:43:35 PDT 2016


Hey Andrew!

Since talking about it with you over the weekend, I've been working on
something pretty similar: a fishnet <https://github.com/niklasf/fishnet> clone
that basically installs the latest pachi (pre-configured with a pattern
table, the 54% DCNN network, an opening book, etc) onto a willing
participant's machine, and enters it into a pool of workers that a central
server can farm position/SGF evaluations to (which includes winrate, among
other things) through an API.

The idea is to make the API available to servers like OGS or community
sites like GoKibitz so that users can request lichess-like computer
annotation, so at the moment there's no collaboration between workers; each
one gets a single SGF as a unit of work and returns it to the server
independently. But Pachi *does* have a distributed mode that might allow
multiple workers to work on the same position, to increase the quality. But
I haven't tried it yet... I'm not sure how well it'll react to a wide
network like this, my guess it was designed for an HPC cluster.

But even with a single beefy machine, a well-tuned Pachi can give
reasonable win probabilities starting from the mid-game on. Even if it's
wrong, it can be a conversation-starter for the broadcasters, I suppose :)

Let me know if you're interested in giving it a spin. I'm certainly
interested in supporting this use-case too.

Cheers!
-Adrian

On Wed, Aug 31, 2016 at 8:24 PM, Dan Schmidt <dfan at dfan.org> wrote:

> Hi Andrew, I enjoy the AGA broadcasts. I am mostly just an observer on
> this list rather than a go programming expert, but I do have a couple of
> things to say on this topic:
>
> 1. This is done all the time on chess broadcasts these days, and I hate
> it. I want to know what a strong player thinks is going on, not a computer.
> This is a bigger "problem" with chess than go because computers are now
> hugely stronger than the pro commentators, so the commentators become
> largely reduced to interpreting the computer output rather than getting
> across their own human ideas. Of course, a) this is less of an issue right
> now as go programs (except for AlphaGo, which is private) are not quite pro
> strength yet, b) top-level go games have less volatile swings than chess
> games, and perhaps most importantly c) my opinion may be in the minority.
>
> 2. Win rate is tough. Many programs can produce the win rate of their
> Monte Carlo Tree Search, but because that involves looking at lots of
> suboptimal moves on both sides, the reported win rate is closer to 50% than
> it should be. A reported win rate of 60% (in the futures explored by MCTS)
> probably means at least 80% in reality, assuming high-level play on both
> sides.
>
> Crazy Stone reports an evaluation (like B+3.5) and confidence though it
> cautions against taking it too seriously. My assumption is that it's
> something like the median result, and standard deviation, of all the MCTS
> playouts. I find this more useful than the win rate it provides.
>
> Dan
>
>
> On Wed, Aug 31, 2016 at 6:41 PM, Jackson, Andrew <andrew.jackson at usgo.org>
> wrote:
>
>> Hello all!  Long time lurker, first time poster:  I'm Andrew Jackson, i
>> volunteer w/ the AGA in a number of capacities.
>>
>> Most recently, i've been putting together live broadcasts of games --
>> professional games from CJK, and also games from AGA events [1]
>>
>> These broadcasts feature professional commentary to help amateurs
>> understand what's going on, but the number one question we get in the chat
>> during the broadcasts is invariably "who's winning?"  I was hoping
>> computer-go could provide some suggestions on how we might run the
>> game-in-progress through an engine to get an 'estimated winrate' -- or
>> better yet, a graph -- that could be overlaid in the corner of the screen.
>>
>> Which brings me to the computer-go mailing list :)  It seems like someone
>> on this mailing list would probably have some good ideas about how this
>> might be accomplished :)  What do you think?
>>
>>
>> Here are some more details:
>>
>> The game sgf is kept mirrored on KGS by a human transcriber w/ permission
>> from the KBA.
>>
>> The overlay for these broadcasts is already rendered as a local webserver
>> hosting a simple html shell holding the graphics; my thought was that this
>> server could easily make requests to another service somewhere.  That
>> service would be tracking the game and able to send back some json
>> describing the engine's view of the game, its confidence, etc.  We could
>> then plot with d3.js right on the overlay.
>>
>> The service would need to run somewhere else, as the computer streaming
>> the game is already using all the CPU for transcoding & pushing the video.
>>
>> Given general overview of the goal & constraints, I ask you:
>>
>>  - Does this sound plausible?
>>  - Any ideas for an engine that would be easy to hook this into?
>>  - Are bots capable of spectating games on KGS?
>>
>> Thanks in advance for any help you might provide!
>> -Andrew
>>
>>
>> _______________________________________________
>> Computer-go mailing list
>> Computer-go at computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
> _______________________________________________
> Computer-go mailing list
> Computer-go at computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20160831/ed9d7e6b/attachment.html>


More information about the Computer-go mailing list