[computer-go] use for Monte Carlo on 19X19?

[Gunnar Farnebäck]

Don Dailey wrote:
> Lars,
> 
> If I do anything to CGOS it would be handicap games.  But I think your
> suggestion is sensible for Japanese scoring.    GnuGo won't score
> perfectly every time, but I understand it is rarely incorrect.        
> 
> Does anyone have statistics on how well GnuGo scores professional 19x19
> games?  

That depends on how difficult you make the problem. I have used Dave 
Dyer's test set of 623 scored professional 19x19 games, see 
http://www.andromeda.com/people/ddyer/go/scoring-games.html for 
information. As can be seen in the example on that page, those game 
records generally leave out all dame moves and even some moves required 
to finalize territories when it's clear how many points will be obtained 
there. Worse still, final endgame kos are frequently left out since it's 
assumed to be obvious who will win them.

With this in mind, the results for GNU Go 3.7.11 are 534 (85.7%) 
correctly scored, 79 (12.7%) off by one point, 5 off by 2 points, 2 off 
by 3 points, and 1 each off by 12, 34, and 38 points.

But mind you, GNU Go is trained on those games. It would certainly do 
worse on unseen games of a similar difficulty.

On the other hand, if the game records are complete up to and including 
dame filling, I would expect the error rate to be less than 1%, possibly 
much better.

/Gunnar