[computer-go] New engine? From a Chess programmer perspective.

[Don Dailey]

Robert Jasiek wrote:
> Don Dailey wrote:
>> Just a few years ago it was widely held that computers will not reach
>> Dan level "in my lifetime" even in 9x9 Go.    When it happened in 9x9
>> go,  it was not accepted - the day it happened passed us by and nobody
>> noticed it.     It's probably still not common knowledge and it will
>> take time for it to be generally believed.
>
> What is the basis of the claim that a program has reached a certain
> human rank level?
>
> There should be systematic tests. Let it play against many humans. Let
> it enter human tournaments. Use a meaningful evaluation context.
Yes, there should be.   Right now it's very informal and anecdotal.   
Strictly speaking it is not possible to evaluate any player - the ELO
system is subject to the laws of probability.    So no matter how many
games you play people can claim it's a fluke and blame the playing
conditions or other factors.      That's the point I was really trying
to make.    People will be resistant to the idea no matter what - so the
best you can do is supply overwhelming empirical evidence and then if
some want to be unreasonable they are easily identified - at least by
reasonable people.

>
> For some such evaluation, let me refer to a useful handicap system for
> 9x9, which has been used in some European 9x9 "Championships": For the
> first 10 rank differences (0, 1,.., 9) decrease the komi from 6.5 for
> an even game by 1 point per extra rank. (Komi can become negative.) -
> OC, I  prefer to see even games. OTOH, until the program rank is well
> known, it may be suitable to let a simgle human (the more humans the
> better) play until the handicap becomes stable.
>
For 9x9 ELO works better.     For 19x19 it's less clear cut.    The
handicap system appears to be a good system at 19x19 and has the very
nice merit of allowing grossly mismatched players to compete.       I
think the two systems can be married by adding a fixed offset per stone
handicap to your ELO.

- Don