[computer-go] Where and How to Test the Strong Programs?

[Rémi Coulom]

Don Dailey wrote:
> I don't really know what you mean by "one-dimensional."   My
> understanding of playing strength is that it's not one-dimensional
> meaning that it is foiled by in-transitivities between players with
> different styles.    You may be able to beat me,  but I might be able to
> beat someone that you cannot.      If that's what you are saying how
> does the kyu/dan system  handle it that makes it superior to ELO for
> predicting who will win?    Is there some mechanism that measures
> playing styles?    I don't see this.  
>   

That's what I am saying. The kyu/dan system does not handle it better.

> What I THINK you mean is that the gap between various GO ranks is not
> consistent in ELO terms.   In other words there is no single constant
> that works in my simple formula.     I definitely think this is probably
> the case but surely it can be easily handled by a slightly more
> sophisticated formula that "fits the curve."
>   

This is not what I meant, but I agree.

What I mean is that if human player H beats computer C1 65% of the time, 
and computer C2 also beats computer C1 65% of the time, then I would 
expect that H would be stronger than C2, especially if both C1 and C2 
are MC programs. If it is the case, then it would make it difficult to 
compare human scale to computer scale. But that is just my intuition.

For instance, against computers, I estimate that Crazy Stone improved 
about 3 stones between this summer and now. But it clearly did not 
improve 3 stones on KGS. I vaguely remember that Sylvain also noticed 
that MoGo could beat GNU go with a 4-stone handicap, but was only 2 
stones stronger than GNU on KGS.

Rémi