[computer-go] CGOS 5 minute server

[Rémi Coulom]

Christian Nilsson wrote:
> Hello all,
>
> I've recently unleashed a few versions of my program "Haste" on CGOS.
> Seeing their ratings sparked a bit of curiosity about the relative
> ratings they achieved. In self-play ( thousands of games ) they differ
> by more than twice as many ELO-points as on the server.
I don't test with self play, but I am not very surprised by the 
difference you observe.
>
> I know it's bad to use self-play as a measure of improvement... but
> I'd still like to know if anyone else is seeing this rather large
> difference in self-play vs. CGOS. After all, most programs on CGOS
> appear to be using MonteCarlo/UCT with either light, medium or heavy
> playouts. Making them similar in many ways. 
The presence of a few GNU Go's on CGOS moderates this phenomenon. For 
instance, you can notice that CrazyStone-Fast has a score of 23/23 
against Fatman (1800) and 9/17 against GNU Go 3.7.9 (1814). The reverse 
is true for weaker bots. For instance Haste-10k-pure has 0/21 against 
FatMan and  4/21 against GNU. These number may not be extremely 
significant, but I believe they indicate a strong tendency.

That makes GNU Go a very good sparring partner for MC program: whatever 
you do, there is little risk of overfitting GNU, because it works in 
such a different way.

Rémi