[computer-go] Two-headed problems

[Don Dailey]

I agree with Magnus,

In general,  you cannot expect too much from problem sets if you expect
them to be an indication of program strength.

They are good for testing and tuning various aspects of your program
however.   For instance a few problem to make sure it handles seki well,
or to give an indication of whether it can handle ladders etc.    But
this doesn't translate very well in general to strength.

Even in chess, the correlation is weaker than one might hope, despite
the erroneous opinion of many who believe chess is mostly about
tactics.  

- Don


On Thu, 2006-09-07 at 19:45 +0200, Magnus Persson wrote:
> Hi!
> 
> If you tune the parameters to artificial situations, it will not generalize to
> real playing strength I am afraid.
> 
> These positions may be interesting however to test what an engine can 
> or cannot
> do. Then that might lead to more programming or bug hunting which is probably
> more efficent than parameter tuning.
> 
> I tend to collect interesting positions from real games, where my programs did
> something bad or missed a great opportunity.
> 
> Tuning of parameters I do with long matches 150-300 games against 
> gnugo. But that is also probably bad in principle because it just makes 
> my programs
> stronger against Gnugo at the prize of becoming weaker against other programs.
> 
> -Magnus
> 
> Quoting Peter Drake <drake at lclark.edu>:
> 
> > Rather than run all of these time-consuming self--play tournaments  
> > when selecting parameters for Orego, I'd like to give one or more  
> > short problems and see how often (or how quickly) Orego finds the  
> > correct answer.
> 
> _______________________________________________
> computer-go mailing list
> computer-go at computer-go.org
> http://www.computer-go.org/mailman/listinfo/computer-go/