[computer-go] Making Java much faster

[Don Dailey]

I have heard this many times - but it doesn't always apply.   In fact I
have heard that IMPROVEMENTS always look better against your
twin-brother but if that were true, I would always want to test against
my twin since it makes improvements stand out.  It's hard to measure
small improvements so this would be like using a microscope to help me.

But unfortunately a change can help you beat your twin but make the
program worse against other opponents - but I have only occasionally
seen this be a big factor although I admit it does happen.   I just
think the effect is exaggerated by people.   A general rule is that if
you are better against Joe, you are "probably" better against Fred.

I'm doing some experiments with automated tuning (with my chess
program.)  I have had a lot of success with this - in self test games I
increased my win percentage significantly over a lot of games.   When I
play Toga, I actually achieved an even HIGHER win percentage.   

But I never trust an "improvement" without testing against a variety of
opponents, at the minimum a self-test and a test against a different
opponent.

- Don
   




On Wed, 2006-11-29 at 12:22 +0100, Chrilly wrote:
> The paper mentions the relative comparison of 2 versions. This is
> common 
> practice in the scientific literature, but it is a very poor choice if
> one 
> wants to measure the effect of a new method. The effects of changes is
> much 
> more pronounced than against another opponet. A method which is good
> against 
> the twin-brother must not be good against other opponents at all.
> Even 
> against other opponents it happens frequently that a method works
> quite well 
> against opponent A but it fails against B. Its relative easy to make
> a 
> version which crashes e.g. Rybka, but this version is poor against
> Fritz and 
> Shredder. The really difficult task is to find a combination which
> plays 
> good against all.