[computer-go] Making Java much faster
Chrilly
c.donninger at wavenet.at
Wed Nov 29 03:22:18 PST 2006
I am confused. In your paper you write "Orego is a Monte CarloGo programm
written in C++". Is Orego now in C++ or in Java or both?
The paper mentions the relative comparison of 2 versions. This is common
practice in the scientific literature, but it is a very poor choice if one
wants to measure the effect of a new method. The effects of changes is much
more pronounced than against another opponet. A method which is good against
the twin-brother must not be good against other opponents at all. Even
against other opponents it happens frequently that a method works quite well
against opponent A but it fails against B. Its relative easy to make a
version which crashes e.g. Rybka, but this version is poor against Fritz and
Shredder. The really difficult task is to find a combination which plays
good against all.
But its of course a good method for papers where the authors want to proove
how good their idea is. But it demonstrates the lack of competence of the
academic world for game-programming. Otherwise such experiments would not be
accepted as a proof for a concept. There is also Vincent Diepeveens law: In
a weak programm is every change an improvement. I do not know how good Orego
is, but playing e.g. against the top-3 programms would be a much better
experiment.
This remark is not against the Orego team which does a fine job to explain
their work. I like to read the Orego project "news".
Its against a very common and bad practice in papers. One can even get an
award for the best paper of the year and become a standard reference with
this poor methodology. In my "Null-Move" paper I did the same. The Null-Move
Version of Nimzo was running on a 386, the Non-Null-Move Version on a 486
and the result was about equal. My conclusion was: The Null-Move is worth
one hardware generation. At that time I was not really aware of the problem
and the bad comparision was not on purpose. But the reviewer/editor of the
ICCA-Journal should have insisted on better experiments. The only
"improvements" he made was changing the original title from "Null move and
deep search: Selective search heuristics for. stupid chess programs" to
"obtuse chess programms". I know what a stupid programm is, but I do not
know till today the meaning of "obtuse".
Chrilly
More information about the computer-go
mailing list