[Computer-go] A Regression test set for exploring some limitations of current MCTS programs in Go
ajahuang at gmail.com
Mon May 14 19:39:31 PDT 2012
Martin Mueller and I are writing a paper about exploring some limitations
of current MCTS programs in Go. For this purpose we have carefully designed
a regression test set which consists of 20 seki and 15 two-safe-groups
cases on 9x9 board. If you are interested, it is available at
We will appreciate if you would like to run your program over our
regression test and send us the results for our publication.
It's easy to run your program through these positions (.sgf). Mainly, the
script run.sh under /utility is able to run a given program for a given
regression test file (.tst) and produce the result in a related html file.
For example, for the seki test you can simply type
./run.sh -p PATH_TO_PROGRAM -t g_seki_moves.tst
1. Your program must support the command sg_compare_float for the
two-safe-groups test. If it doesn't support reg_genmove then the test file
g_seki_moves.tst is good to use which performs genmove instead.
2. On Windows platform, you will be able to execute 'run.sh' directly at
the command prompt after cygwin is installed.
3. If your program doesn't support the GTP command 'loadsgf', gogui-adapter
is able to translate 'loadsgf' into a sequence of 'play' commands. The file
gogui-adapter.jar under /utility is good to use because Markus has fixed
some bugs for us, see
Under /experimental results, there are results of several programs such as
Fuego (tilburg version), pachi, ManyFaces and GnuGo. We thank David for
providing us the valuable results of ManyFaces. The test set is really not
easy because these programs all failed in many cases.
Questions are very welcome. If you find any error in the test set please
inform us. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Computer-go