[Computer-go] A Regression test set for exploring some limitations of current MCTS programs in Go

Lars Schäfers slars at upb.de
Tue May 15 03:34:32 PDT 2012


thanks for the regression test set. I will make a run with Gomorra.

Can you give some details to the sg_compare_float command? What are the
parameters, what should it return..

As I couldn't find something in the mail I guess there are no
restrictions in terms of time or number of simulations to use.

Best wishes,

On Tue, 2012-05-15 at 02:39 +0000, Aja Huang wrote:
> Dear all,
> Martin Mueller and I are writing a paper about exploring some
> limitations of current MCTS programs in Go. For this purpose we have
> carefully designed a regression test set which consists of 20 seki and
> 15 two-safe-groups cases on 9x9 board. If you are interested, it is
> available at
> http://webdocs.cs.ualberta.ca/~mmueller/ps/seki-and-two-safe-groups-regression-test.zip
> We will appreciate if you would like to run your program over our
> regression test and send us the results for our publication. 
> It's easy to run your program through these positions (.sgf). Mainly,
> the script run.sh under /utility is able to run a given program for a
> given regression test file (.tst) and produce the result in a related
> html file. For example, for the seki test you can simply type
> ./run.sh -p PATH_TO_PROGRAM -t g_seki_moves.tst
> Some notes:
> 1. Your program must support the command sg_compare_float for the
> two-safe-groups test. If it doesn't support reg_genmove then the test
> file g_seki_moves.tst is good to use which performs genmove instead.
> 2. On Windows platform, you will be able to execute 'run.sh' directly
> at the command prompt after cygwin is installed.
> 3. If your program doesn't support the GTP command 'loadsgf',
> gogui-adapter is able to translate 'loadsgf' into a sequence of 'play'
> commands. The file gogui-adapter.jar under /utility is good to use
> because Markus has fixed some bugs for us, see
> https://sourceforge.net/tracker/?func=detail&aid=3522401&group_id=59117&atid=489964
> https://sourceforge.net/tracker/?func=detail&aid=3519829&group_id=59117&atid=489964
> Under /experimental results, there are results of several programs
> such as Fuego (tilburg version), pachi, ManyFaces and GnuGo. We thank
> David for providing us the valuable results of ManyFaces. The test set
> is really not easy because these programs all failed in many cases.
> Questions are very welcome. If you find any error in the test set
> please inform us. Thanks.
> Best regards,
> Aja

More information about the Computer-go mailing list