[computer-go] 19x19 CGOS

[Hideki Kato]

Hi all,

Jeff Nowakowski: <1193580228.3062.10.camel at acer-debian>:
>On Sun, 2007-10-28 at 11:54 +0100, Edward de Grijs wrote:
>> 
>>  
>> Hi all,
>>  
>> For CGOS 19x19 I prefer a short time control (10min/game) because:
>>  
>> 1) More quickly a more accurate rating can be established.
>
>I agree with Don.  10 minutes sudden death is brutally short for 19x19.
>You are limiting the pool and strength of programs available for CGOS.
>
>If all you want is a quick and dirty rating for minor updates, why don't
>you just run your program against Gnu Go and/or MoGo at fast time
>settings on your own machine?  Then when you think you have a stable and
>significant improvement, run your program on CGOS for a beefier test?
>This is how MoGo achieved dominance in 9x9.

We need thousands of games to get a few percent of standard deviation 
on both 9x9 and 19x19.  So, of course I do what you wrote.  When a 
game on cgos takes about one hour, a handred games take a handred 
hours, ie, four days.  When I want to know my program's winning 
rate against a paticular program that I don't have in hand, it takes 
four days times the number of programs running on cgos at an average.  
So, it takes a few weeks in total which is tooooooooooo long for me.

In constrast, I can guess my program's scalability by local 
competitions against GNU Go and/or MoGo.

About fairness, as classical programs including GNU Go and ManyFaces 
need about ten minutes for their best performace, why do you give 
other (Monte Carlo) programs thirty minutes?

I argue ten or fifteen minutes setting is enough and better for 
many developers than thirty minutes.

-Hideki

>-Jeff
>
>_______________________________________________
>computer-go mailing list
>computer-go at computer-go.org
>http://www.computer-go.org/mailman/listinfo/computer-go/
--
gg at nue.ci.i.u-tokyo.ac.jp (Kato)