[computer-go] Bot ratings and strength

[Magnus Persson]

> My basic question is this: what makes some other similar programs so much
> stronger?  I read the Sensai library descriptions for AnchorMan and
> ControlBoy and see that they are very similar, yet do only 5,000
> simulations, and yet are much much stronger programs overall.  Does this
> difference come primarily from the benefits of keeping more of the tree in
> memory?  From better heuristics for selecting top-level and/or simulation
> moves?  I don't imagine anyone will have a completely definitive answer for
> this, but I am just at a loss at this point.  Any guidance would be most
> appreciated.

Does your program try to search deeper than 1 ply at all? If it does you might
have a search bug. My first version of Valkyria_UCT was designed to build a
tree in memory but after every simulation it assigned the same score to all
nodes inside the tree instead of alternating between WIN/LOSS. As a result it
worked correcly searching 1 ply but then it got stuck with best move and never
changed its mind. So even if the search did not work at all it appeared 
to play
well.

Debugging a go-program is crucial. My rule is this: "Everything that is 
computed
in a go program has to be visible to my eyes". My Debug window has a list of
strings and three grids where go-positions are displayed.

If I want to debug for example my evaluation in the end of the 
simulation I add
the following code.

Plot(GameState, 'Game end ' + IntToStr(score));
for all Pos on board: AddDot(Pos, Color(AssignedScoreAtPoint(Pos));

This creates a bitmap of the game associated with the string and stores a list
of plot commands that add black and white dots on the board.

Then I run the program for 1 second and the List of strings is filled 
with 'Game
End' strings. When I click such a string the associated position is 
showed with
all the dots. If there is a bug you will see it instantly. A common 
problems is
that the simulation was not played out to the end. I once had a bug where the
program refused to capture chains with one eye with 2 adjacent spaces.


If there was no bug there then I would plot every move of every 
simulation. One
might need to get throught a 1, 10 or 100 simulations, but since my user
interface takes care of all details I just scroll throught the list quickly.
The problem here would be that you only see moves played out but not the moves
that were not played. So I would plot all positions where the program did not
play a random move and add a red dot on that point.

A warning though. Using this debugging technique with random games often works
well since most of time all possible things that can happen to your code code
do happen. But, sometimes, there are bugs connected with some shape that is
very rare in a random game, but might be very natural and common in the games
actually played. These kind of bugs tend to A) crash the program every 100
games or so B) Make evaluation go up in a lost position and then suddenly drop
when the problematic shape disappears from the board or the search tree.

-Magnus