[computer-go] Some ideas how to make strong heavy playouts

[Magnus Persson]

Quoting Don Dailey <drdailey at cox.net>:
>
> I have a response to this comment:
>
>     Still I think predicting the best moves is very important in the
>     tree part, but this may be much less important in the playouts, and
>     perhaps even detrimental as some people have experienced.
>
> A class of "bad" moves is moves on the edge of the board when nothing
> else is nearby.  In other words, the candidate edge placement doesn't
> "touch" any other stones of either color.     In my experiments I found
> that it strengthens the program significantly to avoid trying these
> moves in the tree search (I call them veto patterns even though they are
> not necessarily veto'd permanently.)

Sofar I have not tried removing those moves in the tree only. I need  
to try that.

> But I also discovered that there seems to be no benefit whatsoever in
> removing them from the play-outs.    I have no real explanation for
> this.   But it does tell me that the play-outs are very different in
> nature from the tree - you cannot just use the same algorithms for
> selection and prioritizing moves.

My experience is that sometimes I get really bad interactions between  
special patterns in my playouts and plays on the first and second  
line. For example, if ladders are implemented then a move on the first  
line will be able to capture a contact move to the side of it. Thus I  
had situations where I think removing those moves made a positve  
difference, because the program liked those bad moves too much. But an  
alternative solution is also to prune bad moves, for example playing  
next to a stone of the first line allowing a ladder is perhaps even  
worse than the first move. So my program solves these kind of problems  
in a very adhoc multiple ways so it is hard to tell if my eperineces  
and your experiences here are universal or just specific to certain  
programs as a whole.

Still I recently had the idea that perhaps AMAF works best if in all  
moves are played because then each move played in the playouts are  
tested under a larger variety of situations. But I do not remember now  
if I tested it properly.

What I did do recently was to stop pruning moves in the trees and  
allowed all legal moves. This dropped the winrate of 1500 playouts vs  
gnugo with 10%, it seemed to have a lot of temporary hallucinations  
early in search where a lot of really bad moves were searched perhaps  
50-200 times before finding more decent moves to search.

I planned to make a version that make some soft pruning allowing them  
to be searched later so that tree will have full width near the root  
and be strongly pruned below. I would be happy if I could do that and  
keep the playing strength becuase then my program would at least in  
principle be able to play perfectly. Right now the pruning is to  
aggressive for that to be true.




> I think I like your theory that eliminating noise is a good thing and
> probably the primary thing.
>
> Your last comment is what I am interested in the most,  and what I have
> spent quite some time looking at.  I call it  "learning on the
> fly."       The use of patterns is acceptable in this setting if those
> patterns are learned during the game.     The basic important
> observation is that most of the things learned during the game is
> relearned over and over and over and over ....
>
> All-moves-as-first is a form of this learning, but very brittle of
> course.     And I don't have a problem with imposing knowledge from the
> outside if it's used only as a "kick start" - a way to overcome the
> initial learning inertia,  but not imposed in any absolute way.

I agree also here but I have no really good practical ideas, just  
loose feeling on what it should be like. But every time I try to think  
in more concrete terms I never get anywhere...

-Magnus