[computer-go] Rapid action value estimation

[Benjamin Teuber]

Yeah, this is one of the many pitfalls when trying to implement this stuff..
A clean reference implementation of all important algorithms in an
easy-to-read language like e.g. python or ruby (don't think you guys want to
read my Lisp-stuff *gg*) would really be neat (and damned slow, but that's
another issue - I just don't want to learn a new algorithm by studying
highly optimized C code).

It also makes a difference whether you represent win/loss by 1/0 or 1/-1 (I
did the latter, as I believe this is used for the UCT-formula in MoGo)
I initialized uctVisits and raveVisits both to 1, uctWins to -1 (this means
unexplored nodes are very bad for the next player and thus very good for the
player before, so they will automatically be played first) and raveWins to 1
(I don't think the value here matters a lot as all moves will be visited
more or less the same number of times).
To get some randomization, you might need to add small random numbers to
everything or something like that (I didn't need that as I used some
previously calculated AMAF stats as a prior).

Hope that helps a bit - I would also be very interested in other approaches
(is there something about initializing in the MoGo papers?)

Cheers,
Ben

On 11/5/07, Jason House <jason.james.house at gmail.com> wrote:

> I implemented this yesterday.  In doing so, I realized I didn't know the
> proper way to initialize new leaves in the UCT tree.  MoGo papers seem to
> talk about a progression from always picking an unexplored leaf (AKA using
> infinity for the upper confidence bound), to "first play urgency" (using a
> fixed ucb for new leaves), to using patterns.
>
> I don't yet have patterns and am curious what is recommended.  If no real
> sims exist for a child, I use the first play urgency of 110%.  If no amaf
> sims exist for a child, I pick it for immediate simulation.
>
> Have any techniques (without patterns) proven more effective?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://computer-go.org/pipermail/computer-go/attachments/20071105/c46cfcbe/attachment.htm