[computer-go] Interesting problem

[Nick Apperson]

This is an interesting problem.  It seems to me that the reality is that
when you are talking about non-ideal play, ranking systems aren't linear.
Program A could beat B which could beat C which could beat A.  How would you
rank those?  Clearly there is going to have to be some degree of arbitrary
selection.  I propose convenience as the best reason for picking one anchor
over another.  I think a completely random player is the only other choice
from a theoretically perfect player that doesn't have arbitrariness.  But,
by defining players relative to that anchor, we would really be measuring
how effectively a program exploits a weak player rather than how good the
program is.

It is my opinion that it is more important to have a relative ranking system
than an absolute system.

- Nick

On 12/28/06, Aloril <aloril at iki.fi> wrote:
>
> On Wed, 2006-12-27 at 21:34 -0500, Don Dailey wrote:
> > I'm having an interesting problem - my hope is to set
> > a random legal move making player (who doesn't fill
> > 1 point eyes) at ELO zero.
> >
> > I feel this would define a nice standard that is
> > easy to reproduce and verify experimentally and
> > at least would be a known quantity even 100 years
> > from now.
> >
> > But I'm having a difficult time creating players
> > who are slightly better than this at 19x19.  I need
> > incrementally better and better players.
>
> I suspect this is quite hard problem. On 9x9 we have some of this and I
> suspect even there "do not fill eyes random" (PythonBrown) has not yet
> settled (maybe 100-200 ELO overrated). Probably too few weak players ;-)
> On 19x19 I think problem is much harder and required amount of
> intermediate players is much bigger. I'm of course interested in hearing
> your experimentation results. Maybe I'm wrong and it is actually
> feasible.
>
> My vague recollection was that random player is maybe 200 kuy, "do not
> fill eyes" adds 60 stones, atari detection adds about 20-30 stones,
> idiotbot is maybe 100 kuy, weakbot50k maybe 50 kuy. However differences
> between computers tend to be much bigger than when they play against
> humans! For example GNU Go 2.0 can give Liberty 1.0 easily 9 stones and
> win more than 50% of games (based on few ha9 test games), but at KGS
> they are rated at 10k and 14k. Even WeakBot50k is rated at 20k while
> latest GNU Go rated at 6k can give it numerous handicap stones (much
> more than 14 stones, I think it's more than 40 stones).
>
> Here is my proposal for anchor player: Use GNU Go 3.7.10 (or any enough
> recent with super-ko support) at level 0 and use well defined
> randomization on top of moves it returns. Ie. ask all_move_values (lists
> only moves that gnugo considers positive) and add remaining moves and
> then apply slight randomization so that it still plays close to original
> strength but is much more unpredictable than GNU Go.
>
> Example program (by blubb and me):
> http://londerings.cvs.sourceforge.net/londerings/go/gtpTuner/
>
> Reasons:
> - reasonably strong, no need for huge amount of intermediate players
> - source code available
> - well known entity
> - with some randomization should be unpredictable
>
> I suspect that GNU Go without randomization is too predictable. This is
> very clearly case on 9x9 board and possibly on 19x19 too.
>
> --
> Aloril <aloril at iki.fi>
> _______________________________________________
> computer-go mailing list
> computer-go at computer-go.org
> http://www.computer-go.org/mailman/listinfo/computer-go/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://computer-go.org/pipermail/computer-go/attachments/20061228/604d6758/attachment.htm