[Computer-go] algorithm quality assessment

Don Dailey dailey.don at gmail.com
Wed Jun 12 12:46:40 PDT 2013


On Wed, Jun 12, 2013 at 3:33 PM, Oleg Barmin <j2ee_designer at mail.ru> wrote:

> > For quality assessment, play many games against one or more reference
> opponents.
> It's difficult to assament algorithm with a game against humans. The game
> is young and there are no recognized masters at the moment. So it's very
> hard to find human-opponent with a really good game skills.
>
> >  With card games you can get some serious intransitivity,  rocks, paper,
> scissors type of stuff.
> The aim of this game is to max your scores. Each turn you need to select
> one of three choices. Each choice has an expectation value of your scores.
> Optimal strategy here is to select a choice with max expectation value. But
> it will take a years to calculate an expectation value at the start of the
> game. So the game has no such intransitivity as rocks, paper, scissors.
>

There are clearly  in-transitives in any matchup with computer players (or
human players.)    This is true even in games of perfect information where
different players may have different notions of how the game should be
played and this is despite the fact that perfect strategy exists.    In
card games, which are not games of perfect information,  there exists many
more opportunities to vary your strategy based on opponent modeling.    For
example you can play top level chess without caring anything about your
opponent but you cannot play top level poker that way.

So I am quite certain that in-transitivity will play a much bigger role in
card games between computer players.

I'm not sure you understand transitivity in the game playing sense.   It's
the problem that you can create a program A that might easily beat program
B.   Program B might easily beat program C.    And yet program C can easily
beat program A.     You would expect that A should easily beat C since it
can beat B and B can beat C.

This is a very minor problem in chess,  but in poker (for example) where
your strategy is highly dependent on the strengths and weaknesses of your
opponent it is likely to be a major issue.     I think that is why David
Fotland gave his caution.

Don






> At the last turns we can make a complete choice enumeration and calculate
> an exact scores expectation value ( does go algorithms use the same
> technique? ) . It's not the way for the first half of the game. But the
> first half is more important.
>
> Oleg
>
>
> Среда, 12 июня 2013, 14:24 -04:00 от Don Dailey <dailey.don at gmail.com>:
>
>
>
> On Wed, Jun 12, 2013 at 11:30 AM, David Fotland <fotland at smart-games.com<https://e.mail.ru/sentmsg?mailto=mailto%3afotland@smart%2dgames.com>
> > wrote:
>
> For quality assessment, play many games against one or more reference
> opponents.
>
>
> Especially for a game that is not a game of perfect information such as go
> or chess.   With card games you can get some serious intransitivity,
>  rocks, paper, scissors type of stuff.
>
> Don
>
>
>
>
> ****
>
> ** **
>
> David****
>
> ** **
>
> *From:* computer-go-bounces at dvandva.org<https://e.mail.ru/sentmsg?mailto=mailto%3acomputer%2dgo%2dbounces@dvandva.org>[mailto:
> computer-go-bounces at dvandva.org<https://e.mail.ru/sentmsg?mailto=mailto%3acomputer%2dgo%2dbounces@dvandva.org>]
> *On Behalf Of *Oleg Barmin
> *Sent:* Wednesday, June 12, 2013 8:02 AM
> *To:* computer-go at dvandva.org<https://e.mail.ru/sentmsg?mailto=mailto%3acomputer%2dgo@dvandva.org>
> *Subject:* [Computer-go] algorithm quality assessment****
>
> ** **
>
> Hi, everybody,****
>
> I am working at the development of a cards game algorithm using MCTS.
> Technically, the game model is expect minmax tree search, where direct
> search takes up too much time, that is why I decided to use MCTS.****
>
> The issue of using MCST, like any other approximation algorithm is its
> quality assessment. I am developing an algorithm for a game where no
> recognized masters exist. How do you think, guys, if for instance Go (or
> Amazons) provided no way to assess an algorithm playing with professional
> gamers (or other programs), how would you assets its quality?****
>
> My second question: I have not yet learned Go in and out, however in my
> opinion, any search of a next step should identify a number of options with
> similar or even the same assessment. How do you resolve this issue?****
>
>
> Best regards,
> Oleg Barmin.****
>
> _______________________________________________
> Computer-go mailing list
> Computer-go at dvandva.org<https://e.mail.ru/sentmsg?mailto=mailto%3aComputer%2dgo@dvandva.org>
> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>
>
>
>
> Best regards,
> Oleg Barmin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20130612/e1bb8b77/attachment.html>


More information about the Computer-go mailing list