[Computer-go] monte carlo weakness

Don Dailey dailey.don at gmail.com
Tue Sep 7 14:18:42 PDT 2010


I think the test can be constructed to favor either side and I don't think
it will be possible to be completely fair.

The new modern program are clearly more scalable than the old ones,  so I
think if you play really fast the difference will be less, just as you say.


But your idea is not fair, by a long shot.   One of the things that makes
modern chess programs so smart is that they are highly selectivity - they
play the game a lot more like humans play the game in this respect.

So clearly, if you strip the modern programs of this advantage,  of course
you will see a much smaller difference.     This is like getting in a fight
and asking one of the fighters to tie both hands behind his back and if he
still wins by kicking his opponent,  demanding this his legs also be bound!


There is a an issue here though that you have identified.   No matter which
way we go, it's not quite fair.    On modern hardware the new programs have
the advantage of better compilation and the older programs were not designed
to run on newer hardware - so they are somewhat crippled by this factor.   I
think that is probably worth a factor of something like 1.5 or so but I'm
just guessing.

But regardless of the exact ratio,  it's clear that software advancement is
enormous, and it's quite non-trivial.

Don



On Tue, Sep 7, 2010 at 4:52 PM, Dave Dyer <ddyer at real-me.net> wrote:

>
> It's interesting to compare new and old programs, but I think it would
> be more fair to handicap in the opposite direction.    The moore yardstick
> suggests that the same program evaluate 4000 times as many positions
> as on 16 year old hardware.
>
> Whatever the multiplier, restrict the "old" program to time that allows
> that number of nodes, and restrict the modern program by the same time.  (I
> realize this isn't really possible or fair, as there are many other factors
> affecting performance) but to press on...
>
> You'll end up comparing the "old" program at its old skill level,  with a
> new
> program running in some amazingly short amount of time, like 0.1 seconds
> per move.
>
> I bet that at that level, the skill differential is much less significant.
>
> --
>
> So, getting back to my original point, for chess "more horsepower" resulted
> in much better performance, even without changes in algorithms.  It looked
> at least plausible that MC techniques had pushed Go into a zone where that
> was also true, but if human players can easily adapt to defeat strong MC
> programs, perhaps not.
>
> _______________________________________________
> Computer-go mailing list
> Computer-go at dvandva.org
> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20100907/376e4e9a/attachment.html>


More information about the Computer-go mailing list