[computer-go] Is skill transitive? No.

[Nick Apperson]

I completely agree with everything you have said.  I would also just add
that one thing humans learn to do, is to stear the game into zones where
they play better.  When facing a computer opponent, this generally means
stearing the game into positions where "judgement" is a much larger factor.
Computers generally don't understand their own weaknesses and strengths so
they are particularly prone to this type of assault.  It is also a similar
strategy that a strong player will use when they play double approach moves
in a handicap game.  For me, I haven't been playing go for as long as many
other people, but I have a very good intuition for the game.  I just lack
the mental stamina to read at "my level" so my friend will exploit this by
fighting in situations where he expects a slightly negative result because
he knows I will have to expend much of my finite mental energy on this which
will cause me to make a blunder later.

- Nick

On 1/29/07, Don Dailey <drd at mit.edu> wrote:
>
> I was looking at many of the posts on the threads about
> how things scale with humans and computers and I'm
> trying to reconcile many of the various opinions and
> intuitions.  I think there were many legitimate points
> brought up that I appeared to be brushing off.
>
> In computations done by computer, there can usually be a
> trade-off between time and memory.  In the discussions,
> we rarely talked about memory and how it figures in to
> the picture.  A lot was said about just knowing
> something (where a strong player looks at a position and
> instantly knows a weaker player made a mistake for
> instance) and the feeling expresses by many was that
> this was a barrier that could not be penetrated by
> thinking about the position no matter how much time
> was allowed.
>
> Although I consider the evidence pretty strong for
> rating curve in both humans and computer, the model of a
> fixed strength increase per doubling is actually a
> simplification - in the real world it is more
> complicated than that.
>
> It's important to realize that the ELO formula is based
> on assumptions about human playing strength that are
> only approximations.  One of those assumptions is that
> playing strength is transitive and can be expressed as a
> single value - a number that we call a persons "rating."
>
> Nevertheless, intransitivity is a real thing.  The way
> we sometimes erroneously think about GO is that you have
> some fixed strength expressed as a kyu or dan "number"
> and that every move is a reflection of this level of
> play.
>
> A better model, which is still a simplification, is that
> a move is either right or wrong and the stronger you
> are, the more likely you will choose the better move.
> Some move are easy to find and the weaker players find
> them, but on average you are faced with moves of every
> level of difficulty and the difference between stronger
> and weaker players is how many of these positions they
> solve - kind of like a big test with a mixture of easy
> and hard problems and the one that gets the most answers
> right wins!
>
> >From a purely theoretical point of view, a move really
> is either best or not-best but as humans we judge moves
> on a sliding scale of "goodness" and refer to some moves
> as being horrible and others as being brilliant, good,
> second best, etc.  On this group we recently discussed
> how to define error vs blunder and so on.
>
> The intuition behind judging moves like this is that
> indeed, some moves give you better practical chances in
> the real world.   So if you are slightly losing, a move
> my be referred to as "a good try" because it complicates
> things, or at least requires the opponent to find a
> refutation that in human terms is difficult to find.
>
> Sometimes a good player, or even a computer can
> instantly find the right move where a weaker player has
> no clue and is not likely to discover the correct
> principle even given several hours of meditation.  This
> has been mentioned a number of times recently.  This an
> example of a chunk of knowledge having a profound effect
> on the quality of a single move.  Even with computers it
> is possible that a good life and death routine can
> discover things (more or less) instantly that might take
> a very long time to find with a global brute force
> search.
>
> Because knowledge can be imperfectly and unevenly
> applied, one player might play some types of positions
> much better than others.  So even among players of
> roughly equal abilities, one player may see at a glance
> what another player would have a very difficult time
> discerning.
>
> What this causes in my opinion is instransitivity.  It
> doesn't cause a player to stop improving substantially
> with time as many experiments have proved.  But it's a
> known phenomenon that because of intransitivity and
> these knowledge gaps, you might improve much more
> against a particular opponent (opponents just like
> yourself for instance) and much less against other kinds
> of opponents.
>
> But this is also about memory scalability.  Better
> players have more knowledge about the game.  It's very
> difficult to measure knowledge quantitatively in humans.
> How do you have twice as much knowledge in Go?  How do
> you test this?  But it's clear that stronger players
> have much more knowledge, probably much of it in the
> form of trained intuition about go positions in the form
> of pattern recognition.  Some knowledge is expressed as
> cute little proverbs of wisdom such as "the opponent's vital
> point is my vital point" among others.
>
> Because no two players play alike, and especially computers
> and humans, bits of knowledge and processing power have
> different scaling characteristics.  Even a particular piece
> of knowledge could help you more against one opponent that
> another.
>
> So let me restate my feelings based on the above
> considerations:
>
> 1. Game playing skill is a function of time.
>
> 2. Memory (or knowledge) can proxy for time - saving
>     enormous amounts of time in many cases.
>
> 3. "Technique" is a function of knowledge and how it's
>     organized - which translates to a big time savings
>     indirectly.  This is really the ability to apply knowledge.
>
> 4. Because these various aspects of game playing ability
>     can be mixed and matched,  you are sure to get very
>     interesting intransitives.
>
>
> So although I believe in good scalability characteristics
> with time and skill, the improvement may not ramp up as
> quickly against certain kinds of opponents.  You may need more
> "doublings" to make the same improvement against a
> particular opponent than you would against another.
>
> A probabilistic way to look at this is on a move by move
> basis.  Some of you noted that some moves are "beyond them"
> such that a player 2 stones stronger sees it at a glance and
> you have no chance of seeing it.  And yet you still have
> about 7% chance of beating a player 3 stones higher.  I
> suggest that in these types of positions only, you are many
> stones weaker, but in other types of position you might
> actually be equal or even stronger.  You cannot look at one
> position in isolation and draw conclusions that you could
> never beat such a player.
>
> When I played tournament chess I was exposed to players much
> weaker and much stronger than myself.  I discovered that I
> was better in some positions that even players significantly
> stronger.  And I was aware of much weaker players that I had
> to steer away from certain kinds of positions.  Stronger
> players often recovered when I outplayed them - but that
> doesn't change the fact that it's possible to out-play them
> in positions you understand better.  If they are not too
> strong you will even win once in a while due to the fact
> that you did indeed outplay them.
>
> In computer go, the differences between humans and computers
> is enormous - and I don't mean just in strength, but in
> style.
>
> When I was developing Botnoid, I even saw this in
> computer/comptuer.  At one point in 9x9 Botnoid development,
> Botnoid could score about even with Gnugo 3.6 at the levels
> I tested with.  This was quite amazing when you consider how
> good gnugo was tactically compared with Botnoid.  Botnoid
> had a bad habit of playing into self-atari - a problem I
> patched up a bit but never solved completely.  And yet it
> could win about half the games.  You could look at
> individual moves of Botnoid and conclude that it was not in
> the same league as gnugo and should never win, but you would
> be wrong.  That's why I'm not impressed too much with
> arguments about isolated positions and how superior 3 ranks
> can be in selected positions because it's all about playing
> the WHOLE game.  Botnoid excelled in some areas that gnugo
> didn't and there were games where it seemed like everything
> was over for gnugo before it even had a chance to get into
> the game.
>
> Anyway, I think this note reflects a more balanced viewpoint
> of how things really work.  To a certain extent, I think you
> could say that in practical terms it's hard to overcome a
> serious rank difference - as you would not only have to
> overcome the geometric time explosion which by itself is a
> practical barrier for anything more than 4 or 5 ranks but
> you might also have to overcome the "intransitivity barrier"
> where you don't get the same effective strength increase
> against a particular opponent.  I still believe that many of
> us on this group underestimate our abilities at longer time
> controls but it's not productive to debate this here.
>
> - Don
>
>
> _______________________________________________
> computer-go mailing list
> computer-go at computer-go.org
> http://www.computer-go.org/mailman/listinfo/computer-go/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://computer-go.org/pipermail/computer-go/attachments/20070129/da0c1e6c/attachment.htm