[computer-go] cgos

[Łukasz Lew]

On 9/18/06, Don Dailey <drd at mit.edu> wrote:
> My Anchor player will prevent the rating from inflating or deflating too
> much.   It's possible that the anchor is too small - like a big ship
> with too small an anchor.
>
> But if the rating pool becomes too deflated, the Anchor will fight back
> even harder.  Also, AnchorMan plays more games than any other player on
> CGOS.   So if the pool is seriously inflated, everyone that beats
> AnchorMan will effectively be getting bonus points.

But, You know. If MogoDuo has 42/42 wins and most of strong programs
have more than 80% then from this distance Anchor works weakly and
slowly.

The pool of players is changing so quickly that it would be very good
to have a second anchor around 1800.


Another solution is to implement TrueSkill rating system.
The main difference is that it has two numbers per player
- it's strength and uncertainty about it.
This way MoGo, Valkyria_UCT3/4 etc would have still large uncertainty
what would solve both problems: would grow faster to the "destination
rating" and would not drain points from their opponents so badly until
they would get to the destination.
increase their rating faster, and not


>

>
> - Don
>
>
>
> On Mon, 2006-09-18 at 06:24 +0200, Łukasz Lew wrote:
> > On 9/18/06, Don Dailey <drd at mit.edu> wrote:
> > > You bring up a good point - I would like to get the opinion of the whole
> > > group and have a question at the bottom of this message.
> > >
> > >
> > > On Sun, 2006-09-17 at 22:26 +0200, Łukasz Lew wrote:
> > > > There is a rating drift on CGOS. but its strength and direction is
> > > > dependant on current configuration of players on server.
> > >
> > > I doubt there is a serious rating drift problem on CGOS.   At any given
> > > moment of course a player can be significantly over or under rated due
> > > to natural statistical fluctuations (a "string of bad or good luck) etc.
> > >
> > > The program "ControlBoy" is identical to "AnchorMan."   AnchorMan of
> > > course has a fixed rating so I keep an eye on ControlBoy as a sort of
> > > "control" to see if the rating pool is inflating or deflating.   I don't
> > > see any statistical evidence that this is happening.
> >
> > I see a strong evidence.
> > Recently a lot of new very strong programs appeared and they are
> > sucking points out of rest of the pool.
> > I was observing few ZG1's rating, ControlBoy and some other bots. They
> > *all* are dropping down very consequently.
> >
> > I guess that this is because a bot with a proper ranking of let's Say
> > 2000, will get only 1900 with 100 games played. Later he will continue
> > to climb, but sucking points out of the rest.
> >
> >
> > >
> > > Of course the rating pool is almost like a living thing,  it's possible
> > > that a given player is particularly strong against certain types of
> > > players and can move them around a bit.   But there is no general
> > > solution that I am aware of.    It shouldn't surprise you if a players
> > > rating varies by 50 points in either direction from some center value.
> > >
> >
> > But a correlated drift of more than 50 points is a very strong evidence that
> > something is happening.
> >
> > >
> > > > Maybe games against new players shouldn't give points to opponents?
> > >
> > > CGOS is designed so that new players have close to zero impact on their
> > > opponents.  From the point of view of the more experienced player, it's
> > > almost (but not quite) as if the game never happened.   The amount of
> > > impact a game has on your rating is a function of BOTH players
> > > experience.  If you have not played many games your rating is impacted a
> > > lot by each game played but your opponents rating is not.
> >
> > Maybe a player becomes "experienced" to soon?
> >
> > Ideally I would like to have a Bayesian rating (KGS like) parallel to ELO.
> > It would give a *very* nice comparison of those systems.
> > (one property of Bayesian system is that when A won with B, and B ranking is
> > increasing, then A ranking is increasing as well)
> >
> > >
> > > > There is some problem with pairing algorithm.
> > > > for instance games MoGoDuoV1 vs Valkyria_UCT3 are very rare.
> > >
> > > There is a problem with any pairing algorithm.   There are several
> > > conflicting goals with any pairing algorithm.   What you would like to
> > > have is this:
> > >
> > >   1.  a good variety of opponents.
> > >
> > >   2.  preference to playing opponents of similar strength.
> > >
> > >   3.  try to keep each player busy playing games.
> > >
> > >
> > > With very little thought, you can see that each of these goals conflict
> > > with the other two.
> > >
> > > In my opinion, item 3 is the worst offender.  Some of the players on
> > > CGOS play instantly, others take 10 minutes.  So a game can last from
> > > just a few seconds up to 20 minutes.
> > >
> > > When a player is finished playing a game,  it's preferably to not make
> > > him wait too long for the next game.     But usually, the most available
> > > opponents are the ones that play fast.   Therefore the strongest
> > > (slowest playing) players on CGOS will tend to not see each other much.
> > > That is the problem you are seeing.
> > >
> > > I have considered the possibility of discarding goal 3.   In this model,
> > > CGOS would start a new round of games at the moment the previous round
> > > was finished and I could meet goals 1 and 2 MUCH BETTER.
> > >
> > > This would make CGOS more friendly to the stronger program which tend to
> > > take the full amount of time - but the simple programs that play quickly
> > > would spend most of their life on CGOS waiting to play a game (playing
> > > about 1 game every 20 minutes like everyone else would be.)
> > >
> > > I don't mind catering more to the slow programs which generally tend to
> > > be the stronger programs - maybe I take this opportunity to see what
> > > everyone on this list thinks since I am working on a new server
> > > anyway.
> > >
> > > So the question is, should I go for discreet rounds of play which will
> > > greatly ease the ability to pair players of similar strength and still
> > > provide variety, but will decrease  the number of games your program
> > > will play, especially if it plays fast?  The decrease in number of games
> > > won't impact the slow programs very much, but will impact programs that
> > > play fast - but will enable "fairer" scheduling by criteria 1 and 2.
> > >
> > > Without goal 3, it might even be the case that the slow program play
> > > more often, since they will essentially be guaranteed a game very 20
> > > minutes and never have to wait for an opponent to become available.  As
> > > it is now, a player can get scheduled to play against a player who is
> > > still playing a game and you have to wait.
> > >
> > > What says you?
> >
> > I propose a following pairing algorithm.
> > For each pair of players, server track a number of games played.
> > Then in each moment we have three parameters for each pair.
> > - g = numbers of games played
> > - d = rating difference
> > - t = expected waiting time (when player waiting for other)
> >
> > for each pair available on server we calculate formula f(g,d,t)
> > we make a pairing for a pair that f is highest - they are waiting for each other
> > And repeat the process. (In next iteration waiting time of those two
> > already paired will be increased),
> >
> >
> > Best,
> > Łukasz
> >
> > >
> > > - Don
> > >
> > >
> > >
> > >
> > >
> > >
> > > > Best Regards,
> > > > Łukasz Lew
> > > > _______________________________________________
> > > > computer-go mailing list
> > > > computer-go at computer-go.org
> > > > http://www.computer-go.org/mailman/listinfo/computer-go/
> > >
> > > _______________________________________________
> > > computer-go mailing list
> > > computer-go at computer-go.org
> > > http://www.computer-go.org/mailman/listinfo/computer-go/
> > >
>
>