[computer-go] cgos

[Łukasz Lew]

On 9/18/06, Don Dailey <drd at mit.edu> wrote:
> You bring up a good point - I would like to get the opinion of the whole
> group and have a question at the bottom of this message.
>
>
> On Sun, 2006-09-17 at 22:26 +0200, Łukasz Lew wrote:
> > There is a rating drift on CGOS. but its strength and direction is
> > dependant on current configuration of players on server.
>
> I doubt there is a serious rating drift problem on CGOS.   At any given
> moment of course a player can be significantly over or under rated due
> to natural statistical fluctuations (a "string of bad or good luck) etc.
>
> The program "ControlBoy" is identical to "AnchorMan."   AnchorMan of
> course has a fixed rating so I keep an eye on ControlBoy as a sort of
> "control" to see if the rating pool is inflating or deflating.   I don't
> see any statistical evidence that this is happening.

I see a strong evidence.
Recently a lot of new very strong programs appeared and they are
sucking points out of rest of the pool.
I was observing few ZG1's rating, ControlBoy and some other bots. They
*all* are dropping down very consequently.

I guess that this is because a bot with a proper ranking of let's Say
2000, will get only 1900 with 100 games played. Later he will continue
to climb, but sucking points out of the rest.


>
> Of course the rating pool is almost like a living thing,  it's possible
> that a given player is particularly strong against certain types of
> players and can move them around a bit.   But there is no general
> solution that I am aware of.    It shouldn't surprise you if a players
> rating varies by 50 points in either direction from some center value.
>

But a correlated drift of more than 50 points is a very strong evidence that
something is happening.

>
> > Maybe games against new players shouldn't give points to opponents?
>
> CGOS is designed so that new players have close to zero impact on their
> opponents.  From the point of view of the more experienced player, it's
> almost (but not quite) as if the game never happened.   The amount of
> impact a game has on your rating is a function of BOTH players
> experience.  If you have not played many games your rating is impacted a
> lot by each game played but your opponents rating is not.

Maybe a player becomes "experienced" to soon?

Ideally I would like to have a Bayesian rating (KGS like) parallel to ELO.
It would give a *very* nice comparison of those systems.
(one property of Bayesian system is that when A won with B, and B ranking is
increasing, then A ranking is increasing as well)

>
> > There is some problem with pairing algorithm.
> > for instance games MoGoDuoV1 vs Valkyria_UCT3 are very rare.
>
> There is a problem with any pairing algorithm.   There are several
> conflicting goals with any pairing algorithm.   What you would like to
> have is this:
>
>   1.  a good variety of opponents.
>
>   2.  preference to playing opponents of similar strength.
>
>   3.  try to keep each player busy playing games.
>
>
> With very little thought, you can see that each of these goals conflict
> with the other two.
>
> In my opinion, item 3 is the worst offender.  Some of the players on
> CGOS play instantly, others take 10 minutes.  So a game can last from
> just a few seconds up to 20 minutes.
>
> When a player is finished playing a game,  it's preferably to not make
> him wait too long for the next game.     But usually, the most available
> opponents are the ones that play fast.   Therefore the strongest
> (slowest playing) players on CGOS will tend to not see each other much.
> That is the problem you are seeing.
>
> I have considered the possibility of discarding goal 3.   In this model,
> CGOS would start a new round of games at the moment the previous round
> was finished and I could meet goals 1 and 2 MUCH BETTER.
>
> This would make CGOS more friendly to the stronger program which tend to
> take the full amount of time - but the simple programs that play quickly
> would spend most of their life on CGOS waiting to play a game (playing
> about 1 game every 20 minutes like everyone else would be.)
>
> I don't mind catering more to the slow programs which generally tend to
> be the stronger programs - maybe I take this opportunity to see what
> everyone on this list thinks since I am working on a new server
> anyway.
>
> So the question is, should I go for discreet rounds of play which will
> greatly ease the ability to pair players of similar strength and still
> provide variety, but will decrease  the number of games your program
> will play, especially if it plays fast?  The decrease in number of games
> won't impact the slow programs very much, but will impact programs that
> play fast - but will enable "fairer" scheduling by criteria 1 and 2.
>
> Without goal 3, it might even be the case that the slow program play
> more often, since they will essentially be guaranteed a game very 20
> minutes and never have to wait for an opponent to become available.  As
> it is now, a player can get scheduled to play against a player who is
> still playing a game and you have to wait.
>
> What says you?

I propose a following pairing algorithm.
For each pair of players, server track a number of games played.
Then in each moment we have three parameters for each pair.
- g = numbers of games played
- d = rating difference
- t = expected waiting time (when player waiting for other)

for each pair available on server we calculate formula f(g,d,t)
we make a pairing for a pair that f is highest - they are waiting for each other
And repeat the process. (In next iteration waiting time of those two
already paired will be increased),


Best,
Łukasz

>
> - Don
>
>
>
>
>
>
> > Best Regards,
> > Łukasz Lew
> > _______________________________________________
> > computer-go mailing list
> > computer-go at computer-go.org
> > http://www.computer-go.org/mailman/listinfo/computer-go/
>
> _______________________________________________
> computer-go mailing list
> computer-go at computer-go.org
> http://www.computer-go.org/mailman/listinfo/computer-go/
>