[computer-go] Congratulations to LeelaBot2 and to CzechBot

[Nick Wedd]

In message <482246BC.4080208 at gmail.com>, Michael Williams 
<michaelwilliams75 at gmail.com> writes
>I agree with Evan 100%.
>
>I probably would have gone berserk if I were Jason.  Instead he handled 
>it with relative grace, considering what was in that report.

Ok.  I am persuaded that I have acted wrongly here.  I withdraw the 
probation on Jason's bot, and offer my apologies to him.  I shall 
rewrite my report (and archive the old one, for the historic record).

Nick

>Evan Daniel wrote:
>> On Wed, May 7, 2008 at 11:45 AM, Nick Wedd <nick at maproom.co.uk> wrote:
>>> In message <6054b7b40805070659u543a69afvf8b9dffbe75a743e at mail.gmail.com>,
>>> Jason House <jason.james.house at gmail.com> writes
>>>
>>>
>>>> Correction: HBotSVN was not reconfigured for speed in round 3.  It was
>>>> set to use two search threads in round 4, and was compiled in debug
>>>> mode for the whole tournament.  I apologize for the confusing PM's
>>>> during the tournament about this.
>>>>
>>>  Thank you for explaining this, I have changed the report accordingly.
>>>
>>>
>>>
>>>>   What is "HBotSVN's technique"?
>>>>
>>>  Its technique is to refuse to admit that its dead groups are dead, and then
>>> to waste time in the resolution phase playing meaningless stones. This
>>> sometimes gives it a win on time, and is the only way that it wins games.
>>> This is annoying for the other competitors.  I know it is not your intention
>>> that it behaves like this, but it is in your power to prevent it.  It is not
>>> in my power to do anything about it, except by reassigning the results of
>>> games which it wins like this:  this is the purpose of the probation.  It is
>>> in Bill Shubert's power to change the way the server works so that if only
>>> one player sends a final_status_list, it will accept what that player says.
>>> I shall suggest it to him.
>>>
>>>
>>>
>>>> The game end protocol says "To play
>>>> in a tournament, programs must either implement both
>>>> "kgs-genmove_cleanup" and "final_status_list dead", or they must play
>>>> until all of their opponent's dead stones are removed from the board.
>>>> It's OK if "play until dead stones removed" is an option, but they have
>>>> to make sure that this option is turned on whenever they are going to
>>>> be in a tournament, or they will do poorly in the tournament!".
>>>> HouseBot (HBotSVN) handles this by playing until all of its opponent's
>>>> dead stones are removed.
>>>>   "final_status_list dead" is not supported.  It's kgsGtp (not
>>>> HouseBot!) that insisists that all stones are alive.  It annoys me
>>>> every time I see the description that it's the bot that's behaving
>>>> badly when it's really a problem with how the combination of kgsGtp and
>>>> the KGS server represent this stuff.
>>>>
>>>  I have changed the wording of my report from "claimed they were alive" to
>>> "failed to admit that they were dead".  I have done so because you have
>>> persuaded me that it is correct and what I said before was wrong. I do not
>>> expect you to be appeased by this.
>>>
>>>
>>>
>>>> I consider it a bug in kgs that
>>>> this perpetually gets misinterpreted by spectators.
>>>>   Please stop saying that my bot insists all of its stones are alive.
>>>> This could be simplified by either fixing the game end protocol rules,
>>>> or getting kgs fixed (kgsGtp and/or the server).
>>>>
>>>>   In the round two game, it was HBotSVN that had 3 seconds left on the
>>>> clock.  Its opponent, MonteGNU, had almost a minute left (51 seconds).
>>>>
>>>  Thank you for pointing this out.  I have corrected my mistake.
>>>
>>>
>>>
>>>> The only games where HBotSVN's opponent got down to very little time
>>>> left was the game against Leela.
>>>>   The whole probation thing has really pissed me off.  Maybe one
>>>> component of that is first finding out about it by reading it in the
>>>> report.  I have not been implementing "difficult things" for quite a
>>>> while.  Because stuff wasn't working, I suspended all forward progress
>>>> on my bot two months ago.  Since then, I've been building test
>>>> harnesses, writing unit tests, and eliminating bugs.
>>>>   Did you know that weakbot50k and idiotbot don't actually handle the
>>>> game end at all?  Once both players pass, they switch to using gnu go.
>>>>
>>>  I didn't know that, but it seems a sensible, robust, solution.
>>>
>>>  Nick
>>>
>>>
>>>
>>>
>>>>   I will no longer participate in these tournaments for the foreseeable
>>>> future.
>>  I fail to see the problem with HBotSVN's behavior.  It is playing
>> according to the protocol as specified.  Humans judging intent and
>> reasonableness belong in human tournaments, and possibly
>> human-computer tournaments, but most emphatically not in
>> computer-computer tournaments.  What would you do if HBotSVN
>> implemented final_status_list dead and always returned the empty set?
>> What if it only returned stones that were unsalvageable even in the
>> face of opponent passes?  HBotSVN seems to be requiring its opponents
>> to demonstrate that they are actually capable of killing the groups
>> they claim are dead.  Given the skill level of some programs, and that
>> programs are not offendable, this behavior seems at worst mildly rude,
>> possibly deserving of derision and disrespect, and completely
>> undeserving of any sort of sanction.
>>  It is entirely within the power of the other bots to not lose on 
>>time.
>>  If they cannot manage their time, that should be viewed as a defect
>> no different from not being able to manage sente.  If you dislike the
>> effect this has on time management, add a single 1-3s byo yomi period
>> (or some other time system; I dislike Japanese byo yomi but it works
>> well enough here and is traditional).  Any program that cannot play
>> out the dispute resolution accurately in 1s byo-yomi had no business
>> being so confident in its assessment that it should run out all its
>> time before passing.
>>  Attitudes like this make me think that these games are to be taken
>> solely as exhibition games, and not as actual tournaments.  I find
>> this disappointing, as I think KGS could be a good complement to CGOS
>> as a forum for computer Go.
>>  Someday I hope to return to writing a computer Go program.  However, 
>>I
>> can't see myself entering it in any tournament that does not precisely
>> and accurately specify the required behavior.
>>  Evan Daniel
>> _______________________________________________
>> computer-go mailing list
>> computer-go at computer-go.org
>> http://www.computer-go.org/mailman/listinfo/computer-go/
>>
>
>_______________________________________________
>computer-go mailing list
>computer-go at computer-go.org
>http://www.computer-go.org/mailman/listinfo/computer-go/

-- 
Nick Wedd    nick at maproom.co.uk