[computer-go] Congratulations to LeelaBot2 and to CzechBot
Nick Wedd
nick at maproom.co.uk
Thu May 8 15:08:42 PDT 2008
In message <6054b7b40805081347h8f1e739o1dd9ce237f628b51 at mail.gmail.com>,
Jason House <jason.james.house at gmail.com> writes
>You have a typo "deasd" should be "dead".
Thank you, fixed.
>My personal preferences would be to see the final section change its
>title from "Probation" to "Losses in Cleanup" or some other title
>addressing undesirable issue uncovered in this past tournament.
I have renamed this section to "Losses after Game Stop". Is this
reasonable?
>I'd also appreciate it if the old versions of the report can be removed
>completely. The last thing I want is for a potential employer to
>google me and find one of those pages.
Ok, understood, I have removed them. I too am glad to see them removed,
as they reflect badly on me.
Nick
> They're very easy to
>misinterpret, especially given that my uniform-playout MC bot's
>performance was on par for 19x19 with relatively short time limits.
>Once upon a time, people thought MC bots would not scale to 19x19.
>Local biases in playouts, move ordering, and possibly progressive
>widening have changed all that. I hope to do that one day, but I don't
>yet.
>
>PS: You'll be happy to hear that I've been discussing alternate
>resignation strategies with people. The lack of resignation in this
>past tournament occurred because time it was a large board with short
>time limits (and my bot is slow). Everyone I've talked to seems to
>think the threshold I have for resigning are reasonable. What I have
>right now considers a single move in isolation, but it should be
>possible to consider a sequence of moves together. Essentially, it'll
>increase the available data to use in the resignation decision and
>avoid incorrect resignations at idiotic points.
>
>
>On Thu, May 8, 2008 at 4:21 AM, Nick Wedd <nick at maproom.co.uk> wrote:
> In message <482246BC.4080208 at gmail.com>, Michael Williams <
> michaelwilliams75 at gmail.com> writes
>
>
>> I agree with Evan 100%.
>
>> I probably would have gone berserk if I were Jason. Instead he
>> handled it with relative grace, considering what was in that
>> report.
>
>
> Ok. I am persuaded that I have acted wrongly here. I withdraw the
> probation on Jason's bot, and offer my apologies to him. I shall
> rewrite my report (and archive the old one, for the historic
> record).
>
> Nick
>
>
>
>> Evan Daniel wrote:
>
>>> On Wed, May 7, 2008 at 11:45 AM, Nick Wedd <nick at maproom.co.uk
>>> > wrote:
>
>>>> In message <
>>>> 6054b7b40805070659u543a69afvf8b9dffbe75a743e at mail.gmail.com
>>>> >,
>>>> Jason House <jason.james.house at gmail.com> writes
>
>
>
>>>>> Correction: HBotSVN was not reconfigured for speed in
>>>>> round 3. It was
>>>>> set to use two search threads in round 4, and was
>>>>> compiled in debug
>>>>> mode for the whole tournament. I apologize for the
>>>>> confusing PM's
>>>>> during the tournament about this.
>
>
>>>> Thank you for explaining this, I have changed the report
>>>> accordingly.
>
>
>
>
>>>>> What is "HBotSVN's technique"?
>
>
>>>> Its technique is to refuse to admit that its dead groups
>>>> are dead, and then
>>>> to waste time in the resolution phase playing meaningless
>>>> stones. This
>>>> sometimes gives it a win on time, and is the only way that
>>>> it wins games.
>>>> This is annoying for the other competitors. I know it is
>>>> not your intention
>>>> that it behaves like this, but it is in your power to
>>>> prevent it. It is not
>>>> in my power to do anything about it, except by reassigning
>>>> the results of
>>>> games which it wins like this: this is the purpose of the
>>>> probation. It is
>>>> in Bill Shubert's power to change the way the server works
>>>> so that if only
>>>> one player sends a final_status_list, it will accept what
>>>> that player says.
>>>> I shall suggest it to him.
>
>
>
>
>>>>> The game end protocol says "To play
>>>>> in a tournament, programs must either implement both
>>>>> "kgs-genmove_cleanup" and "final_status_list dead", or
>>>>> they must play
>>>>> until all of their opponent's dead stones are removed
>>>>> from the board.
>>>>> It's OK if "play until dead stones removed" is an
>>>>> option, but they have
>>>>> to make sure that this option is turned on whenever they
>>>>> are going to
>>>>> be in a tournament, or they will do poorly in the
>>>>> tournament!".
>>>>> HouseBot (HBotSVN) handles this by playing until all of
>>>>> its opponent's
>>>>> dead stones are removed.
>>>>> "final_status_list dead" is not supported. It's kgsGtp
>>>>> (not
>>>>> HouseBot!) that insisists that all stones are alive. It
>>>>> annoys me
>>>>> every time I see the description that it's the bot
>>>>> that's behaving
>>>>> badly when it's really a problem with how the
>>>>> combination of kgsGtp and
>>>>> the KGS server represent this stuff.
>
>
>>>> I have changed the wording of my report from "claimed they
>>>> were alive" to
>>>> "failed to admit that they were dead". I have done so
>>>> because you have
>>>> persuaded me that it is correct and what I said before was
>>>> wrong. I do not
>>>> expect you to be appeased by this.
>
>
>
>
>>>>> I consider it a bug in kgs that
>>>>> this perpetually gets misinterpreted by spectators.
>>>>> Please stop saying that my bot insists all of its
>>>>> stones are alive.
>>>>> This could be simplified by either fixing the game end
>>>>> protocol rules,
>>>>> or getting kgs fixed (kgsGtp and/or the server).
>
>>>>> In the round two game, it was HBotSVN that had 3
>>>>> seconds left on the
>>>>> clock. Its opponent, MonteGNU, had almost a minute left
>>>>> (51 seconds).
>
>
>>>> Thank you for pointing this out. I have corrected my
>>>> mistake.
>
>
>
>
>>>>> The only games where HBotSVN's opponent got down to very
>>>>> little time
>>>>> left was the game against Leela.
>>>>> The whole probation thing has really pissed me off.
>>>>> Maybe one
>>>>> component of that is first finding out about it by
>>>>> reading it in the
>>>>> report. I have not been implementing "difficult things"
>>>>> for quite a
>>>>> while. Because stuff wasn't working, I suspended all
>>>>> forward progress
>>>>> on my bot two months ago. Since then, I've been
>>>>> building test
>>>>> harnesses, writing unit tests, and eliminating bugs.
>>>>> Did you know that weakbot50k and idiotbot don't
>>>>> actually handle the
>>>>> game end at all? Once both players pass, they switch to
>>>>> using gnu go.
>
>
>>>> I didn't know that, but it seems a sensible, robust,
>>>> solution.
>
>>>> Nick
>
>
>
>
>
>>>>> I will no longer participate in these tournaments for
>>>>> the foreseeable
>>>>> future.
>
>>> I fail to see the problem with HBotSVN's behavior. It is
>>> playing
>>> according to the protocol as specified. Humans judging intent
>>> and
>>> reasonableness belong in human tournaments, and possibly
>>> human-computer tournaments, but most emphatically not in
>>> computer-computer tournaments. What would you do if HBotSVN
>>> implemented final_status_list dead and always returned the
>>> empty set?
>>> What if it only returned stones that were unsalvageable even
>>> in the
>>> face of opponent passes? HBotSVN seems to be requiring its
>>> opponents
>>> to demonstrate that they are actually capable of killing the
>>> groups
>>> they claim are dead. Given the skill level of some programs,
>>> and that
>>> programs are not offendable, this behavior seems at worst
>>> mildly rude,
>>> possibly deserving of derision and disrespect, and completely
>>> undeserving of any sort of sanction.
>>> It is entirely within the power of the other bots to not lose
>>> on time.
>>> If they cannot manage their time, that should be viewed as a
>>> defect
>>> no different from not being able to manage sente. If you
>>> dislike the
>>> effect this has on time management, add a single 1-3s byo yomi
>>> period
>>> (or some other time system; I dislike Japanese byo yomi but it
>>> works
>>> well enough here and is traditional). Any program that cannot
>>> play
>>> out the dispute resolution accurately in 1s byo-yomi had no
>>> business
>>> being so confident in its assessment that it should run out
>>> all its
>>> time before passing.
>>> Attitudes like this make me think that these games are to be
>>> taken
>>> solely as exhibition games, and not as actual tournaments. I
>>> find
>>> this disappointing, as I think KGS could be a good complement
>>> to CGOS
>>> as a forum for computer Go.
>>> Someday I hope to return to writing a computer Go program.
>>> However, I
>>> can't see myself entering it in any tournament that does not
>>> precisely
>>> and accurately specify the required behavior.
>>> Evan Daniel
>>> _______________________________________________
>>> computer-go mailing list
>>> computer-go at computer-go.org
>>> http://www.computer-go.org/mailman/listinfo/computer-go/
>
>
>
>> _______________________________________________
>> computer-go mailing list
>> computer-go at computer-go.org
>> http://www.computer-go.org/mailman/listinfo/computer-go/
>
>
> --
> Nick Wedd nick at maproom.co.uk
> _______________________________________________
> computer-go mailing list
> computer-go at computer-go.org
> http://www.computer-go.org/mailman/listinfo/computer-go/
>_______________________________________________
>computer-go mailing list
>computer-go at computer-go.org
>http://www.computer-go.org/mailman/listinfo/computer-go/
--
Nick Wedd nick at maproom.co.uk
More information about the computer-go
mailing list