Man vs Machine match: Final results and game commentary

The ‘Man vs Machine’ Go match ended on January 16, with the computer winning 3-1.

Final results

Here’s are the full results with links to the games:

go game 300x300 picture

When will computers become good enough at Go to beat professional players?

It was a very interesting match and we all got to see just how good computers are at Go now.

They may not be good enough to challenge a professional player yet, but I felt that Zen’s performance was quite impressive when you think about where computer Go was at just a few years ago.

Thanks to everyone involved

Thanks John Tromp for putting up a good fight for the humans, Yoji Ojima (who created Zen), Hideki Kato (who provided the hardware that Zen ran on during this match), Darren Cook for organizing this event and making the original Shodan Go Bet with John, and anyone else who was involved behind the scenes.

If you’re interested in learning more about computer Go, subscribe to the computer Go mailing list.

Commentary of game 2

A lot of people asked if the games could be commented, so I’ve chosen one game and reviewed it. I chose to review game 2, because I thought it was the most tactically interesting of the four and it shows some of the strengths and weaknesses of Zen well.

Since I’m expecting some people who are not serious Go players, but are interested in computer Go to look at this, I’ve gone into more detail than I normally would in a review. Please forgive me if you’re a strong Go player and you feel like I’m pointing out the obvious at times :).

Also, if you have any questions about the game, feel free to ask here.

Commented game record

John Tromp vs Zen19

Download SGF File (Go Game Record)

What do you think of Zen

What do you think about Zen? Are you surprised to see computers playing this well? Leave a comment below.

About David Ormerod

David likes teaching, learning, playing and writing about the game Go. He's taught hundreds of people to play Go, including many children at schools in Australia. In 2010 David was the Australian representative at the 31st World Amateur Go Championships. He's a 5 dan amateur Go player and is the editor of Go Game Guru. You can find David on Google+ and follow Go Game Guru on Facebook, Google+ or Twitter.

Comments

  1. Yes, I am surprised. We discussed this event in our club and observed there is a current of romantics, who feel the game has lost a bit of its magic now that computers have become strong so quickly and a current of positivists who think it’s just great.

    Myself I am happy that there is now a player of about 4-5d strength who plays rather different from the usual 4-5d. The similarities are probably confirmed conventional wisdom (I suspect Zen uses almost no libraries of preconditioned wisdom), the differences fall into categories of plain mistakes and surprisingly good moves.

    The surprisingly good moves are those we can learn from, particularly unlearn some deeply grained habits of automatic play. We’re strong in visual pattern recognition and that strength is also our handicap. Zen obviously computes more and therefore seems to have a wider arsenal of conceivable moves.

    Its strength at L&D surprises me less than some of its well timed reduction moves. I think it is great to have a strong program and would love to play it. That was another difference between the romantics and the positivists: the former would miss the feeling of playing a human, the latter saw the benefit of playing “someone” who only plays moves he genuinely thinks are the best.

    Thanks for the coverage.

    • David Ormerod says:

      Interesting Dieter. Now that you mention it, I’ve noticed those two groups too when observing people’s reactions to Zen (though I didn’t clearly label them in my mind).

      Beginners are usually still told that “Go is a game that computers can’t play well”, and it is part of the attraction for a good deal of people. Maybe we’ll have to stop saying that soon though…

      I really wanted John to win the match, partly because – as someone who wants to introduce more people to Go – I thought there’d be more opportunities for getting the story into the mainstream media if he won. ‘Human outsmarts computer’ is just more a story for people with no interest in Go.

      However, like you I still think there are many positives to this. Computers are strong enough now to give me a challenging game on 9×9 (and that’s fun sometimes). Beginners were always told not to practice against the computer because it would teach them bad habits, but I wonder if that’s still true, especially on 9×9?

      For those who have a phone capable of it, having a strong Go player ready and willing to practice with you any time (even if only on 9×9) is very nice for many players. Especially those who have very little time to play normally. It’s like a genie in a bottle. :)

    • I find myself with mixed feelings about all this: I too feel a sense of loss to realize how strong computers are getting. I think it may not be so long before computers can match up with pros. After all, monte carlo is a highly parallelizable type of algorithm: You just add more machines to play more random games and pick the results with the highest score.

      But it is also remarkable and kind of wonderful that such a well known and simple idea has produced such amazing results. I found out about Go originally int the mid 90s and back then it seemed as if either a) We’d need to develop hardware that made a comprehensive minimax search feasable, which seemed unlikely in the foreseeable future or b) someone would have to gain deeper insight into how human beings actually conceptualize Go and implement it as an AI algorithm.

      It seems to me that so far the success of programs like Zen and CrazyStone is somewhere in between: Faster and cheaper hardware makes things possible, but using monte carlo turns out to be a reasonable conceptual framework to get programs into the “pretty darn strong amateur” range.

  2. I’m having difficulty reading commentary. Sometimes the comment is longer than the commentary box allows, but I don’t see how to access the rest of the quotation that is off-screen. Help, please?

    • There should be a slide bar on the right side of the comment box that you can click to go up and down, same as is on the side of your web page.

  3. These are some nice comments. Very few games are commented this thoroughly in a way most people can understand.

    • David Ormerod says:

      Thanks Oh, we’ll think about doing more commentaries like this in the future then. They do take a long time to write though :).

      • I really enjoyed this review. I’m KGS 3dan and still really appreciated the extra detail.

        In general the game reviews are what bring me to this site and they’re great, but there was something that connected with me more seeing an amateur game commented. The situations felt more similar and relevant, while the lower level comments clarified a number of things I’ve wondered about.

        I’d love to see more like this.

        I was wondering: did you run the comments by Mr. Younggil before sending out, or were you able to confidently make the claims you did by yourself as a 5d amateur (especially in the opening)?

        Thanks again, this was really great,
        Niall :-)

        • Younggil An says:

          I’d like to answer to your question Niall.
          I had a look at the commentary before David published, but he did explain most of comments by himself. That’s why it took him a long time to write this commentary. :)

  4. Just wanted to say thanks for the commented game, especially since you mentioned that you were feeling a bit under the weather. I’m actually going through it in parts (Right now I’m at move 110) due to the large amount of variations.

  5. I’m a mid-sdk player and I really enjoyed the “over”-commented game. It was great to see such detailed commentary; even the comments that touched on familiar concepts were helpful in that they showed important principles in context. Thanks a lot!

  6. Chris, thanks for your suggestion. I looked for the slide you mentioned, but there doesn’t appear to be one. :’-(
    Would it matter that I’m using a not-new iPad?
    jean

    • Anonymous says:

      jean, try to get the dolphin browser on your not new ipad, hope this helps.

      As for the commented game I find it great that you went into so much explaination; I am sorry to say I do not understand the game well enough to follow much of Younggil’s commentaries.

    • David Ormerod says:

      Jean, other browsers would display a scroll bar on the right hand side of the comments, but mobile/tablet browsers seem not to, presumably to save screen real estate.

      The best thing to do would be to download the file and view it on a Go iPad app. SmartGo Kifu: http://www.smartgo.com/kifu.htm is very nice, for example.

      If you have a Go app installed you should be able to tap and hold [link] and get an option like ‘Open in SmartGo’. Please let me know if you need anymore help with this.

  7. Adalberto Duarte says:

    It is clear that Zen is a very strong Go playing program under a 24 core computer. I wonder how strong can it be in a normal Core 2 duo or an i5 processor PC. Anyone has any information about this?

  8. Adalberto, I’m being pedantic but it was actually 12-cores (at 4.2Ghz) for this match.
    If you assume half a rank for each doubling of power, a 4-core 2Ghz machine would be 1.5 ranks weaker than the machine used in the match. So, very roughly, 2 dan KGS at slow thinking times, 3 dan KGS at fast games?

    • Annoymouse says:

      Any reason why it’s weaker at slow games? One would expect computer calculation power divided to be human computational power to be monotonically linearly increasing.

      • David Ormerod says:

        I think it’s more a case of humans making more mistakes in fast games, but there could be more to it than that…

  9. Having a strong opponet to play against constantly is a great way to get stronger. Seeing as im weaker than the program…

    It is very annoying to me I can not purchase the software and play against the computer.

    • David Ormerod says:

      Great point ljl. I’m trying to get the publisher of Zen to let us sell it in our shop, but no luck so far.

  10. Loved the commentary !
    I’m really a beginner but the commentary make it look Epic and I kind of understand some moves, it makes me want to learn a lot more !

    I’m subscribing to the news feed right now, and I would really enjoy reading things like this in the future :)

    • David Ormerod says:

      Great :). Go is really fun, even as a beginner. Unlike a lot of games though, it keeps getting more interesting the more you learn. Good luck!

  11. Hi David,

    As a weak player who doesn’t get to play very often (~12-14kyu, but it’s been a while since I’ve been able to rank myself against humans) your commentary was invaluable! I usually find commented games aim for low-dan players and forget those of us who don’t eat life & death problems for lunch. The analysis of the struggle in the bottom-left was particularly eye-opening for me. Thank you!

    Dave.

    • David Ormerod says:

      Hi Dave, thanks for the feedback. Quite a few people have told me that the detailed commentary was helpful, so we’ve decided to comment some more games in this way. It really takes a long time though, so Younggil and I are going work on 5-10 games together and publish it as a go book when we’re finished.

      By the way, you made me laugh with ‘eat life & death problems for lunch’. I often solve problems during my lunch break or on the bus, so I guess you’re talking about people like me :).

  12. Anonymous says:

    Computer may some day beat a human either by brutal force or true understanding of the game. But I feel this particular game is probably at the level of 1k. Hope John is not intensionally losing to Zen to stir up the discussion.

    • David Ormerod says:

      I don’t think he would do that, and John doesn’t claim to be especially strong.

      In fact, when I asked him if he’d had time to play some training games or study Zen’s games before the match he said he’d been too busy at work, unfortunately. So if anything, he was just dealing with the normal human stresses that affect our games.

      Zen is quite strong though. Not long after we published this article, it managed to beat Takemiya Masaki with a four stone handicap.

Speak your mind