Go Commentary: DeepMind AlphaGo vs Fan Hui – Game 5

This is a commentary of the final game of the five game match between Fan Hui 2p and Google DeepMind’s AlphaGo.

AlphaGo-logoNews of AlphaGo’s 5–0 victory over Fan shocked the Go community and made mainstream news headlines around the world.

The game was played on October 9, 2015, but the startling performance of AlphaGo wasn’t revealed until a paper detailing the feat was published in the science journal Nature, on January 27, 2016.

This means we have only recently heard about the games and had a chance to analyze them.

Commented game record

Fan Hui vs DeepMind AlphaGo

 

Download SGF File (Go Game Record)

 

Why Go?

Go has long been a significant challenge to artificial intelligence (AI) researchers, because the large number of possible Go games make it infeasible for computers to perform well using brute force alone.

This has meant that the best human players have until now remained out of reach of the best computer players, despite decades of research into AI and advances in computing power.

Fan Hui

Fan Hui playing against AlphaGo in London.

Fan Hui playing against AlphaGo in London.

Fan Hui is a professional with the Chinese Go Association and has been living in France, where he has taught and promoted Go since the early 2000s. He was born in 1981 and became a pro in 1996.

Google DeepMind contacted Fan to arrange the match and he played against AlphaGo in London, under the supervision of Toby Manning from the British Go Association.

Ten games were played in total; five official games and five unofficial games. Fan chose a time limit of 1 hour main time and 3 x 30 seconds byo-yomi each for the official games. He won two unofficial games against AlphaGo (30 seconds per move), but lost all the official games.

The first game of the match was quite leisurely and territorial. After Fan lost that game by 2.5 points, he thought that perhaps AlphaGo didn’t like to fight, so he played more aggressively in the games that followed. Unfortunately for Fan this game plan didn’t pay off.

DeepMind AlphaGo

AlphaGo is a Go AI developed by DeepMind — a British AI research company which was acquired by Google in 2014. They are undertaking a self-described “Apollo Program for AI,” in a project involving more than 100 scientists.

Neural Networks

DeepMind-AlphaGoThe backbone AlphaGo’s strength lies in its successful application of neural networks to Go.

In this context a ‘neural network’ is a technology for processing information and forming connections in a way that is modeled on the neural connections in the human brain.

The goal of this technology is to enable computers to learn in a way that is more general and human like.

DeepMind aims to develop a general learning algorithm which can be applied to many problems instead of a pre-programmed AI which is only capable of doing one thing (e.g. playing Go or chess).

The chess computer Deep Blue, which defeated chess grandmaster Garry Kasparov in 1997, is an example of the latter (pre-programmed) AI.

It appears that AlphaGo, being a stepping stone along this path, is currently a little bit of a hybrid of the two approaches. The more general purpose neural network has been ‘trained’ by giving it access to a huge number of Go games between skilled humans. The ‘knowledge’ it has acquired throughout this process has been reinforced by allowing it to play an enormous number of games against itself and evaluate them using some serious hardware.

Monte Carlo

However, its strength is further boosted by the use of Monte Carlo Tree Search (MCTS) — a technology which has already been applied to Go for about a decade and has led to computer Go programs making great strides against amateur level players.

MCTS applies a statistical approach to finding good moves. It is a search algorithm where the computer simulates many possible games and, after seeing the result of each random game, aggregates all the results to calculate a probability of success for a selection of next moves. If this sounds counter-intuitive, that’s because it is!

MCTS does not require a great deal of domain specific knowledge (knowledge of Go provided by a human creator) to perform well, but a programmer still has to configure and tune this approach for the game in question. One of the problems AI researchers have faced with Go is that it’s difficult to evaluate whether a position is good or bad.

For example, you can’t assign scores to pieces like you can with chess, because the pieces all look the same. MCTS has, until now, evaluated positions by simulating all the way to the end of the game, counting the score, and then aggregating the results of many simulations.

Putting it all together

AlphaGo changes the way MCTS is applied by using a neural network to evaluate whether a position is good or bad. DeepMind has actually trained two neural networks as part of AlphaGo. The first, called the ‘policy network’, chooses promising looking moves for deeper analysis — similar to what humans do when they rely on instinct.

The second, called the ‘value network’, specializes in positional judgment. The value network allows AlphaGo to evaluate a position without playing each simulation all the way to the end of the game. This makes MCTS work more efficiently than it did in the previous generation of Go AIs.

Further reading

The above is a relatively basic explanation of how AlphaGo works and may contain errors (though they will be happily corrected). For more detailed information about computer Go, please see:

Follow the match between Lee Sedol and AlphaGo

Having defeated Fan Hui, AlphaGo has challenged Lee Sedol 9p to a match in March 2016.

Match details and frequent updates will be posted on the DeepMind AlphaGo vs Lee Sedol page.

If you would like to follow the match, you can click here to subscribe to our newsletter and receive free, weekly updates.

 

Related Articles

About Younggil An

Younggil is an 8 dan professional Go player with the Korean Baduk Association. He qualified as a professional in 1997 and won an award for winning 18 consecutive professional matches the following year. After completing compulsory military service, Younggil left Korea in 2008, to teach and promote the game Go overseas. Younggil now lives in Sydney, Australia, and is one of the founders of Go Game Guru. On Friday evenings, Younggil is usually at the Sydney Go Club, where he gives weekly lessons and plays simultaneous games.

You can follow Go Game Guru on Facebook, Twitter, Google+ and Youtube.

Comments

  1. Xavier Combelle says:

    Fantastic comment! looks like Lee Sedol will have hard time

    • Younggil An says:

      Thanks Xavier.

      Yes, Lee will have hard time, but I believe he will find a weakness of AlphaGo.

  2. Thank you for the commentary!
    Always a pleasure to read.

    I have studied the game with the great help of you analysis.
    My first comment on the game in the previous topic said if
    white wants to play 56, it is better to omit the 54-55
    exchange. You mention that black should have connected at
    p10 intead of 55. And that white could consider p10 at move 58.

    White could also have played 54 at p10 himself.

    So if white plays 56 instead of 54, black could play p10-white 54,
    then answer at m2.

    In the game, I think black could have gone ahead and played
    57 straight at 73. If white plays, 74, black wins the ko.
    White will probably still take the four stones in the bottom right
    corner as an exchange. But his time there would be no damage
    to black’s shape in the lower left.

    Black 65 seems to have been a really serious mistake.
    Black should play g5 as you mention. White j4-black k5.
    Then I think white should cut at h5 to avoid a black move
    at the same place. Black h4 white j5 would follow. J3 and
    k6 would be miai. If black k6, then after white j3 it is better
    not to have the white f4-black g3 exchange. If black j3, white k6,
    black l5, white l6 would be hard to take.

    You show 74 as t10 to play a ko, which is a great move.
    It seems it’s also 65’s fault that it would be so effective.

    I have made an attempt in some other comments (also
    in the previous topic) to identify losing moves and /or
    major contributing factors to Fan Hui’s defeats (82 in
    game 2, 93 in game 3, 75 and 93 in game 5).

    At the same time trying to point out the computer AI
    seems to have been prone to make seemingly forcing
    moves that turned out to be damaging exchanges such
    as 31-32 in game 2 (given the continuation of 35) and
    54 here in game 5 (given the continuation of 56,58,70).

    • Younggil An says:

      Thanks ParuBaru for your detailed analysis based on deep researching of the games.

      Yes, 54 can be omitted and your idea about Black 65 makes sense too.

      As you said, the AI doesn’t yet seem to be perfect, but it will be interesting to see how much he will be improved by the time of the match against Lee Sedol.

      • Kaan Malçok says:

        Exactly what i think An! When people makes comments about the game, they always think AlphaGo would only be as strong as it was in the game against Fan Hui. The AI can improve dramatically in a short time after breaking through a certain threshold, especially when its developed by a team of 100 genius scientists. I am pretty the sure the AlphaGo that is going to play against Lee Sedol will be really different from what people expect in terms of strength.

  3. I really enjoyed your comments, thank you! Twice you wrote something like “this is what White wants”, which is funny when White is a computer…

    Of the eleven games of Fan in go4go he only won two, not quite an impressive record. Comments by some high level professionals like Ke Jie that I read on Facebook suggest the strength of Alfago is that of a starting young professional. If this is still true in March, Lee should win easily, but Alfago is a self learning program, it will be interesting to see what happened in the five months in between. Because of the money prize of $ 1 Million I hope Lee wins. After that, I hope the computer shows a steep increase in strength, it becoming 20p, so that it can show us secrets of go still unheard of for us, human beings.

    Kind regards,
    Paul

    • Younggil An says:

      Thanks Paul for your opinion.

      Yes, I also read what Ke jie said about the games of Alphago, and he’s also quite curious how strong it is and wants to play against the machine.

      As Ke said, most of pros would be interested in Alphago, and we will soon see the real strength of the mysterious AI.

  4. great review! thank you! 🙂

  5. Josh Hoak says:

    Great commentary! I wonder if Fan would have played differently if he thought he was playing against a Pro rather than an AI.

    • Younggil An says:

      Hi Josh, long time no see.

      Fan Hui must have played differently if he thought he was playing against a pro. I assume that he was shocked when he lost his first game, and he was under big pressure as well. He wouldn’t be expected that the computer can be that strong and smart like a strong human player.

      I don’t think he had enough time to recover from the psychological damage from defeat, and he couldn’t keep a balance himself in the rest of his games.

  6. Jorik Mandemaker says:

    Thanks for the commentary.

    When I first saw this game I was really impressed by the white moves from 56 onwards. They just felt so creative. And it’s wonderful that I can now say that about moves made by an AI.

    • Younggil An says:

      Yes, right. White’s technique at the bottom from 56 was impressive, and they were quite tricky as well.

      You can feel it’s creative, but compared to other strong programs such as Zen or Crazy Stone, Alphago’s style of play is rather conservative, but much more accurate and subtle.

  7. Thanks for the commentary. It’s good to see that we can identify mistakes from both sides 🙂 I can’t help wondering how much AlphaGo was “intrinsically” stronger than Fan Hui, and how much was due to consistency (the computer never gets tired or careless) and the challenge for Fan of playing in a very foreign environment.

    A couple of decades ago, people predicted that computers would mean the end of competitive chess. The opposite has been true, and it has been exciting to learn from the computers. Also it took about a decade to get from “computers are strong enough to challenge average professionals” to “computers are clearly stronger than the world champion”. It’ll be interesting to see how things develop in the go world.

    One thing I’m a little disappointed about is that AlphaGo has been trained so strongly to make human-like moves. I remember when Zen19 reached amateur dan strength ( https://gogameguru.com/man-machine-showdown-board-game-go ): it was figuring out its own moves to a much larger extent, and came up with some very creative ideas in the opening!

    • Younggil An says:

      Yes, I felt so too. AlphaGo plays like a pro, but not like Takemiya Masaki 9p. 🙂

      However, AlphaGo seems to be much stronger than other competitive programs partly because it doesn’t play creative moves. Probably a half of creative moves in the opening can be nonsense, so I don’t think it’s disappointing to see that AlphaGo doesn’t play creative moves.

  8. Thanks form the comments. Can B129 play on O15? Thanks

    • Younggil An says:

      Yes, that’s also possible, but Black 129 was right.

      If Black played at O15 instead of 129, White can try to cut Black off with White L15, Black L16 and White K16 later.

  9. Thank you for the commentary, Younggil. I hope you will be doing commentary for the Lee Sedol match as well. Poor Lee Sedol…always has a target on his back.

    • Younggil An says:

      Hi Wasuji, thanks for your comment.

      Yes, we’re going to have live commentaries for the Lee Sedol match. I think Lee would be excited to play the historic match. 🙂

  10. Do you know if Lee Sedol will be able to study more of the games of AlphaGo than those played against Fan Hui? AlphaGo has access to all of Lee Sedol’s games. My understanding is that AlphaGo is playing hundreds of thousands of games against itself to improve, but they are not public.

    • Younggil An says:

      I don’t know if Lee Sedol will be able to study more of Alphago.

      In my opinion, even if Alphago can study hundreds of thousands of games by itself and play perfectly, Lee will be able to make the computer’s perfect moves become bad by subtle probes and trades.

      I’m wondering how well Alphago can deal with the ko fights, and that would be another important issue for AI to overtake human I think.

  11. Younggil one important thing is if Alphago has the same habit like standard Monte Carlo programs making moves that guarantees Victory (even with 0.5) points rather playing the strongest move in the board.
    This would help to understand whether some mistakes he made were actually mistakes.

    • Younggil An says:

      Thanks Billy for your explanation about the system of AlphaGo, and that’s very helpful for me to understand more of it.

      Yes, then the match in March will be more interesting and exciting to watch.

  12. Anonymous says:

    I have a question about AlphaGo’s move 64. To my Laymans Eyes, this seemd the most stylish/creative move made by Alphago, and indeed following it B 65 was probably a mistake (as in your analysis). However in your analysis If B 65 is at G5, it seems that B has the upper hand, as both your variations for W reply there At F4 or J4 seem to end badly for W. So does B 65 at G5 makes W64 at G4 a lucky mistake, or is there a reasonable continuation for W also after B move at G5 not shown in your analysis ?

    • Younggil An says:

      White 64 was a sharp probe, and even if Black played at G5, White might choose the second variation to reduce Black’s influence.

      After Black 5 in second variation from Black G5, White will continue with K6, Black L5, White L6 to reduce Black’s bottom, and it will still be playable for both.

      Therefore, White 64 wasn’t a mistake, but a tricky tesuji, but Black 65 made the move bright.

  13. It’s not a big deal for AI to beat human in Go game. Actually it is simple game, only one kind of pieces for each side and it stands still there after putting on the board.

  14. Hi! I’m especially intrigued in your comment on move 80. Is it possible that what would ‘usually’, i.e. from the human perspective, be regarded as OVERPLAY, it might have actually be the best move at that moment? My reasoning is that white didn’t have any problems to settle its center group afterwards.
    And while you suggested in the variant that the move on L16 would limit black’s influence in the center, on the other hand with the move 80 and the follow-up, white practically ERASED any prospective black’s territory in the center, not only reduced it.
    I’m aware of your later explanation that black’s move 93 should have been at S16, gaining lots of points. By this, you explain the previous overplay (move 80). However! White wouldn’t have had to play at K10 in the follow-up move 94 as you suggested in the variant [AlphaGo did not have this strengthening move in the actual game either; in fact white ‘dared’ to play on S16, so black got to play TWO attacking moves in the center! And still, white got away with that pretty easily], so instead it could have played a big move on the left. TWO big moves on the left as a matter of fact, if black had decided to attack white similarly as in the game, which would more than make up for the loss in the upper right group – R14 group, if black chose to cut them off as suggested in the variant.

    • Younggil An says:

      Your idea is reasonable.

      Yes, I agree that White 80 can be still playable. In the actual game, Black 93 was too early, and he lost nearly one stone as White played at 94. White 94 wasn’t only a big endgame move, but it helps its center group indirectly.

      Because of that, the game was practically finished after Black started to attack, but if White didn’t cut at 80, the game would have been clearer and simpler for White.

      White’s lost from cutting at 80 would be around 30 points if Black played at S16 for 93, but erasing Black’s center wouldn’t be that valuable because Black will still harass White’s center group later.

  15. I have another question:
    How many stones would you say AlphaGo is stronger than Fan Hui? It seems to me that AlphaGo dominated all the games, not giving him much chances (it really led on Fan Hui, the difference in strength was pretty ‘tangible’). In my eyes, it would mean something like two or three stones difference.
    I know that some strong professionals opined that AlphaGo’s strength might be somewhere around a beginning pro. What is your opinion on that?

    I’m just an amateur player, but seeing AlphaGo playing very solid moves throughout the whole games, having a good understanding of joseki, being flawless in reading and perfect in the end game, I would actually put my money on AlphaGo in the match with Lee Sedol 😉
    Although I wish Lee Sedol could still win the match, I think we all might be really surprised!
    We all still remember how Kasparov was flabbergasted after losing to Deep Blue, only shaking his head after not being able to match its skills and its calculation.
    I WOULDN’T be surprised if we saw something similar after the AlphaGo – Lee Sedol match!

    • Younggil An says:

      Thanks Ortenix for your question and opinion.

      On those games, Alphago seems to be much stronger than Fan, but it’s still hard to tell how much.

      It made some mistakes on joseki, but Fan didn’t take the advantage because didn’t punish the mistake properly. He must have been under big pressure, and he didn’t show his strength in the match.

      Its play is very solid as you said, but there were some mistakes in reading. I still think Lee will win the match in March, but if AlphaGo can win a game, it will be really surprising.

  16. Roland CA says:

    I think Google set the date in March, knowing that they will win at least 1 game.
    Whether it takes 3 months more to win vs Lee, or 6 months to win vs Ke, it’s going to happen.
    The Ai can learn just like human, but much faster.

    It can also master voice/face recognition, or pattern recognition in general.
    It can also learn physics, art and psychology.

    The newer version of iPhone Siri will not only answer questions, but also chat with us like an old friend 🙂

  17. I wonder whether AlphaGo grasped the intricacies of the ko fight. But if it knows the basics, it should be much better at it than human beings: one should continue the ko fight when the sum of the values of the current ko threat and the next ko threat is bigger than the sum of the values of the ko and the ensuing sente. For human beings: don’t end the ko fight too early, it may cost you quite some points, never mind who wins the ko fight.

    Kind regards,
    Paul

  18. Mate Matolcsi says:

    I would like to see a detailed commentary of game 1. Apparently, that was the only game where Fan Hui was ahead for a while. At least, this is what Myungwan Kim said. It would be interesting to see whether you think that Alphago could be beaten in a quiet game without fighting. Thanks!

    • Younggil An says:

      Thanks Mate for your kind suggestion.

      When I had a look at the game, I thought White had a nice opening and kept on leading the game, and Alphago played safely in the endgame.

      Fan hui might have thought so too, and that’s why he changed his style of play from game 2.

  19. thanks for the comments. im waiting for the computer to solve go and open with tengen!