In the game won by Lee Sedol, AlphaGo was apparently surprised by a brilliant and unexpected move from Lee Sedol.After analysing the logs, the Deep Mind CEO said that AlphaGo had evaluated a 1/10000 probability for that specific move to be played by Lee Sedol.What I don't understand here is : whatever the probability for a good move to be played, why taking the risk ? Why not instead expecting the opponent to always plays the best moves ?Of course it's always possible that you miss the best move your opponent could play when playing Monte Carlo to evaluate his possibilities, but here it seems that the move was founded. If AlphaGo knew that its strategy could be counter by such a move, why not choosing another strategy, where the worst case scenario would be less worse.
Why doesn't AlphaGo expect its opponent to play the best possible moves?
machine learning
It appears that AlphaGo did not rate the move as a best possible move for Lee Sedol, just as one that was within its search space. To put into context the board is 19x19, so a 1 in 10000 chance of a move is much lower than chance of the square being picked at random. That likely makes the move that it found not worth exploring much deeper.It is important to note too that the probabilities assigned to moves are equivalent to AlphaGo's rating for quality of that move - i.e. AlphaGo predicted that this was a bad choice for its opponent. Another way of saying this, is there is a probability p that this move is the best possible one, and therefore worth investigating further. There is no separate quality rating - AlphaGo does not model opponent's chance of making a move separately from opponent's chance of gaining the highest score from this position if he/she makes that move. There is just one probability covering both those meanings 1As I understand it, AlphaGo rates the probabilities of all possible moves at each game board state that it considers (starting with the current board), and employs the most search effort for deeper searches on the highest rated ones. I don't know the ratios or how many nodes are visited in a typical search, but expect that a 1 in 10000 rating would not have been explored in much detail if at all. It is not surprising to see the probability calculation in the system logs, as the logs likely contain the ratings for all legal next moves, as well as ratings for things that didn't actually happen in the game but AlphaGo considered in its deeper searches.It is also not surprising that AlphaGo failed to rate the move correctly. The neural network is not expected to be a perfect oracle that rates all moves perfectly (if it was, then there would be no need to search). In fact, the opposite could be said to be the case - it is surprising (and of course an amazing feat of engineering) just how good the predictions are, good enough to beat a world-class champion. This is not the same as solving the game though. Go remains unsolved, even if machines can beat humans, there is an unknown amount of additional room for better and better players - and in the immediate future that could be human or machine.There are in fact two networks evaluating two different things - the policy network evaluates potential moves, and the output of that affects the Monte Carlo search. There is also a value network which assesses board states to score the end point of the search. It is the policy network that predicted the low probability of the move, which meant that the search had little or no chance of exploring game states past Lee Sedol's move (if it had, maybe the value network would of detected a poor end result from playing that through). In reinforcement learning, a policy is set of rules, based on known state, that decide between actions that an agent can take.