Artificial intelligence is the field where computing power gets machines to do things that lie in the domain of thinking. Simple applications are where machines process data to make evaluations or predictions. And then, computers operate machinery, or play simple games.

It becomes more complex when computers develop strategy or modify their way of working, on the bases of how good their predictions or the results of their decisions have been. Card games like bridge or poker, where opponents use deception, introduces another level of complexity.

Advertisement

But the pinnacle was programming a computer to match grand masters in playing chess. When this bastion fell, the challenge shifted to the game of Go — an ancient Chinese board game that has even greater complexity. This too, was mastered in 2016, when the British Artificial Intelligence company, DeepMind Technologies, created the programme, AlphaGo, which defeated Lee Sedol, the 18- times world champion.

David Silver, Julian Schrittwieser, Karen Simonyan, Loannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel and Demis Hassabis, all from DeepMind, now report in the journal Nature, an improvement, named AlphaGo Zero, which achieves “a long-standing goal of AI — an algorithm that self-trains to superhuman proficiency without expensive human-input”.

This ability, of the algorithm to train itself, is of great importance, as machine learning is now used in a variety of commercial, industrial and scientific fields, all of which face high costs in training the AI machine.

Applications include analyses of customer trends, improving supply of products and services, traffic control, health administration, automated diagnosis and drug discovery, voice and image recognition, running driverless motor vehicles in a busy street.

The basic method used is to fit complex but known data to a mathematical formula. The formula is then tested on other sample data and if it qualifies, it could predict unknown data too. While predictions were improved with massive data sets, it was soon realised that the animal brain does even better through a different angle of attack.

In playing a game of chess, for instance, a computer could work out all possible moves and counter-moves, by both players, from a given board position, and then choose the move to make. Human players lack such ability but they still defeat even powerful computers. It was realised that the human brain does not follow the brute force method of the computer, but takes in some features of the chess board position, which may seem to be unrelated, and use these, and experience, to play in a more effective way. The brain charts the different responses of brain cells to the features of the chess board, and the outcomes.

Given a set of responses, the responses are strengthened or weakened, according to the outcome. Over a series of actual instances, the brain adapts to making more effective responses and continues to learn with experience. This mechanism leads not only to good chess playing but is also the way a child internalises the nuances of a language faster than years of study by scholars.

Computers were now programmed to simulate this architecture by creating virtual neurons, or software that behaved like brain cells. In a simple instance of recognising a just one feature, the feature could be presented to a single virtual neuron. The neuron responds at random, from a set of choices. If the answer is correct, there is feedback that adds to the probability of that response, and if the answer is wrong, the feedback lowers the probability. We can see that this device would soon learn, through a random process, to consistently make the correct response. A brace of artificial neurons that send responses to another set of neurons, and so on, could deal with several inputs with greater complexity.

A network like this could learn to identify an image as being that of a car or pedestrian, for instance, and if a pedestrian, whether a man or woman. In learning to play games, AI systems are designed to take in different features like immediate threats, potential threats, a metric to evaluate positional advantage, devices to evaluate patterns in opponents’ moves, relative values of costs to be paid for gains and so on. The system then plans a series of responses to possible future moves of the opponent and could evaluate the options against a data base of past games.

Alternately, the system could shadow real games of expert players and evaluate proposed moves by comparison with moves selected by the experts. One strategy to filter moves is to evaluate a move till it is found to be inferior to a previously evaluated one. The evaluation itself may be by traversing the tree that is formed when each move can have different responses, each of which can have different responses, and so on. Or there could be ways of assigning approximate values to moves. A combination of both is the Monte Carlo Tree Search, which employs statistical methods.

AlphaGo, which beat Lee Sedol in four out of five games, employed a pair of neural networks — one to predict the probability of opponent moves and another to evaluate the potential of a board position. The first network was trained with the help of human experts and the second by simulating the game after the given position. These two networks gave rise to a branching inverted tree of moves and countermoves, which helped select the way forward.

AlphaGo Zero, the current version uses a single neural network and a simpler tree search to evaluate moves and positions. The greatest improvement, however, is that the algorithm needs no human input for training. The game, which it plays with itself, starts with simple, untrained, random moves and the positions created are “evaluated”. Evaluation results in reinforcing some choices in the random selection and the game are played again. “In each iteration, the performance of the system improves by a small amount, and the quality of the self-play games increases, leading to more and more accurate neural networks and ever stronger versions of AlphaGo Zero,” two of the authors say in a press release. The algorithm hence starts to form “tabula rasa” or a ‘blank slate”, as philosopher John Locke described the mind of an infant, and learns from itself to rapidly become the strongest Go player (and hence teacher) in the world. The method “is no longer constrained by the limits of human,” the press release says.

AlphaGo Zero was hence able to beat AlphaGo 100-0 in a trial, with just three days of training on a single machine with four specialised neural network chips against multiple machines and 48 specialised chips.

But the real gain is not a better Go player, it is an artificial intelligence strategy that would help machines handle complexities of an interconnected world of rising mechanisation and under pressure to optimise energy and resources.

(The writer can be contacted at response@simplescience.in)