Written by Guy Haworth and Nelson Hernandez
Reading, UK and Maryland, USA
This is the third in a new series of analytical articles on past TCEC events. The main text can be read below on this webpage, and at the bottom you will find a link to the full layouted article in pdf format, including the important tables, graphs and images.
TCEC is very grateful to the authors for their kind permission to publish these substantial and scholarly analyses of its events!
Introduction
The knockout format of TCEC Cup 1 (Haworth and Hernandez, 2019a/b) was well received by its audience and was adopted as a regular interlude between the TCEC Seasons’ Division P and Superfinal. TCEC Cup 2 was nested within TCEC14 (Haworth and Hernandez, 2019c/d) and began on January 17th 2019 with 32 chess engines and only a few minor changes from the inaugural Cup event. The ‘standard pairing’ was again used, with seed s playing seed 26-r-s+1 in round r if the wins all go to the higher seed. Thus, seed s1 plays s32, s16, …, s2 if all survive long enough. STOCKFISH was top seed as the TCEC Cup holder but would have been anyway because of its TCEC14 placing as used for the other engines. It is worth noting that the TCEC14 discounting of HANNIBAL’s games because of ‘technical breaks’ affected others’ seeding. PEDONE was s23 rather than s21: ARASAN and VAJOLET stepped up one, given the residual points and tiebreaks. Fig. 1 depicts the logos of the engines in seed order. Basic engine details have been published elsewhere (CPW, 2019; Haworth and Hernandez, 2019d) but some fourteen engines as indicated in Fig. 2 were upgraded for the Cup, again a testimony
to the energy and enthusiasm of their authors.
The format was of 8-game matches at the Rapid tempo of 30’+5″/m rather than the ‘+10″/m’ of TCEC Cup 1. This time, all eight games were played whatever the running score. Openings were repeated with colours reversed after every odd-numbered game. The first few ply in all games were randomly selected from two sets of openings created by the second author here, their relative frequency reflecting that seen in human play: over 200 four-ply openings constituted the repertoire for the opening round of 32 and over 300 twelve-ply openings served thereafter. Tiebreaks were resolved, this time at end of round, by further pairs of games with openings after game 16 from 232 TCEC Superfinal 9-13 options. There was no Armageddon backstop even though the longest TCEC Cup 1 match went to 20 games. Adjudications were as for TCEC14.
As in TCEC Cup 1, interest focused on actual performance ‘%P’ compared with expected performance ‘E%P’ implied by TCEC ELO difference ‘ELO Δ’. The accuracy of the TCEC ELOs, the upgrades to nearly half the field and the character of the random openings would be the main influences.
Round 1
As expected, STOCKFISH opened its campaign with an 8-0 salvo. ‘LC0’ LEELA CHESS ZERO also achieved this feat but was notably more cautious in securing its wins, preferring the gentle ascent of sunlit uplands to a knife-edge scramble up a slippery mountain ridge with its greater risks. A newly refreshed ETHEREAL also scored the whitewash that three higher seeds did not. KOMODO, HOUDINI (twice), FIRE, CHIRON, LASER, FRITZ, GINKGO and remarkably, middling seeds XIPHOS and BOOOT qualified without losing a game. JONNY staged the one ‘come from behind’ victory at the last gasp, winning games seven and eight against ARASAN. The only upset of the seedings came in the third match, new-version ROFCHADE scoring 4½ rather than an expected 1½ and furthermore, ensuring that TCEC14 Division 3 would be represented in the second round.
Surprisingly, there were no ‘Act II’ tie-breaks to play out, perhaps because of the halved 5 move-increment. However, Round 2 was delayed by the setup and replay of the HOUDINI match, this time featuring the submitted WASP 3.54 which had, in error, not made it to the board in the normal course of events. Both matches are included in Table 1. The openings used were different but the end result was the same and even more in HOUDINI’s favour.
Round 2
Eleven of the sixteen victorious engines were sporting new versions for this event so TCEC’s ELO ratings were going to get a more severe examination. Only FIZBO was missing from the fancied half of the line-up. At this point, the second author here drew on his bank of 12-ply rather than 4-ply openings, arguably to the disadvantage of LC0 according to its proponents.
STOCKFISH opened the round as befits the Cup holder – 7-1, the ELO prediction. Game five ended in repetition after only ten calculated moves when the engines bit on a Pringle, a saddle-shape in their joint evaluation surface. GINKGO scored two firsts, one bad one good: the first ‘disconnect’ of the event and the first round two win by the underdog, albeit after the match was decided. The team of unfancied seeds did better than par but still, they all lost: there were no ‘come back’ wins or tiebreaks.
The quarter-finals, semi-finals, small final and final
The locker room for the quarter-finals therefore had the look of a Division P reunion, the top eight seeds perhaps thinking of negotiating their own TV contract. Who knows where AI may lead. STOCKFISH came through easily enough and again without loss but just short of ‘ELO expectations’. HOUDINI and FIRE also came to their match without a loss: something had to give. In their last 22 TCEC games, HOUDINI was +2=19-1 but FIRE came in a new version. The result was the first 4-4 tiebreak, indeed a deadlock of 8 draws – an echo of Carlsen-Caruana, 2018. The tiebreak was played after the other two matches, with the engines unfortunately not able to consider their positions meanwhile.
KOMODO–ETHEREAL was on next. The drawfest continued with the exception of the fourth game in which KOMODO emerged three pawns to the good in the ending. The TCEC adjudication pre-empted an interesting demonstration of technique here. ‘LC0’ LEELA began confidently enough with a win as Black against ANDSCACS: it is as well that engines do not get discouraged. Two more draws followed: it is tough at the top, defences are strong and, though the odd result may go one way or the other, there is little to choose between these engines. ANDSCACS however was notably devoid of the EGT support that LC0 enjoyed. With the advantage of two connected passed pawns and a neat R-v-BN demonstration, LC0 won the ending of game 4: match-score 3-1. LC0 continued undefeated and even won the last game to return arguably the most impressive performance of the quarter-final. The HOUDINI-FIRE tiebreak finished the round with, surprisingly, a 0-1 win for HOUDINI after just two more games. In this way with some suspense, the top four seeds went forward to the semi-finals.
The semi-finals opened with a remarkable nineteen draws: eight by STOCKFISH–HOUDINI and three more after the eight by LC0–KOMODO. Has this ever happened before in computer chess? The deadlock was broken when HOUDINI took control of the open d-file on move 58, eventually lined up its Alekhine’s gun (Chessgames, 2019a/b) on move 80, cramped up Black’s position, created a passed-pawn with a neat tactic, and pushed that pawn to the seventh rank on move 100. STOCKFISH may have seen the advantage before HOUDINI but even so, it was too late to counter. With STOCKFISH a minor piece down, the ‘TCEC win’ adjudication followed quickly. Maybe not the biggest-shock result in TCEC history but certainly the biggest shock-result, especially given that HOUDINI had not been updated for some eighteen months. HOUDINI can beat STOCKFISH in a short match even if this is odds-against. ‘Kingscrusher’ (2019a) covered this game well on the day.
And so to the tiebreaker face-off between KOMODO and LEELA CHESS ZERO, the latter not an engine to rush its pleasures or its recognition of the TCEC deci-pawn ‘draw-zone’. This would be a tight match, KOMODO having beaten LC0 +1=5-0 in their recent head-to-head but LC0 was in a new version here, improving on even the LC0 that earned a place in the TCEC14 Superfinal. Sad that psychology does not play a part in computer chess. Eight draws took us into the second phase of the tiebreak, featuring Jeroen Noomen’s TCEC 9-13 Superfinal Opening book. This, by design, includes more volatile opening positions than other TCEC opening books, thus promoting a higher proportion of decisive games. Even so, LC0 surprisingly did the double immediately to qualify for the final. TCEC at this point introduced a bronze medal ‘small final’ between STOCKFISH and KOMODO which STOCKFISH duly won +2=5-1 sustaining only its second loss of the tournament. That game was the second half of a 1-0/1-0 pair so perhaps the opening handed too big a carrot to White. Apart from that game, KOMODO did not seem to build advantage.
The contest opened with seven draws and we pass quickly over to the game in which LC0 as White scored the decisive win. The evaluation, depth and EGT-support curves of Fig. 2 tell a tale. LC0 dramatically revised its expectations on finding 27. h5, Figs. 2 and 3a. Its situational awareness was clearly superior: it was significantly ahead of HOUDINI which only grew alarmed after 29. f4! and 32. … Nbd7, Fig. 3b: it was even ahead of the mega-threaded and 7-man-EGT-armed STOCKFISH, usually the leading prophet of doom. There is hope for humans yet: the audience had been saying for some time that the optics were at least dramatic – White with two bishops eyeing the black king’s h8-corner, relatively developed pieces and more freedom. LC0 was notably less dependent on depth of search as
the game went on and consulted the EGTs far less than HOUDINI did. After 35. Nxe5, Fig. 3c, LC0’s confidence increased again and after 39. … Rb8 as in Fig. 3d, remarkably the best option, HOUDINI was the exchange down and clearly in trouble at ‘-3’. With 54. Rb3 as in Fig. 3e, LC0 gave up a rook for a knight and converted pawn but by this time both engines knew the game was over. LEELA superfan ‘Kingscrusher’ (2019b) covers this game in detail, indicating some even less attractive byways for Black.
So, this special moment took on extra significance. An engine based on an artificial neural network and ‘MCTS’ Monte-Carlo Tree Searching had taken top step on a TCEC podium. Congratulations must go to Gian-Carlo Pascutto, Gary Linscott, Alexander Lyashuk (Chessdom, 2019) and the choir of angels ‘in the cloud’ who contributed to its self-play training.
A summary
Clearly, the Shannon (1950) mould, now over sixty years old, has been badly cracked if not completely broken. How this has been done is only semi-clear and even then, only to a few. Many will want to do their homework on the new paradigm, the ‘DCNN’ deep convolutional neural network architecture as exemplified by ALPHAZERO and LEELA CHESS ZERO. The ‘zero’ indicates that neither has been trained on human games but there is human input in the adoption of the DCNN approach and, of course, in the definition of the game domain and of what constitutes learning. We hope that further reading of the available sources (Sadler and Regan, 2019; Silver et al, 2017/18) will demystify the magic of these new machines.
Advised by Sadler (2018), what can we say about LEELA’s style of play? Compared to what we have seen before, it seems to be more strategic and less tactical, more interested in space and movement than in material, more interested in the prophylactic reduction of its opponent’s options. Questions remain about LEELA’s tactical ability where one would expect minimax to be more effective, where a narrow ‘study like’ path has to be found despite the presence of serious dangers.
MCTS has changed the game in several domains as evidenced by past articles in this journal and results at past ICGA Computer Olympiads. Was TD-GAMMON (Tesauro, 1995) the first demonstration of the new way? The combination of two concepts, MCTS and DCNN, has clearly been key: the future may combine further sets of ideas, e.g., MCTS and minimax-search. When should resources be shifted from one towards the other? We can expect to see new types of computer-chess engine in the future.
On a broader canvas, it appears that once again computer chess has been the Drosophila melanogaster fruit fly vehicle of choice for artificial intelligence research, showing the way forward for new kinds of analysis and insight. Improved early-detection in the medical field, driven by better computer vision, is a laudable priority with potential impact rivalling Hopkins’ invention of the fibrescope and his contribution to medical optics (Hopkins and Kapany, 1954/5; McCombie and Smith, 1998).
Wrapping up this report, we give the usual generic statistics on the defined openings and game-lengths, plus a selection of interesting endgames that arose, see Tables 6-8. Fuller data and the somewhat annotated pgn files of TCEC Cup 2 are available (Haworth and Hernandez, 2019c). Congratulations to the winning and unbeaten LEELA CHESS ZERO team (Linscott, 2018), also focusing on their TCEC14 Superfinal against STOCKFISH (Haworth and Hernandez, 2019d). Special thanks should also go to all the semi-finalists who produced such close matches: there is little between them. Once again, let us point out that this event would not be the same without the administrators’ efforts, the participation of the entire field of thirty-two engines and the audience in the ‘online lounge’ who contribute an enjoyable mix of questions and information, leavened by interspersed jokes and off-topic comments.
References
- Chessdom (2019). https://tinyurl.com/icgaj048. Interview with Alexander Lyashuk, core member of the LEELA CHESS ZERO team.
- Chessgames (2019a). https://tinyurl.com/icgaj041. The ‘Alekhine’s Gun’ game against Nimzovitsch.
- Chessgames (2019b). https://tinyurl.com/icgaj052. Lagno–Wenjun, WWCC 2018, Alekhine’s Gun.
- CPW (2019) https://tinyurl.com/icga046. Biographies of chess engines, authors and developers.
- Haworth, G. McC. and Hernandez, N. (2019a). http://centaur.reading.ac.uk/80284/. TCEC Cup 1. This note plus annotated statistics and pgn files. Submitted to the ICGA Journal.
- Haworth, G. McC. and Hernandez, N. (2019b). http://centaur.reading.ac.uk/78820/. TCEC13: the 13th Top Chess Engine Championship. Submitted to the ICGA Journal.
- Haworth, G. McC. and Hernandez, N. (2019c). http://centaur.reading.ac.uk/81390/. TCEC Cup 2. This note plus annotated statistics and pgn files. Submitted to the ICGA Journal.
- Haworth, G. McC. and Hernandez, N. (2019d). http://centaur.reading.ac.uk/82052/. TCEC14: the 14th Top Chess Engine Championship. To be submitted to the ICGA Journal.
- Hopkins, H. H. and Kapany, N. S. (1954). A Flexible Fibrescope, using Static Scanning. Nature 173, 39-41.
- Hopkins, H. H. and Kapany, N. S. (1955). Transparent Fibres for the Transmission of Optical Images. Optica Acta, 1(4), 164-170.
- ‘Kingscrusher’ (2019a). http://tinyurl.com/icgaj039. HOUDINI-STOCKFISH, semi-final game 12.
- ‘Kingscrusher’ (2019b). https://tinyurl.com/icgaj040. LC0-HOUDINI, final game 5.
- Konoval, Y. (2019). Private communication of DTC(onversion) depths for some positions.
- Linscott, G. (2018). https://github.com/LeelaChessZero/lc0/wiki LC0 on Github.
- McCombie, C. W. and Smith, J. C. (1998). Harold Horace Hopkins. 6 December 1918 – 22 October 1994. Biographical Memoirs of Fellows of the Royal Society, 44, 239–252.
- Sadler, M. (2018). https://tinyurl.com/icgaj045. Sadler on ALPHAZERO’s play.
- Sadler, M. and Regan, N. (2019). Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. New in Chess. ISBN-13: 978-9056918187.
- Shannon, C. E. (1950). Programming a Computer for Playing Chess. The London, Edinburgh and Dublin Philosophical Magazine, 41(314), 256-275. doi: 10.1080/14786445008521796.
- Silver, D. et al (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learn-ing Algorithm. arXiv: 1712.01815.
- Silver, D. et al (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140-1144. doi: 10.1126/science.aar6404.
- Tesauro (1995). Temporal Difference
Full article
To read the full article in pdf, click HERE
