Chessdom

Subscribe to Chessdom feed
Chess, chess news, live chess games
Updated: 10 hours 29 min ago

TCEC Cup 3 – pairings and information

Sat, 04/27/2019 - 15:39

TCEC Cup is coming, with the top 32 engines in a knockout competition

The TCEC Cup is now an integral part of the seasons of the Top Chess Engine Championship. It brings a special twist to the regular pace of the season, gives a chance for lower rated engines to meet the best, and also provides opportunity for engines with new versions to show their best and climb up the ladder.

The first TCEC Cup was convincingly won by Stockfish. It proved superior in Season 13 and did not give a chance to its opponents at any moment during the cup competition as well. TCEC Cup 2, however, was a different story. Stockfish stumbled at the semi-final against the S10 champion version of Houdini. This pitted Houdini against the bearer of the NN revolution Lc0, and the latter won its first major title in computer chess.

Replay TCEC Cup 2 final

TCEC Cup 3 starts this weekend. The cup holder Lc0 is going to defend its title in a race with the top 32 engines of the ongoing season. Its first match will be against a newcomer to the TCEC race – Marvin by Martin Danielsson (see Marvin’s rating at CCRL).

The main challenger of of Lc0 will be Stockfish. It starts at the other end of the brackets and the two can meet only at the final. The first match of Stockfish is against the 31st seeded – Rodent III by Pawel Koziol (see Rodent’s rating at CCRL). Rodent comes with last minute improvements and although having little chance against the many times TCEC champion, it will be a nice test altogether.

Despite an easy pairing in round 1, both Lc0 and Stockfish will have a difficult way to the final. Last year the surprise was Houdini. This year the pitfalls along the way are many. Komodo is playing with a new stronger version and is keeping up with the pace of the favorites. The new NN engine AllieStein is bringing hard time to the top engines, often putting decisive pressure on them and even defeating Houdini. Komodo MCTS is also among the dark horses – it just does not lose against its brother Komodo or against Houdini, while it drew 6/7 games against Lc0 in the Premier division. Xiphos, a stable Ginkgo, the updated Fire, or a new Laser can also bring havoc to the field.

TCEC Cup 3 pairings

The TCEC Cup 3 seedings are traditionally based on the ongoing season. As TCEC_Spectator explains, “TCEC Cup 1 pairings followed entirely the standings of the regular season. TCEC Cup 2, and subsequent cups, will have the winner of the previous Cup seeded as #1, while all the other participants will be seeded according to the regular season. The Premier Division will give the rest of the top 8 seeds of the competition. Seedings based on Divisions 1, 2 and 3 are straight forward. The 6 engines that did not promote are seeded in order of finish – Division 1 gives seeds 9 through 14, Division 2 gives seeds 15 through 20, Division 3 gives seeds 21 through 26. That leaves Division 4a and 4b. The teams finishing #3 and #4 in the 4 engine playoff for promotion to Division 3 make up seeds 27 and 28 respectively. The final 4 seeds are the 3rd and 4th place finishers (that did not make the playoff) in each of the two divisions 4a and 4b. The 3rd place teams are seeds 29 and 30 with the engine with higher points given the 29th seed. Likewise the 4th place teams are seeds 31 and 32 (finishing with equal points but Rodent III had more wins so was given seed 31).”

The full pairings will be completed once the Premier Division is completed – follow it live here. For engine ratings check out the computer chess engine rating list CCRL

1. Lc0 (title holder)

2. Stockfish

3. Div P #3

4. Div P #4

5. Div P #5

6. Div P #6

7. Ethereal

8. Fire

9. Xiphos

10. Laser

11. Andscacs

12. Fizbo

13. Jonny

14. Chiron

15. Ginkgo

16. ChessBrainVB

17. Booot

18. rofChade

19. Fritz

20. Nirvana

21. Arasan

22. Texel

23. Vajolet2

24. Gull

25. Pedone

26. Nemorino

27. pirarucu

28. RubiChess

29. Wasp

30. Winter

31. Rodent III

32. Marvin

Categories: Ενημέρωση

TCEC Season 14 – the 14th Top Chess Engine Championship

Tue, 03/12/2019 - 02:59

Written by Guy Haworth and Nelson Hernandez
Reading, UK and Maryland, USA

Corresponding author: g.haworth@reading.ac.uk

This is the latest in our series of analytical articles on past TCEC events. The main text can be read below on this webpage, and at the bottom you will find a link to the full layouted article in pdf format, including the important tables, graphs and images.

TCEC is very grateful to the authors for their kind permission to publish these substantial and scholarly analyses of its events!

Introduction

TCEC Season 14 started on November 12th 2018 and introduced a number of changes from TCEC 13 (Haworth and Hernandez, 2019b). An enlarged Division 4 featured twelve engines and seven newcomers to accommodate the increasing interest in computer chess and this competition in particular. The other divisions remained eight strong. The five divisions played two or more double round-robins (‘DRR’) each with promotions and relegations following. Tempi gradually lengthened from ‘Rapid’ to ‘Classical’, and the Premier division’s top two engines played a 100-game match to determine the Grand Champion.

The trio of STOCKFISH, KOMODO and HOUDINI have dominated the TCEC medals for several seasons and a key point of interest was whether others would reach the podium. LEELA CHESS ZERO and ETHEREAL were certainly expected to perform well in Division P, having shown remarkable improvement in the previous few months. KOMODO MCTS was a dark horse.

There were a few nudges to TCEC’s adjudication rules. Draw adjudication could be invoked after move 35 (rather than move 40) and the two engines had to both evaluate within ±0.08 (rather than ±0.05) for eight consecutive plies and with plycount≠0. While draw-adjudication requirements were relaxed, win-adjudication requirements were tightened. Engine evaluations had to be outside ±10 (rather than ±6.5) for ten consecutive plies (rather than eight); plycount was not a factor. This change was welcomed by those of us who wanted to see a clearer demonstration of superiority on the board: it will be interesting to see how long it prolongs the decisive games and what mysteries remain.

The common platform for TCEC14 consisted of two computers. One was the established, formidable 44-core server of TCEC11-13 (Intel, 2017) with 64GB of DDR4 ECC RAM and a Crucial CT250M500 240 GB SSD for the EGTs. The ‘GPU server’, a Quad Core i5 2600k, was sporting Nvidia (2019) GeForce RTX 2080 Ti and 2080 GPUs for those engines which could exploit them.

The engines

Season 13 competitors BOBCAT, DEUS X, HANNIBAL and SENPAI rested for this TCEC season. TCEC welcomed first appearances for engines DEMOLITO, KOMODO MCTS, PIRARUCU, ROFCHADE, SCHOONER, SCORPIONN and WINTER, see Fig. 1 and Table 1.

Division 4: 2 DRRs, 4 round robins, 264 games, 30′+10″/m

As for TCEC12/13, each engine played both White and Black from four-ply openings defined by the second author here. The results are as in Table 2: ‘P%’ is the %-score and ‘ELO±’ is the change to the engine’s nominal ELO based on its performance. Generic stats are in Tables 9 and 10.

Online interest naturally focused on the newcomers, especially KOMODO MCTS (Chessdom, 2019), a further innovation from the Lefler/Kaufman camp. The engines had a wide range of ability leading to only 34.1% of games being drawn: those given a default ‘TCEC-entry ELO’ of 2900 ranged across the field. WINTER was always headed for a demotion spot. SCORPIONN clearly was not ready for the contest and even though it disconnected eight times, it did not impact the ranking elsewhere. The bottom three missed TCEC Cup 2 (Haworth and Hernandez, 2019c). The three engines promoted were clearly ahead: KOMODO MCTS, ROFCHADE and NEMORINO.

Division 3: two DRR phases, 14 rounds, 112 games, tempo 30′+10″/m

Again, the eight engines involved played both sides of 14 prescribed four-ply openings. With GPU operating temperatures more stable, LCZERO was expected to do well after its performance in TCEC Cup 1 (Haworth and Hernandez, 2019a) and it did not disappoint, see Table 3. KOMODO MCTS also distanced the rest of the field and continued on up the divisions. Crashes remained a problem: this time, HANNIBAL incurred five. In game 26/7.2, ROFCHADE disconnected in a 7-man tablebase drawn position. In g93, NIRVANA retained the KBNPKRN draw for 101 moves but claimed the 50-move draw with 165. Bd6 – which loses to 165. … Nf7+ 166. K~ Nxd6. Do chess programs do irony?

Division 2: two DRR phases, 14 rounds, 112 games, tempo 30′+10″/m

Game 64/16.4, KOMODO MCTS – LEELA, ended in a rare stalemate on m172. Game 93/24.1, NIRVANA–LEELA, was drawn at position 115b but a mate for Black in 29 moves when the 50-move draw rule intervened. Demoted GULL beat BOOOT and BOOOT beat LEELA which otherwise moved smoothly away to win the division again, see Table 4. The silver medal went to KOMODO MCTS, courtesy of one less loss to LEELA than XIPHOS and one more win to the rest of the field.

Division 1: two DRR phases, 28 rounds, 112 games, tempo 60′+10″/m

The penultimate game 28.3/111 was the longest ever for TCEC Division 1 at 308 moves: ‘new wave’ LEELA versus ‘old guard, oldest brand’ FRITZ 16. The win is routine enough with rook and passed pawn against a half-sighted bishop but endgame solver FINALGEN (Romero, 2012) sees 20 moves before a clear win, a line that results in mate on move 337 at best (Haworth and Hernandez, 2019d).

GINKGO surprisingly crashed four times and was disqualified so the formal results are slightly different from those of Table 5 even if promotions/relegations are otherwise unaffected. FRITZ never saw a win in this company and also was demoted to division 2.

Division P, three DRR phases, 42 rounds, 168 games, tempo 90′+10″/m

The line-up for Division P had only a semi-familiar look. After the TCEC13 podium trio of STOCKFISH, KOMODO and HOUDINI, we had the other survivors FIRE, ETHEREAL and ANDSCACS. Interest however centered on the newcomers LEELA CHESS ZERO and KOMODO MCTS, both bringing MCTS search to the game. The contest was three DRRs rather than the four of TCEC13.

After the first round-robin, STOCKFISH had jumped out into the lead with four wins. After the first DRR, with colour-bias eliminated, STOCKFISH maintained a healthy lead and remained unbeaten, a feat shared with KOMODO and LEELA. Was the TCEC podium about to change? KOMODO MCTS had disconnected and lost twice against KOMODO in drawn positions. A third disconnection would be bad for both engines: disqualification for MCTS and elimination of Komodo’s crash-wins from the table.

Game 64 saw STOCKFISH beat KOMODO, opening the door for LEELA. In game 68 at the foot of the table, ANDSCACS beat ETHEREAL with Black. At the half-way point, LEELA was edging the contest for second place and remained unbeaten. The fourth round-robin saw LEELA consolidate its second place with four straight wins against the tail including one as Black against ETHEREAL. The competition for second place remained open as STOCKFISH finally ended LEELA’s unbeaten run in the last RR4 game, g28.4/112.

The fifth round-robin saw plenty of drama. LEELA lost as Black to both KOMODO and FIRE, the first having serious tie-break significance and the second being seriously unexpected. GPU fan-settings were thought to be a contributory factor but not enough to trigger replays. In game 33.1/129 v HOUDINI, KOMODO MCTS disconnected for a third time, was disqualified and relegated with its games discounted. Hopefully, Mark Lefler will sort out the technical problems for TCEC15. This restored LEELA to second place. With one round-robin to go, adjusted scores at the top were STOCKFISH well clear on 21, LEELA 16.5, KOMODO and HOUDINI 16. The second relegation spot was between ETHEREAL on 11.5 and ANDSCACS on 11.

Every win was now going to be a major event, especially as the last round of 28 games started with seven draws. KOMODO as White lost to STOCKFISH in g37.4/148. Both LEELA and KOMODO beat FIRE. In the penultimate game, KOMODO beat ANDSCACS: ETHEREAL breathed again, having narrowly survived without a single win in this division. In the last game, a cliffhanger, STOCKFISH searched the endgame tables a thousand times more than LEELA and thought it had a feasible advantage, but LEELA held out in KRPPKRP to draw on move 93.

The raw figures of Tables 6 and 7 need adjustment because KOMODO MCTS’ disqualification flipped the ranking at both ends of the table. In fact, STOCKFISH ultimately had 25 points, LEELA 20, KOMODO 19.5, ETHEREAL 14 and ANDSCACS 13.5. The ‘big three’ became the ‘big four’ but the Shannon-AB engine mould was cracked again: the still-improving LEELA had remarkably progressed from Division 3 all the way to the TCEC Superfinal.

As in TCEC13, a knockout event was interposed between this tournament and the Superfinal. Would the LEELA team roll out an improved network in preparation for the big finish? A hint came in a ‘bonus match’ between a more recent ‘LEELA 32585’ and ‘STOCKFISH 8’, the latter having only 12 threads and a 4M hash-table. This was an echo and ‘simulation’ of the ALPHAZERO–STOCKFISH match: LEELA won +24=71-5. We reported on TCEC Cup 2 separately (Haworth and Hernandez, 2019c).

The TCEC14 Superfinal match: 100 games, tempo 120′+15″/m

TCEC’s ELOs suggested a STOCKFISH win by eleven. However, both engines came to the board in new versions: the match was now STOCKFISH v190203 versus LEELA v20.2-32930. There was bound to be a clash of styles occasioned by the different modes of evaluation and use of hardware. This dynamic was eagerly anticipated with viewer numbers often topping 2000. Jeroen Noomen (2019) again created a suitable opening book, aiming as before for at least 20% decisive results. Assaf Wool returned from his ‘TCEC Cup break’ to comment on all the games. GMThechesspuzzler and Kingscrusher were active on Youtube (Wool (2019) picked out positions from games 7, 8, 10-11, 13, 16-17, 20-22, 25, 27, 29, 35, 41, 49, 53, 55, 58, 63, 65-66, 71, 75, 80, 85 and 87. Kingscrusher (2019) commentated on games 7, 10-11, 13, 16, 17, 53, 66 and 85. Games 2, 7-8, 13, 17 20, 29, 49, 65-66, 80, 85 and 100 were covered by GM Thechesspuzzler, 2019). Soren Riis provided the authors with detailed analysis of games 7-8, 20-22, 65-66 and 71 which we provide via our pgn file for reader convenience rather than here. GM Matthew Sadler (2019), having analysed the STOCKFISH–ALPHAZERO games (Sadler and Regan, 2019) has also contributed his own view of this Superfinal.

The play and the results did not disappoint. STOCKFISH opened its account with wins from games 7 and 10 but LEELA replied with wins from games 11 and 13. There were twelve wins in the first thirty games, a hit rate of exactly 40%, see Table 8 and Fig. 2. At this point, the score was 15-15, suggesting that this would be the closest TCEC Superfinal since Season 5 in 2013 even though LEELA had never led. The same situation appertained at 24-24 after a run of 19 draws (not a record: the TCEC8 KOMODO 9.3x – STOCKFISH 021115 Superfinal games 14-37 and 47-71 were all draws). At this point, LEELA dramatically jumped out front with wins in games g49 and g53. This lead held until game 80 which STOCKFISH won. Ultimately, it was the single 0-1 win in another sea of 19 draws that allowed STOCKFISH to retain the title. Each game was closely contested with average length being one ply short of 100 moves – and not just because LEELA was reluctant to visit the draw zone.

Of course, suitably equipped grandmasters could write a book about this entirely gripping match and this would be most welcome. Here, we can only pick out a few chessic highlights which perhaps complement the analyses of the commentators above.

The hints from the evaluations of STOCKFISH suggest that it welcomed LEELA’s 15. Bb2 (g07), 51. … Be3 (g08, a missed win), 34. Kf1?? (g21) and 31. … Qd6 (g22). In game 35, 29. Ke1 rather than h7 seemed to lose LEELA’s winning advantage. Game 58 was adjudicated with a rare ‘mate in one’ on the board: the camera cut away just before the blow was struck. Game 63: LEELA was happy to trade pawns for position as early as eleven moves into the play. STOCKFISH did not see a serious problem until six moves later. LEELA create a passed pawn despite being three pawns down and this led to a crushing 41-move win, the shortest of the match.

If there was a pivotal juncture in this Superfinal, it was games 65-66 – a crucial one or two-point swing to STOCKFISH. In game 65, LEELA missed a KNP(c4)P(d5)KBP(c5) win with the winning capture admittedly 26 moves down the line (de Man, 2018). STOCKFISH clearly saw it was lost and LEELA would have been awarded the win under the TCEC13 ‘6.5+’ win-adjudication rule. LEELA was within 11 ply of winning with 9 ply to go and it is worth speculating as to how soon it would have found the winning idea, K on b5/c6 before Nxc5, had the plycount not intervened. Game 66 had to be restarted after two server crashes before LEELA – lost. Had it been possible to return to the game-state after the last completed move, the temperature of the partisanship in the chat room would have been lower. A minor cost, but transaction-checkpoint/restart might be applicable here.

Game 85 was the final win: the 12-move King’s Indian opening had already defined the major asymmetry of Queen versus BBPP. LEELA went from apparent equality to negative territory by move 25. Ultimately, LEELA’s QR were unable to prevent mate by a BBNNPP team, only five moves away when the referee stepped in. Game 86 was the longest ever TCEC game at 362 moves.

The Bonus 4-way and 2-way Rapid events

TCEC treated us to two bonus events at the Rapid tempo of 12′+3″/move. The first featured the top four - HOUDINI, KOMODO, LEELA CHESS ZERO and STOCKFISH: 20 DRRs, 40 round robins, 120 rounds and 240 games. STOCKFISH had a good first half and was never headed even if pursued closely by LEELA. HOUDINI and KOMODO tailed off, eventually in that order as KOMODO fared poorly in the second half. ELO-predicted net scores were +9/+1/-2/-8 but ‘actuals’ were +12/+6/-7/-11. The longest wins were g116.1 (1-0, 139 moves) and g37.2 (0-1, 125m): the longest draw, g12.1 (318m). Game 18.1 between Leela and Komodo was something of an anti-climax as a 3x-repetition draw after ten played moves. Full details are included with the repository e-version of this note (Haworth and Hernandez, 2019d).

The second event was a 100-game STOCKFISH–LEELA match from the initial position: no prescribed openings. LEELA won 16-4, perhaps by being single-minded about its openings (Wool, 2019).

Summary

The Google DeepMind company in St. Pancras, London have been remarkably open in sharing the core ideas of their intelligence initiative. In the year it has taken for DeepMind’s papers on ALPHAZERO (Silver et al, 2017/18) to mature and satisfy the referees, we have seen TCEC invest in Nvidia GPUs and foster several innovations going beyond the classic Shannon (1950) minimaxing AB model of a chess engine. We have seen a leading chess-engine author, Mark Lefler, move his focus successfully from top engine KOMODO to KOMODO MCTS (Chessdom, 2018). With one less technical break, this engine would have come all the way through the divisions to fully justify its place in Division P at the first attempt.

We have also seen a community come together to support and train the open-source LEELA CHESS ZERO echo of ALPHAZERO. Again, this has been rewarded by success, and how. LEELA edged out KOMODO and HOUDINI to take the challenger’s place in the Superfinal here. It was not expected to beat STOCKFISH but came within one game of drawing the classic phase.

Chess24 and Chessbomb, with its useful colour-coding of moves, covered the TCEC14 Superfinal so we were treated to kibitzing by three different, objective but hardly neutral versions of STOCKFISH. The Twitch TCEC channel claims that viewers’ computers have to date had a window open to TCEC Seasons 10-14 for a total of over half a million hours.

REFERENCES Full article

To read the full article in pdf, click HERE

published March 11, 2019

Categories: Ενημέρωση

TCEC Season 15 – To A/B or NNot to A/B, That is the Question

Tue, 03/05/2019 - 17:14

Season 15 of the Top Chess Engine Championship will begin this March 5th at 20:00 CET. It will feature a record number of engines to accommodate the booming sector and a record ELO of the participants. A total of 44 engines will face off in a division based system to determine the current status quo of strength and the grand champion of TCEC.

Stockfish has dominated the Top Chess Engine Championship for four consecutive seasons or a total of 10 months. It has a chance to expand this dominance in Season 15 for over 1 year (see the analysis of GM Matthew Saddler). Stockfish will have to show its best as significant challenge is coming from DeepMind’s Alpha Zero open source sister Leela Chess Zero. LCZero applied significant pressure in the S14 Superfinal, losing by a narrow margin 49,5-50,5. It also convincingly won its first major title the TCEC Cup and the inter season rapid bonus, thus showing high inspiration and determination for the title. These two engines, however, cannot relax, as Komodo with its innovative MCTS version and all other engines from the higher divisions will also bring in their best. Additionally, a new unique supervised learning neural network Allie+Stein will start from the bottom division and is aspiring to attack the top from its very first season.

Where to follow the Top Chess Engine Championship?

The Top Chess Engine Championship has a dedicated website with games going on 24/7 at http://tcec.chessdom.com . You can enjoy the games with live commentary by BlueFish (Stockfish running on 172 cores, represented by a blue line on the graph), Redmodo (Komodo on 128 cores, represented by a red line on the graph) and chat with engine authors, computer chess experts, and friends in the ever lively TCEC chat.

The previous season was record breaking in audience, including in the video broadcast of the event. A total of 500 000+ viewer hours were registered in the past 30 days, which is #1 position on Twitch among the chess channel, and higher than 50% of the total chess audience viewership altogether. You can follow the alternative video broadcast at TCEC Twitch TV

TCEC is the most watched chess channel and competition by far, according to the official Twitch Metrics

Growth of TCEC session viewers season by season from 2015 to 2019 visualized by Google analytics. S14 was a !boom (click on the image to expand)

TCEC Season 15 participants

Premier Division

1. Stockfish
2. LCZero
3. Komodo
4. Houdini
5. Fire
6. Ethereal
7. Promotion from Div 1
8. Promotion from Div 1

Division 1

1. Andscacs
2. Komodo MCTS
3. Fizbo
4. Chiron
5. Laser
6. Jonny
7. Promotion from Div 2
8. Promotion from Div 2

Division 2

1. Fritz
2. Ginkgo
3. Xiphos
4. Booot
5. Nirvana
6. ChessBrainVB
7. Promotion from Div 3
8. Promotion from Div 3

Division 3

1. Gull
2. Texel
3. Arasan
4. Vajolet
5. Pedone
6. rofChade
7. Promotion from Div 4
8. Promotion from Div 4

Division 4

Division 4 of the Top Chess Engine Championship will evolve this season 15. It will expand to accommodate all engines that are active and have ELO above 3000. It still remains testing ground for new engines and as support for the computer chess field, however, one cannot ignore that from this division have started engines like LCZero, Komodo MCTS, Ethereal, ChessBrainVB, etc.

Division 4 will be with two groups, !division4a and !division4b. Each will contain ten engines that will play a 1xDRR tournament with time control 30 min + 5 sec. The top two engines of the groups will promote to a playoff – !div4playoff. That will be a 2xDRR event which will determine the two engines advancing to Div 3.

Here is the full list of participants

Division 4 A

1. Winter
2. Tucano
3. Topple
4. Rodent III
5. Nemorino
6. The Baron
7. Cheese
8. Igel
9. Minic
10. RubiChess

Division 4 B

1. Pirarucu
2. Allie+Stein
3. Wasp
4. Chess22k
5. Marvin
6. Monolith
7. ScorpioNN
8. Jumbo
9. Bagatur
10. Gaviota

Rules and regulation changes for Season 15

The rules set of TCEC advances one more step this season 15, to resolve specific cases that arose in previous seasons. Besides the expansion of Division 4, here is what will change in all divisions:

* Increment in Div P, Div 1, Div 2, Div 3, and Div 4 will be 5 sec per move. The Superfinal increment is reduced to 10 sec per move

* Black wins will be removed from the tiebreak criteria, due to the usage of advanced books. The order of tiebreak criteria is as follows 1: # of crashes. 2: Direct encounter. 3: # of wins. 4: SB. 5: TD decision

* In case of a server disconnect or other interruption not caused by the engines:
**If the web server crashes, the game continues unaffected, and broadcasting will be resumed as soon as possible;
**If the evaluation of both engines is >=ABS(10) at the moment of game server interruption, the game is scored as a win ;
**If the evaluation of both engines is <=ABS(0.10) at the moment of game server interruption, and both engines have completed 35 moves, the game is scored as a draw;
**If a game interrupts with 7 pieces on the board, position on the board at the instant of game server interruption will be adjudicated according to 7-man EGTB.
** In all other cases the game is restarted from the position that the two engines reached before the disconnect, with time compensation to fill up the cache

* The top 4 engines of !div4a and the top 4 of !div4b, so 8 in total, qualify for the TCEC Cup. The engines in the playoff are seeded by their final standings, the other ones are placed on a random draw seeds 29-32

There will be no change of the 3 strike out rule this season, due to the multiple cases that have to be covered. An engine update during a division will count as a strike.

The engine uniqueness requirement stays intact , as worded in the S14 rules and regulations.

Categories: Ενημέρωση

Allie+Stein, the new neural network entering TCEC S15

Sat, 03/02/2019 - 22:25

The new neural network entering TCEC Season 15

Season 12 of the Top Chess Engine Championship saw the arrival of NN engines, modeled after DeepMind findings Alpha Zero, into public computer chess events. At that season LCZero played with CPU only. In Season 13 it was handed a GPU, and only a season later that neural network won its first major title.

Besides LCZero, one more neural network entered TCEC. It was Scorpio NN by Daniel Shauwl. And even though it did not make it past Div 4, it showed a trend in computer chess: people understood the potential value and advantages of neural networks in chess.

Season 15 is about to start and one more neural network is going to enter TCEC. This is a combined project by Adam Treat and his Allie and Mark Jordan and his Leelenstein. The new engine is called Allie+Stein, a unique engine by the TCEC rules that will start its quest for top positions and climb of the ladder from Division 4.

Here is an extensive interview with the authors Adam Treat and Mark Jordan.

Season 15 of TCEC will start this March 4th, live on the official website of TCEC and the TCEC Chess TV.

Your engine Allie+Stein will be the new neural network in TCEC, making its debut this Season 15. Welcome to the Top Chess Engine Championship!

Adam Treat: Allie is a very new chess engine, but represents a lot of hard work over the last several months, so I’m hoping TCEC can provide an opportunity to see how she stacks up against a host of more established engines. Combining Allie with the Leelenstein network will also be interesting given that both introduce new avenues of research in NN chess engines. I an new to the chess programming community and culture and so I am excited to participate and learn through TCEC.

Mark Jordan: I am excited to see how far an engine that uses supervised learning (SL) exclusively can go and I hope there will be more strong SL networks to compare against for benchmarking average performance and maximum performance of the method.

So far there have been two unique NNs in TCEC – LCZero and Scorpio. What makes your engine the third unique NN engine by the TCEC rules

AT: Like Leela, Allie is based on the same concepts and algorithms that were introduced by Deepmind in the AlphaZero paper(s), but her code is original and contains an alternative implementation of those ideas. You can think of Allie as a young cousin of Leela that can utilize the same networks produced by the Lc0 project or other compatible networks. The Leelenstein network is also a novelty in that it introduces supervised learning into the TCEC competition. Finally, Allie+Stein will be using MCTS for the beginning portions of the tournament, but I’m hoping to switch to AlphaBeta search during later rounds… if she makes it that far :)

MJ: Allie+Stein is a completely new engine and neural network produced, thus easily satisfying 2 out if the 3 conditions for uniqueness. It is possible eventually that I will rewrite the training scripts completely with some more new ideas in the future. Currently, training has some changes to the Leela training scripts.

Can you share more on how is Allie+Stein engine being trained?

MJ: Training started from a random initialized network, and consisted of mostly CCRL computer games (I used all of them available), and some weak Leela t30 games and some games from other experiments I was able to gather. These were were all about 100 elo weaker than t10. I tried several new learning techniques using these same games. My goals have mostly been focused on NN learning experimentation. I compared the learning schedule and optimizer that Leela used to different ones from some academic papers, and it seemed to improve performance, but it was still about 50 elo weaker than the best Leela nets. So I replaced all the weak t30 games with many more recent ones, using all of January’s games, while still keeping the CCRL games in the training window, and continuing on several more of my cyclical learning rate cycles.

And how about search?

AT: To begin the tournament, Allie will perform MCTS based search with absolute fpu where new nodes start off with win pct of -1. The search is modeled after Deepminds paper’s. As I said above, I’m hoping to switch to AlphaBeta for the long term direction of the project. I’ve experimented with many, many ways of doing this with the networks generated by the Lc0 project and I think I’ve hit upon a way to achieve the depths required to maintain ELO level with MCTS based search, but it is not ready yet. In the future, I imagine we’ll see a lot of experimentation with different variations of search (mcts, ab) + eval (handwritten, NN) in computer chess engines. Hoping to be a part of that and to contribute to the shared pool of knowledge.

What is the strength of Allie+Stein? What division do you expect to reach? Do you think stronger versions will come out as the season progresses and thus have a better shot at the Premier Division?

MJ: I think the neural network seems to scale pretty well with more nodes. In bullet testing it loses to Stockfish quite a bit, but seems to hold its own in tournaments with more time. I believe it has potential to get to division 1, but will require some more work to better utilize multiple GPUs and support tablebases. I have some ideas to use tablebases in learning as well that could end up making the neural network Premier Division material. But for this season I will be happy even with just getting to Division 2.

AT: This is a question I get quite often. Most people familiar with chess engines know that it is extremely hard to compare strengths other than through heads up competition with a set of match rules/controls in place. That is exactly what TCEC provides. A levelish playing field and a set of machines/rules agreed upon beforehand to determine the relative strength of different algorithms and their implementations. So with a huge grain of salt I’ll say that on my own rather meager hardware I have experienced heads up matches between Allie and Lc0 at short time controls with the same network with a relative difference of 50-100 elo in Lc0′s favor. Obviously, that is with only one GPU. Throw in the advanced multi-gpu hardware here at TCEC and that is a very big new variable. Add the other engines and we have another large unknown variable. Add in that we’ll be using a Supervised Learning network and yet another big variable. Then we have the fact that Allie will start off the tournament without TB support. Just lots and lots of variables. So, ok I expect she will be able to advance out of Div 4, but anything is possible. I’ll be happy if she is able to get winning positions and mate consistently in them :)

Comparing to the top NN engine now Lc0 , do you expect with your approach to have better future?

MJ: The beauty of my approach using only existing games is I can train whole new networks from scratch in a week or two to try a new idea, something that would take the Lc0 project several months, even with 100x or 1000x more compute than I have. I hope to eventually show that some of my ideas must be good by how strong the network is and Lc0 can try them to become strong as well. And I will also continue to try good ideas from there as well. And the project is a great resource for people who need many millions of strong games, as it already has many more games available than all of CCRL even though it is much younger. And sadly fishtest doesn’t store and host all of its games. So I expect to see some great symbiosis.

AT: I expect that whatever advances are made by one engine will (with time and effort) be incorporated into other engines. As they should be! We are all standing on the shoulders of the pioneers in computer science and chess engine programming that came before. I do think that AlphaBeta is a superior search method and that there is no reason that an AB+(eval method) engine can’t compete favorably with an MCTS+(eval method) regardless the eval method. But this is just my theory and worth very little until proven. Only the future will tell.

What are your thoughts on the current hardware balance at TCEC? Are you happy with the hardware your engine is going to play on?

MJ: There are always debates about hardware balance, and I think a wide range power’s are somewhat fair, and as long as the specifications and NPS numbers are published and maintained throughout the tournament it is a reasonable tournament data point. The balance seems to be within a 2x order of magnitude of fair based on any of purchase price, watts, and total cost of ownership metrics which I think is as close as it can possibly get to please everyone. But it does leave room for future debate and improvement. 2x the power or cost doesn’t translate to 2x the nodes so the real elo difference is probably not enough to change what division any engine will end up in most cases, even if it changes a result or two. The format is not guaranteed to find 20-30 elo differences anyway, so I don’t see it as a big issue except during boring games :)

AT: Considering that I’ve never personally tested Allie on this level hardware, sure I’m happy. I just hope she scales well enough.

Read: Stockfish dominance continues and Analysis by GM Matthew Sadler

Stockfish has been dominant at TCEC for almost 1 year.

AT: Stockfish is the strongest engine until proven (convincingly) otherwise. I have great respect for all the developers who work on Stockfish (and other engines for that matter) and think the community of chess programmers is pretty collegial.

MJ: I very much support non-private engines, so I am glad to see that there has never been a private engine to win the Superfinal. And even more glad that it seems that free and even open-source projects can win. I am glad such excellent chess is truly accessible to all.

Read: interview with Alexander Lyashuk

Alexander Lyashuk from Lc0 shared in an interview that he expects at least 5 NNs to appear and they to dominate computer chess. Do you share this vision? Do you expect one of those NN engines to be Allie+Stein?

AT: I do think it is only a matter of time before NN eval is shown to beat handwritten eval regardless the search method. Right now, if you limit the number of nodes – hands down – any of the NN’s will beat the traditional handwritten eval engines. This is just a fact. Still, I have great respect for the ingenuity of those writing the traditional engines. I hope Allie+Stein will be a meaningful engine in terms of helping to advance the state of art in computer chess programming in the near future.

MJ: As I mentioned above, I am excited to see more projects, and I hope there will be enough of them that we have to pick the most exciting and unique ones. We can use them to develop more tests to determine how unique they are in their ideas of openings and in general play, and which ones are truly beyond average. Of course I hope to have one of these beyond, so the top 5 sounds nice!

Categories: Ενημέρωση

The TCEC14 Computer Chess Superfinal: a perspective

Sat, 03/02/2019 - 02:56

The TCEC14 Computer Chess Superfinal: a perspective
by GM Matthew Sadler
London, UK

Season 14 has been amazing! The clash of styles between STOCKFISH and LEELA produces consistently interesting chess – just like the match between STOCKFISH and ALPHAZERO did. You really hope that this sort of clash of styles can continue for the next few seasons.

Just a few impressions about the games. I was really impressed by LEELA’s handling of the White side of G17, the Budapest. I’ve done a fair amount of analysis on this line and a few good players have got stuck against it. Aronian defeated Ivanchuk in 2013 at the London Candidates but more due to Ivanchuk’s terrible time trouble than due to getting anything out of the opening. Giving the c-pawn back very quickly is a great way of putting the pressure right back on Black. I also liked G45 a lot: a Hedgehog where LEELA generated a huge amount of play with some risky but incisive pawn play on both sides of the board. The LEELA win in G53 was extremely nice too: again, it’s impressive to see Black being stretched so greatly with rook’s pawn thrusts on both sides (18. a4 and 23. h5) then a thrust of the g-pawn (24. g6) followed by returning to the queenside with 28. c4. This sort of whole board vision is what grabbed me so much when I first played through ALPHAZERO’s games – the game ‘exactly how to attack’ (Sadler and Regan, 2019) is the best example of this – and it makes for wonderful chess when you have a player who can execute it so well.

I found G62 incredible: the move 10. … 0-0-0 would never have occurred to me! In some ways, I felt that LEELA was playing like STOCKFISH – grabbing a hot pawn and betting on surviving with a rather dodgy king – and STOCKFISH was playing like LEELA – sacrificing a pawn for open lines against the king. G63 was magnificent: perhaps the most ALPHAZERO-like game of the whole match. Material balance is not important, accurate assessment of an opposite-coloured bishop position, restriction of the opponent’s king, entombing of the Black rook on h8 (a little like the game ‘Python squeeze’ featured in Game Changer). It’s a classic game: it was wonderful to watch and I’ve played through it a few times since with great pleasure. G66 was a wonderful effort from STOCKFISH. It followed a STOCKFISH–ALPHAZERO game until move 16 in which ALPHAZERO was also under great pressure and the power of STOCKFISH’s attack was huge! G81 and G82 (the opening part) was a great pair of games in a sharp Slav gambit. G81 was particularly nice with rook’s pawn thrusts on both sides of the board from LEELA to tie Black up on the queenside – even at the cost of another pawn – while LEELA targets the kingside (it’s another theme you see quite often in ALPHAZERO’s games). G85 and G86 (the opening part) were also fantastic. STOCKFISH handles these positions with so much energy: quite amazing!

REFERENCES
Sadler, M. and Regan, N. (2019). Game Changer, esp. pp 38-43 and 99-100. New in Chess.

Communicating author: https://matthewsadler.me.uk/

Categories: Ενημέρωση

Stockfish continues to dominate computer chess, wins TCEC S14

Fri, 03/01/2019 - 23:05

The collection of Stockfish gold medals – six times winner of TCEC and one time winner of the Cup

Stockfish continues to dominate computer chess after defeating LCZero at the Superfinal of Season 14 of the Top Chess Engines Championship. This is a fourth consecutive title for the open source engine – after winning Season 11 of TCEC last April 2018, Stockfish never let go the title.

This time, however, it was a really close fight. Until the very last game the match could go in any direction. In the end Stockfish won 50,5-49,5 (+10 =81 -9).

Stockfish wins TCEC S11 / Stockfish wins TCEC S12 / Stockfish wins TCEC S13 / Stockfish wins TCEC S14

More about S14: Stockfish wins the Premier Division / LCZero wins the Cup / First major title for Leela / Interview with Alexander Lyashuk / Komodo MCTS – interview / TCEC S14 preview

 

Note: TCEC Season 15 starts this March 4th, you can watch it on the official website or at TCEC Chess TV

Bright start at the Premier Division

Defending its title by Stockfish in Season 14 started in the Premier Division. There Stockfish was merciless – it qualified for the Superfinal as clear first and without losing a single game. Considering this was the strongest Premier Division in the history of TCEC, the achievement of Stockfish may remain a unique feat for many seasons to come.

The Superfinal battle

For the first time in many years, the dominance of the big 3 – Stockfish, Komodo and Houdini – was disturbed this Season 14. The world’s strongest open source neural network LCZero managed to outwit both Komodo and Houdini and grabbed a last minute ticket to the Superfinal by finishing second at the Premier Division.

Few believed that LCZero could put up a serious fight in the Superfinal. The dominant performance of Stockfish in the Premier Division and the shaky play of LCZero were pointing towards a landslide victory for the reigning champion. But every chess fan was in for a surprise – LCZero entered an experimental network , which utilized the latest findings by DeepMind. The result was astounding – the Superfinal turned out to be the most contested TCEC title in the history of computer chess.

In the first three pairs of games – QGD, Tarrasch (von Hennig-Schara gambit), Sicilian, Najdorf (Byrne English attack), and King’s Indian (5.Nf3) – LCZero put up a serious fight and a good understanding of the position. The first decisive blow came in game 9, Stockfish caught LCZero unprepared in a King’s Gambit line.

Replay: Lc0 – Stockfish – King Gambit Accepted, Schallop defence 0-1

Right in the next pair of games, Stockfish converted its game with white in the Catalan opening, increasing the score to 2-0.

Replay: Stockfish – LCZero – Catalan opening

Just as everyone was saying “this is the beginning of the end of the Superfinal”, LCZero returned a game in the French, Tarrasch (Guimard main line) and yet another in Nimzo-Indian, classical (Pirc variation).

Replay: LCZero – Stockfish – French, Tarrasch (Guimard main line) 1-0
Replay: LCZero – Stockfish – Nimzo-Indian, classical, Pirc variation – 1-0

After an exchange of blows in game 16 and game 17 , it was time for Stockfish to speak – 3 consecutive wins!

Replay: Stockfish – LCZero (Ruy Lopez) 1-0
Replay: LCZero – Stockfish (Ruy Lopez) 0-1
Replay: Stockfish – LCZero (English, Bellon gambit) 1-0

One more time the same question, is this the end of the match? No! LCZero needed only 8 games to erase the three points advantage and level the score

Replay: LCZero – Stockfish – QGD, semi-Slav 1-0
Replay: LCZero – Stockfish – Pirc defense 1-0
Replay: LCZero – Stockfish – Dutch 1-0

Nineteen consecutive draws followed in which both engines had chances to score. LCZero was the last one to score before halftime, in a Benoni game

Replay: LCZero – Stockfish – Benoni 1-0

Thus, the two engines finished equal in the first half of the TCEC Superfinal. This was just a virtual mark, but now everyone was convinced – this Superfinal will not resemble anything seen before.

The exchange of blows continued with LCZero taking the lead:

Replay: LCZero – Stockfish – QGD Slav, 4.e3 Bf5 1-0
Replay: Stockfish – LCZero – King’s Indian, Kramer system 1-0
Replay: LCZero – Stockfish – French, MacCutcheon, Janowski variation 1-0

A decisive moment in the match came in games 65. LCZero created a winning position – Stockfish showed +153.1, showing that there is a tablebase win in this position. However, that was a 7 pieces tablebase win and no automatic adjudication happened. Rather the game continued and LCZero did not manage to convert.

Had it been a previous season, the game would have been adjudicated under the tournament rules that if the engines agree on a +6,5 eval for a certain number of moves the game is automatically scored. Under the new rules of Season 14, the engines were forced to show more understanding as to how to convert a given position, triggering the rule after a +10 eval. LCZero’s +8.92 evaluation was not enough and a virtually won game finished in a 50-moves rule draw.

Replay the dramatic game 65

Although this game will remain in the history of computer chess as one of the most dramatic games ever witnessed, the match was far from over as 35% of the games were still to be played. The tension increased game after game – LCZero was building up advantages, but Stockfish as a true “minister of defense” was bringing the games to equal score. What’s more, Stockfish started converting its own advantages and turned the match around.

Replay: Stockfish – LCZero – Queen’s Indian 1-0
Replay: Stockfish – LCZero – Sicilian 1-0
Replay: LCZero – Stockfish – King’s Indian 0-1

With that last win with black Stockfish took the lead in the match and did not let it go until game 100. A very difficult match for both sides which certainly deserved the popular name “first match of the end of an era”. So far the era of Stockfish continues full steam ahead. Season 14 is over with a fourth consecutive title by Stockfish. But there is no time to relax as Season 15 starts this March 4th – with a record number of engines, a new neural network, and the highest ELO ever seen in computer chess.

Follow the action live on the official website of TCEC

Categories: Ενημέρωση

Tiebreak looming at the TCEC Superfinal

Wed, 02/20/2019 - 22:42

Last November, the classical World Chess Championship between Magnus Carlsen and Fabiano Caruana was an amazing adventure of total equality – twelve consecutive draws and a need for a rapid tiebreak. Magnus Carlsen won the World Chess Championship 2018 title and many experts predicted that such an equal match will not be seen in the near future. They have all been proven wrong, as just several months later we are seeing a match for a world title which is more equal and for many fans it is dubbed as more dramatic.

84 games into the final of the Top Chess Engine Championship, the premier event for chess software, the reigning champion Stockfish and the challenger Leela Chess Zero are protagonists in an amazing battle. Despite the frequent exchange of blows between the two participants (a total of 18 decisive games!) the score is equal 42-42. A remainder of just 16 out of the total 100 games is left and the drama ensuing is unseen so far in the history of computer chess.

Leela Chess Zero entered the Superfinal as second in the Premier Division and as a TCEC Cup champion. It is the first neural network to ever reach the final for the highest chess engines title. Yet, in a poll before the match only about 35% of over 1000 people believed that LCZero had a chance, and of those people many admitted they chose the option just to support the engine as fans. When LCZero started winning games the surprise was huge and sent ripples through the chess world.

Only 16 games are left and anything is possible in the first “end of an era” match. Follow them live on the official website

TCEC Superfinal tiebreaks

The equal score after 84 games triggers high expectations for tiebreaks. In case of an equal score after game 100, there will be a rapid match of 16 games with a time control of 25′ + 10″ with random openings selected from earlier in the same Season. In case it is still tied there will be a Blitz match of 8 games with a time control of 3′ + 2″.

Bonus rapid match ?

There is also the possibility that one of the engines will prevail in the remaining classical games. In such a case, TCEC has prepared a bonus event for those fans that want to see a rapid event. It will be a one week event between the end of the Superfinal and the start of Season 15 Div 4 and will have the following rules:

* Four engines will be invited – Stockfish, LCZero, Komodo, and Houdini
* A total of 20 double round robins will be played
* The time control will be 12 min + 3 sec
* The first two engines in the standings progress to a 100 games rapid final

The rapid tournament will be played just for bonus purposes and fun. The real high quality chess happens only at long time controls, which allows the engines to show their best. Yet, it will be a fun event that many of the fans that enjoy the Superfinal would like to see.

Categories: Ενημέρωση

TCEC Cup 2 report

Wed, 02/13/2019 - 00:46

Written by Guy Haworth and Nelson Hernandez
Reading, UK and Maryland, USA

This is the third in a new series of analytical articles on past TCEC events. The main text can be read below on this webpage, and at the bottom you will find a link to the full layouted article in pdf format, including the important tables, graphs and images.

TCEC is very grateful to the authors for their kind permission to publish these substantial and scholarly analyses of its events!

Introduction

The knockout format of TCEC Cup 1 (Haworth and Hernandez, 2019a/b) was well received by its audience and was adopted as a regular interlude between the TCEC Seasons’ Division P and Superfinal. TCEC Cup 2 was nested within TCEC14 (Haworth and Hernandez, 2019c/d) and began on January 17th 2019 with 32 chess engines and only a few minor changes from the inaugural Cup event. The ‘standard pairing’ was again used, with seed s playing seed 26-r-s+1 in round r if the wins all go to the higher seed. Thus, seed s1 plays s32, s16, …, s2 if all survive long enough. STOCKFISH was top seed as the TCEC Cup holder but would have been anyway because of its TCEC14 placing as used for the other engines. It is worth noting that the TCEC14 discounting of HANNIBAL’s games because of ‘technical breaks’ affected others’ seeding. PEDONE was s23 rather than s21: ARASAN and VAJOLET stepped up one, given the residual points and tiebreaks. Fig. 1 depicts the logos of the engines in seed order. Basic engine details have been published elsewhere (CPW, 2019; Haworth and Hernandez, 2019d) but some fourteen engines as indicated in Fig. 2 were upgraded for the Cup, again a testimony
to the energy and enthusiasm of their authors.

The format was of 8-game matches at the Rapid tempo of 30′+5″/m rather than the ‘+10″/m’ of TCEC Cup 1. This time, all eight games were played whatever the running score. Openings were repeated with colours reversed after every odd-numbered game. The first few ply in all games were randomly selected from two sets of openings created by the second author here, their relative frequency reflecting that seen in human play: over 200 four-ply openings constituted the repertoire for the opening round of 32 and over 300 twelve-ply openings served thereafter. Tiebreaks were resolved, this time at end of round, by further pairs of games with openings after game 16 from 232 TCEC Superfinal 9-13 options. There was no Armageddon backstop even though the longest TCEC Cup 1 match went to 20 games. Adjudications were as for TCEC14.

As in TCEC Cup 1, interest focused on actual performance ‘%P’ compared with expected performance ‘E%P’ implied by TCEC ELO difference ‘ELO Δ’. The accuracy of the TCEC ELOs, the upgrades to nearly half the field and the character of the random openings would be the main influences.

Round 1

As expected, STOCKFISH opened its campaign with an 8-0 salvo. ‘LC0’ LEELA CHESS ZERO also achieved this feat but was notably more cautious in securing its wins, preferring the gentle ascent of sunlit uplands to a knife-edge scramble up a slippery mountain ridge with its greater risks. A newly refreshed ETHEREAL also scored the whitewash that three higher seeds did not. KOMODO, HOUDINI (twice), FIRE, CHIRON, LASER, FRITZ, GINKGO and remarkably, middling seeds XIPHOS and BOOOT qualified without losing a game. JONNY staged the one ‘come from behind’ victory at the last gasp, winning games seven and eight against ARASAN. The only upset of the seedings came in the third match, new-version ROFCHADE scoring 4½ rather than an expected 1½ and furthermore, ensuring that TCEC14 Division 3 would be represented in the second round.

Surprisingly, there were no ‘Act II’ tie-breaks to play out, perhaps because of the halved 5 move-increment. However, Round 2 was delayed by the setup and replay of the HOUDINI match, this time featuring the submitted WASP 3.54 which had, in error, not made it to the board in the normal course of events. Both matches are included in Table 1. The openings used were different but the end result was the same and even more in HOUDINI’s favour.

Round 2

Eleven of the sixteen victorious engines were sporting new versions for this event so TCEC’s ELO ratings were going to get a more severe examination. Only FIZBO was missing from the fancied half of the line-up. At this point, the second author here drew on his bank of 12-ply rather than 4-ply openings, arguably to the disadvantage of LC0 according to its proponents.

STOCKFISH opened the round as befits the Cup holder – 7-1, the ELO prediction. Game five ended in repetition after only ten calculated moves when the engines bit on a Pringle, a saddle-shape in their joint evaluation surface. GINKGO scored two firsts, one bad one good: the first ‘disconnect’ of the event and the first round two win by the underdog, albeit after the match was decided. The team of unfancied seeds did better than par but still, they all lost: there were no ‘come back’ wins or tiebreaks.

The quarter-finals, semi-finals, small final and final

The locker room for the quarter-finals therefore had the look of a Division P reunion, the top eight seeds perhaps thinking of negotiating their own TV contract. Who knows where AI may lead. STOCKFISH came through easily enough and again without loss but just short of ‘ELO expectations’. HOUDINI and FIRE also came to their match without a loss: something had to give. In their last 22 TCEC games, HOUDINI was +2=19-1 but FIRE came in a new version. The result was the first 4-4 tiebreak, indeed a deadlock of 8 draws – an echo of Carlsen-Caruana, 2018. The tiebreak was played after the other two matches, with the engines unfortunately not able to consider their positions meanwhile.

KOMODO–ETHEREAL was on next. The drawfest continued with the exception of the fourth game in which KOMODO emerged three pawns to the good in the ending. The TCEC adjudication pre-empted an interesting demonstration of technique here. ‘LC0’ LEELA began confidently enough with a win as Black against ANDSCACS: it is as well that engines do not get discouraged. Two more draws followed: it is tough at the top, defences are strong and, though the odd result may go one way or the other, there is little to choose between these engines. ANDSCACS however was notably devoid of the EGT support that LC0 enjoyed. With the advantage of two connected passed pawns and a neat R-v-BN demonstration, LC0 won the ending of game 4: match-score 3-1. LC0 continued undefeated and even won the last game to return arguably the most impressive performance of the quarter-final. The HOUDINI-FIRE tiebreak finished the round with, surprisingly, a 0-1 win for HOUDINI after just two more games. In this way with some suspense, the top four seeds went forward to the semi-finals.

The semi-finals opened with a remarkable nineteen draws: eight by STOCKFISH–HOUDINI and three more after the eight by LC0–KOMODO. Has this ever happened before in computer chess? The deadlock was broken when HOUDINI took control of the open d-file on move 58, eventually lined up its Alekhine’s gun (Chessgames, 2019a/b) on move 80, cramped up Black’s position, created a passed-pawn with a neat tactic, and pushed that pawn to the seventh rank on move 100. STOCKFISH may have seen the advantage before HOUDINI but even so, it was too late to counter. With STOCKFISH a minor piece down, the ‘TCEC win’ adjudication followed quickly. Maybe not the biggest-shock result in TCEC history but certainly the biggest shock-result, especially given that HOUDINI had not been updated for some eighteen months. HOUDINI can beat STOCKFISH in a short match even if this is odds-against. ‘Kingscrusher’ (2019a) covered this game well on the day.

And so to the tiebreaker face-off between KOMODO and LEELA CHESS ZERO, the latter not an engine to rush its pleasures or its recognition of the TCEC deci-pawn ‘draw-zone’. This would be a tight match, KOMODO having beaten LC0 +1=5-0 in their recent head-to-head but LC0 was in a new version here, improving on even the LC0 that earned a place in the TCEC14 Superfinal. Sad that psychology does not play a part in computer chess. Eight draws took us into the second phase of the tiebreak, featuring Jeroen Noomen’s TCEC 9-13 Superfinal Opening book. This, by design, includes more volatile opening positions than other TCEC opening books, thus promoting a higher proportion of decisive games. Even so, LC0 surprisingly did the double immediately to qualify for the final. TCEC at this point introduced a bronze medal ‘small final’ between STOCKFISH and KOMODO which STOCKFISH duly won +2=5-1 sustaining only its second loss of the tournament. That game was the second half of a 1-0/1-0 pair so perhaps the opening handed too big a carrot to White. Apart from that game, KOMODO did not seem to build advantage.

The contest opened with seven draws and we pass quickly over to the game in which LC0 as White scored the decisive win. The evaluation, depth and EGT-support curves of Fig. 2 tell a tale. LC0 dramatically revised its expectations on finding 27. h5, Figs. 2 and 3a. Its situational awareness was clearly superior: it was significantly ahead of HOUDINI which only grew alarmed after 29. f4! and 32. … Nbd7, Fig. 3b: it was even ahead of the mega-threaded and 7-man-EGT-armed STOCKFISH, usually the leading prophet of doom. There is hope for humans yet: the audience had been saying for some time that the optics were at least dramatic – White with two bishops eyeing the black king’s h8-corner, relatively developed pieces and more freedom. LC0 was notably less dependent on depth of search as

the game went on and consulted the EGTs far less than HOUDINI did. After 35. Nxe5, Fig. 3c, LC0’s confidence increased again and after 39. … Rb8 as in Fig. 3d, remarkably the best option, HOUDINI was the exchange down and clearly in trouble at ‘-3’. With 54. Rb3 as in Fig. 3e, LC0 gave up a rook for a knight and converted pawn but by this time both engines knew the game was over. LEELA superfan ‘Kingscrusher’ (2019b) covers this game in detail, indicating some even less attractive byways for Black.

So, this special moment took on extra significance. An engine based on an artificial neural network and ‘MCTS’ Monte-Carlo Tree Searching had taken top step on a TCEC podium. Congratulations must go to Gian-Carlo Pascutto, Gary Linscott, Alexander Lyashuk (Chessdom, 2019) and the choir of angels ‘in the cloud’ who contributed to its self-play training.

A summary

Clearly, the Shannon (1950) mould, now over sixty years old, has been badly cracked if not completely broken. How this has been done is only semi-clear and even then, only to a few. Many will want to do their homework on the new paradigm, the ‘DCNN’ deep convolutional neural network architecture as exemplified by ALPHAZERO and LEELA CHESS ZERO. The ‘zero’ indicates that neither has been trained on human games but there is human input in the adoption of the DCNN approach and, of course, in the definition of the game domain and of what constitutes learning. We hope that further reading of the available sources (Sadler and Regan, 2019; Silver et al, 2017/18) will demystify the magic of these new machines.

Advised by Sadler (2018), what can we say about LEELA’s style of play? Compared to what we have seen before, it seems to be more strategic and less tactical, more interested in space and movement than in material, more interested in the prophylactic reduction of its opponent’s options. Questions remain about LEELA’s tactical ability where one would expect minimax to be more effective, where a narrow ‘study like’ path has to be found despite the presence of serious dangers.

MCTS has changed the game in several domains as evidenced by past articles in this journal and results at past ICGA Computer Olympiads. Was TD-GAMMON (Tesauro, 1995) the first demonstration of the new way? The combination of two concepts, MCTS and DCNN, has clearly been key: the future may combine further sets of ideas, e.g., MCTS and minimax-search. When should resources be shifted from one towards the other? We can expect to see new types of computer-chess engine in the future.

On a broader canvas, it appears that once again computer chess has been the Drosophila melanogaster fruit fly vehicle of choice for artificial intelligence research, showing the way forward for new kinds of analysis and insight. Improved early-detection in the medical field, driven by better computer vision, is a laudable priority with potential impact rivalling Hopkins’ invention of the fibrescope and his contribution to medical optics (Hopkins and Kapany, 1954/5; McCombie and Smith, 1998).

Wrapping up this report, we give the usual generic statistics on the defined openings and game-lengths, plus a selection of interesting endgames that arose, see Tables 6-8. Fuller data and the somewhat annotated pgn files of TCEC Cup 2 are available (Haworth and Hernandez, 2019c). Congratulations to the winning and unbeaten LEELA CHESS ZERO team (Linscott, 2018), also focusing on their TCEC14 Superfinal against STOCKFISH (Haworth and Hernandez, 2019d). Special thanks should also go to all the semi-finalists who produced such close matches: there is little between them. Once again, let us point out that this event would not be the same without the administrators’ efforts, the participation of the entire field of thirty-two engines and the audience in the ‘online lounge’ who contribute an enjoyable mix of questions and information, leavened by interspersed jokes and off-topic comments.

References
  • Chessdom (2019). https://tinyurl.com/icgaj048. Interview with Alexander Lyashuk, core member of the LEELA CHESS ZERO team.
  • Chessgames (2019a). https://tinyurl.com/icgaj041. The ‘Alekhine’s Gun’ game against Nimzovitsch.
  • Chessgames (2019b). https://tinyurl.com/icgaj052. Lagno–Wenjun, WWCC 2018, Alekhine’s Gun.
  • CPW (2019) https://tinyurl.com/icga046. Biographies of chess engines, authors and developers.
  • Haworth, G. McC. and Hernandez, N. (2019a). http://centaur.reading.ac.uk/80284/. TCEC Cup 1. This note plus annotated statistics and pgn files. Submitted to the ICGA Journal.
  • Haworth, G. McC. and Hernandez, N. (2019b). http://centaur.reading.ac.uk/78820/. TCEC13: the 13th Top Chess Engine Championship. Submitted to the ICGA Journal.
  • Haworth, G. McC. and Hernandez, N. (2019c). http://centaur.reading.ac.uk/81390/. TCEC Cup 2. This note plus annotated statistics and pgn files. Submitted to the ICGA Journal.
  • Haworth, G. McC. and Hernandez, N. (2019d). http://centaur.reading.ac.uk/82052/. TCEC14: the 14th Top Chess Engine Championship. To be submitted to the ICGA Journal.
  • Hopkins, H. H. and Kapany, N. S. (1954). A Flexible Fibrescope, using Static Scanning. Nature 173, 39-41.
  • Hopkins, H. H. and Kapany, N. S. (1955). Transparent Fibres for the Transmission of Optical Images. Optica Acta, 1(4), 164-170.
  • ‘Kingscrusher’ (2019a). http://tinyurl.com/icgaj039. HOUDINI-STOCKFISH, semi-final game 12.
  • ‘Kingscrusher’ (2019b). https://tinyurl.com/icgaj040. LC0-HOUDINI, final game 5.
  • Konoval, Y. (2019). Private communication of DTC(onversion) depths for some positions.
  • Linscott, G. (2018). https://github.com/LeelaChessZero/lc0/wiki LC0 on Github.
  • McCombie, C. W. and Smith, J. C. (1998). Harold Horace Hopkins. 6 December 1918 – 22 October 1994. Biographical Memoirs of Fellows of the Royal Society, 44, 239–252.
  • Sadler, M. (2018). https://tinyurl.com/icgaj045. Sadler on ALPHAZERO’s play.
  • Sadler, M. and Regan, N. (2019). Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. New in Chess. ISBN-13: 978-9056918187.
  • Shannon, C. E. (1950). Programming a Computer for Playing Chess. The London, Edinburgh and Dublin Philosophical Magazine, 41(314), 256-275. doi: 10.1080/14786445008521796.
  • Silver, D. et al (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learn-ing Algorithm. arXiv: 1712.01815.
  • Silver, D. et al (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140-1144. doi: 10.1126/science.aar6404.
  • Tesauro (1995). Temporal Difference
Full article

To read the full article in pdf, click HERE

Categories: Ενημέρωση

Interview with Alexander Lyashuk about the recent success of Lc0

Wed, 02/06/2019 - 14:51

Lc0 qualified for the Superfinal of the TCEC Championship and also won the TCEC Cup

The neural network chess engine Lc0 (aka Leela Chess Zero) is making the headlines with its performance in the Top Chess Engine Championship. First it qualified to the Superfinal of the competition by finishing second in the Premier Division, and a few weeks after it conquered its first major computer chess title by winning the TCEC Cup.

Live now: Follow the Superfinal of TCEC Stockfish vs Lc0

Chessdom spoke to one of the head admins and leading developers of the project Alexander Lyashuk about the history of the engine, its road in competitive chess, development details, and the future of computer chess in general.

 

Congratulations! Lc0 has finally conquered its first major title by winning the TCEC Cup

Indeed, a first major title for Lc0. That does feel really good, although I have to admit that this time it did not happen without some luck. :) Stockfish was eliminated before Leela had a chance to meet it, and that obviously helped a lot.

Anyway, it’s a great achievement for the team, and I’d like to mention that it’s not just the developers who made that progress possible. The main driver of the effort is actually the community, which contributes their computing power to help Leela learn, the people who do lots of strength testing, and the ones who run youtube videos and twitch streams, or just keep the morale up by being active in chat and forum.

Also, a big thanks to TCEC for providing a platform to showcase our progress and get the community excited about the Lc0 project. It’s indisputable that without TCEC Leela would be much less popular.

Lc0 is a very young project. It has been active for about an year. Is this the most meteoric rise to the top that computer chess has witnessed

There are two major dates which can be thought as a project start. The first one is Gary’s [Gary Linscott] official project announcement and start of a GitHub repository. That happened around 2-7 January 2018. Others think, that the actual start of the Leela training pipeline should be counted as the real start. That happened somewhere in the beginning of March 2018.

No matter which date we take as a start of the project, it is indeed probably the fastest rise of a chess engine. Although one can argue whether it’s a fair comparison in terms of developers achievement. Traditionally, developing a chess engine needed lots of chess expertise and trial and error. The developer has to fight with his bare hands for every 3 Elo points. If a developer takes a vacation, Elo growth stops. With Leela we of course do development and infrastructure maintenance work too, but all in all we just wait and Leela becomes stronger. We can go on a vacation, return in 1 week — and Leela magically learnt something new by itself.

Also, I’d like to remind that major part of Leela’s ideas come from DeepMind’s paper. Only recently we started to experiment with neural network architecture ideas which differ from what was written there. But all along the way, AlphaZero was the guide of our project.

Ed. note: Read about the entrance of Leela into competitive chess

The first time Lc0 appeared in a competition was in S12 of TCEC. Back then there was no GPU, it was just running on CPU. But that was also a time when the fanbase !boom began. Looking back at this moment, what are your thoughts?

It is good that TCEC now has GPUs. :) But back then I didn’t really expect TCEC to to make that serious investment in GPUs. For S13 I tried to urgently write a Leela backend to be able to parallelize work to 47 CPU cores, but then it was announced that the competition will have GPUs, so I could relax.

Yes, community growth really exploded with Leela entering TCEC, and we actually always have had a problem (a good problem to have!) that development couldn’t keep up with the number of contributors. It’s still kind of like this: there are lots of people contributing, there are lots of good ideas, too little time to implement.

Now from the look of history, what do you think about the technical difficulties experienced along the way?

That GPU overheating was a good source of burrito memes and jokes about poor Leela’s thermal management. :) Technical difficulties are unavoidable when you try something new. Maybe Leela didn’t show its real strength there, but it allowed her to play more games in the next season 14 (all the way through Div3).

More importantly, it led to changes in Leela’s time management, so now it’s more robust when run on stable hardware.

Currently Test30 net is triumphing. Test 40 is in developing and testing phase. Do you expect it to surpass Test 30? If yes, when? And how much ELO will that mean for Lc0?

Actually I didn’t have time to follow test 40 for the last 2 weeks, and 2 weeks for Leela is an eternity! But from other devs I heard a secret plan of keeping Leela play only draws in TCEC Cup for some weeks in order to delay TCEC Superfinal and send test 40 there. :)

So there is a hope that test 40 will take the lead soon.

A new paper was released by A0 recently, was it helpful in Lc0′s success this year?

It was really helpful as it filled in the remaining parts of a puzzle that we had. (Our findings are summarized here).

For some missing parts, we felt that there is something to be fixed, but didn’t dare to contradict the words of DeepMind’s pre-print. For example, we had an observation that Leela hardly ever changes the best move after 1 million nodes or so. There were different ideas to encourage more exploration as search goes, but in the end, it turned out that so-called “puct constant” was not a constant.

The Superfinal is coming. Lc0 has decisive games against every opponent this season except Stockfish. Is the Superfinal a chance to do that?

Beating Stockfish in S14 is not the most probable result, but it’s surely possible. I’m sure there will be interesting wins and maybe a bit of a drama as usual. It will be the most entertaining TCEC superfinal in recent years, and I hope lots of people will enjoy it.

And what what do you expect from Lc0 in S15?

If you search through our Discord server chat history, I did a prediction early in May there that Leela will win S15. I still hold this prediction. :)

Well, as a side note, I expected S15 to start around October 2018, so to be fair, I was a bit overly optimistic regarding the growth of Leela’s skills… Or maybe it’s vice versa, Leela noticed that there’s still a long time until S15 and relaxed. :)

Do you think the computer chess world is moving towards an era where the NNs will dominate over the AB engines?

First let’s clarify a terminology. There is a search algorithm (AB vs MCTS), and there is evaluation function (handwritten vs NN-based). Leela uses MCTS+NN, Stockfish uses AB+handwritten, but any combination is possible. For example, Komodo MCTS uses MCTS+handwritten, and Scorpio NN uses (as far as I heard) AB+NN.

I’m sure, that purely NN-driven chess engines will take over the Top-5 pretty soon. Currently the main drawback of NN-based evaluation function is it’s slowness compared to handwritten ones, but evaluation of neural networks will speed up and improve.

Classical engines with fast handwritten eval functions also don’t stay at one place though. Stockfish keeps the growth momentum really well, for example. It’s also nice to see that Leela also helps with that growth, providing ideas for Stockfish patches.

There was another NN engine participating this season – Scorpio NN. What do you think of the project of Daniel Shawul?

DeepMind’s publication demonstrated a new area to explore in chess engine world and I’m happy to see that another engine also picked that idea to try. I will be really curious to see its progress and wish it all the best in gaining ELO.

But I would also like to see other NN-based engines appear! Because to fill Top-5 with NN engines, we need at least 5 NN engines!

Categories: Ενημέρωση

Stockfish wins the Premier Division, Lc0 qualifies for the Superfinal

Mon, 02/04/2019 - 15:00

Stockfish, winner of the Premier Division of TCEC S14

The dominance of Stockfish in the Top Chess Engine Championship continues full steam ahead. The open source engine by Marco Costalba, Joona Kiiski, Gary Linscott and a huge community of contributors, won the Premier Division of Season 14 convincingly, not losing a single game along the way. With the victory Stockfish qualifies for the Superfinal, where it will be looking for a 4th consecutive and 6th overall TCEC title.

Stockfish wins TCEC S11
Stockfish wins TCEC S12
Stockfish wins TCEC S13

For the first time in history Stockfish will be in a situation where the “big 3″ of computer chess in not existent anymore. Lc0 aka Leela Chess Zero – the Neural Network built as an open source adaptation of DeepMind’s recent Alpha Zero artificial intelligence demonstration project – has broken the dominance the “big 3″ and has taken second position in the Premier Division ahead of Komodo (3rd) and Houdini (4th).

Live now: TCEC Superfinal at the official website

This is going to be the first match of Stockfish and Lc0 on the highest level of computer chess. The two could not meet in the recently concluded the TCEC Cup, due to a shocking loss of Stockfish to Houdini. In a mid-division test it was demonstrated that Lc0 is highly superior to Stockfish 8, mimicking the conditions and results of the SF8 – Alpha Zero match. However, Stockfish 10 is going to play the Superfinal, and it is much superior to the Stockfish 8 version, which makes the task for Lc0 exponentially more difficult.

This sentiment is shared among the computer chess fans. Out of the over 1000 (and counting) votes in the poll, 65% expect a Stockfish win in the Superfinal, while 35% expect Lc0 to be victorious. However, both Stockfish and Lc0 are going to line-up new versions compared to the TCEC Cup and anything can happen.

About the TCEC Superfinal

The Superfinal is a match between the winner of the Premier Division – Stockfish – and the second placed in the Premier Division – Lc0. It is a 100 games match, which is played with 50 different openings so that each engine plays both black and white of the same position. The match will be presented with opening 1 used in games 1 and 2, then opening 2 used in games 3 and 4 etc. If the match is theoretically won for one side before game 100, the match will still continue until all 100 games have been played. In the case of a drawn match there will be a rapid match of 16 games with a time control of 25′ + 10″ with random openings selected from earlier in the same Season. In case it is still tied there will be a Blitz match of 8 games with a time control of 3′ + 2″. When the Superfinal is over, the current Season ends. The winner is crowned the Grand Champion of TCEC.

Categories: Ενημέρωση

First major title for a Neural Network in chess: LC0 wins TCEC Cup

Mon, 02/04/2019 - 14:59

Neural Network chess fans around the world are celebrating as the first major computer chess title for the self-learning engine Lc0 (Leela Chess Zero) is a fact. Lc0 conquered the golden medal of the TCEC Cup in a 32 player knockout against the best engines in the world. The only NN playing in the competition emerged victorious after knocking out consecutively Tucano (8-0), Xiphos (6-2), Andscacs (5,5-2,5), the three time TCEC champion Komodo at the semi-final (10-8), and the three time TCEC champion Houdini at the final (4,5-3,5). Thus, Lc0 received the TCEC Cup gold medal, ahead of Houdini and the defending champion Stockfish.

Replay: Lc0 – Houdini , the decisive game

It seems that a knockout format with 30 min time control is the favorite of the world’s top NN engine. Lc0 won its first major medal in the same competition last year, while just three months later it is able to win the gold medal, going all the way undefeated.

Final standings (click on the image for full view)

Click on the image for full size view

Houdini – Stockfish (semi-final) by Kingscrusher

The match that forced Lc0 to play direct matches against all champions in TCEC history, analysed by Kingscrusher

First “end of an era” match?

Ever since Lc0 entered the Top Chess Engine Championship back in S12, the Neural Network believers were excited that an “end of an era” is coming. This expression originates in the fact that Stockfish has been dominating the computer chess world as the strongest AB chess engine, and that Leela as a Neural Network has the highest chance of changing the status quo.

Lc0 did not have a direct match against Stockfish in the TCEC Cup this year. However, it defeated in the final Houdini – a version that has won the TCEC Championship and that eliminated Stockfish at the semi-final. At its own semi-final Lc0 defeated another multiple times TCEC champion – Komodo – thus confirming that the second place in the Premier Division was a deserved one. This second place will finally give the chance to Lc0 to face Stockfish for the highest title of computer chess. It will happen at a 100 games long time control head-to-head match, the Superfinal of TCEC, that starts this February 4th at 15:00 CET.

The Superfinal is a chance for Lc0 to conquer yet another pinnacle. The TCEC Cup is the first time this TCEC season that LCZero did not lose a game – it did lose games in the lower Div3, Div2, and Div1. However, Lc0 has a win against every engine that it met this season. Except one – Stockfish.

Is the Superfinal going to be the “end of an era” or just another episode of a long lasting battle to come? Follow Stockfish – Lc0 Superfinal match live at the official website.

Categories: Ενημέρωση