Playing Strength

Home * Chess * Playing Strength

Dragon and Snake on ambos ¹

Photo by Gerd Isenberg, September 18, 2016, detail of the Flottmann gate, Art Nouveau theme of Dragon and Sun designed by Carl Weinhold, art director of blacksmith and foundry Füssmann und Fleeth, Essen, exposed at the industrial and trade exhibition 1902 in Düsseldorf, and baught by Heinrich Flottmann as gate for his jackhammer factory, today adjacent to the exhibition and event space Flottmann-Hallen in Herne, North Rhine-Westphalia, Germany, and part of The Industrial Heritage Trail of the Ruhr area, "The dragon is a symbol of physical strength and intelligence with respect to the snake that symbolizes the tough, glowing wrought iron" from Flottmann-Tor – Hün un Perdün, see also Image by Gerd Biedermann↩︎

Playing Strength, (Performance, Skill Level)
of a chess player, or chess playing entity, program or engine, reflects the ability to win against other players, given by a number or other element of an ordered set such as an Elo number.

The ability to solve test-positions, that is, finding the specified, likely one and only best move, might be an indicator for various particular engine skills, but does not necessarily correlate with playing strength. In his Parallelism and Selectivity in Game Tree Search lecture, Tord Romstad introduced the Worst Moves Observation (WMO), which states the practical playing strength is not primarily determined by the quality of the players best moves nor average moves, but by the quality of the players worst moves.

Contents

Measuring

A statistical valid method to measure playing strength within a defined confidence interval is to play an appropriate huge number of games with both sides versus a wide range of different opponents ¹ with symmetric time constraints, and to apply match statistics. Performance isn't measured absolutely; it is inferred from wins, losses, and draws against other players or engines. Players' rating depend on the ratings of their opponents, and the results scored against them ². While relative playing strength of chess engines is not strictly transmissive over various time controls, the number of games played is more relevant than their duration, the todays de facto standard in measuring playing strength is parallel playing fast chess with (ultra) short time control, such as blitz, bullet or even lightning chess, as for instance used in the Fishtest framework of Stockfish ³.

Strength

The strength of a chess program depends on many things, the quality and efficiency of the algorithms involved to determine the best move of a position, the balance of the so called search versus knowledge tradeoff to evaluate or compare leaf nodes of a search tree, how to shape that tree and to propagate a score up to the root, and time management, that is how to allocate time for searching a move under time control requirements. Time used is roughly proportional to the number of visited nodes of the common depth-first search inside an iterative deepening frame, which grows exponentially by its effective branching factor raised to the power of search depth. Playing strength might be improved over the (playing) time due to learning algorithms.

Computer Analysis of Human Players

⁴

Publications

1970 ...

F. Donald Bloss (1973). Rate your own Chess. Van Nostrand Reinhold Inc.
Tony Marsland, Paul Rushton (1973). Mechanisms for Comparing Chess Programs. ACM Annual Conference, pdf
Peter W. Frey (ed.) (1977). Chess Skill in Man and Machine. Springer, New York, N.Y. 2nd ed. 1983
James Gillogly (1978). Performance Analysis of the Technology Chess Program. Ph.D. Thesis. Tech. Report CMU-CS-78-189, Carnegie Mellon University, CMU-CS-77 pdf » Tech
Arpad Elo (1978). The Rating of Chessplayers, Past and Present. Arco Publications ⁵
Don Beal (1978). Handicapping Computer Chess Programs. ICCA Newsletter, Vol. 1, No. 2
David Cahlander (1979). Strength of a Chess Playing Computer. ICCA Newsletter, Vol. 2, No. 1
Jack Good (1979). On the Grading of Chess Players. Personal Computing, Vol. 3, No. 3, pp. 47
Gary L. Ratliff (1979). Practical Rating Program. Personal Computing, Vol. 3, No. 9, pp. 62 » Bloss
Frieder Schwenkel (1979). Berating the ratings system. Personal Computing, Vol. 3, No. 11, pp. 77 » Ratliff, Bloss
John Shaposka (1979). "JS" Takes the Bloss Test. Personal Computing, Vol. 3, No. 12, pp. 75 » Ratliff, Bloss
Editor (1979). Chess Ratings from Problem Solving. Personal Computing, Vol. 3, No. 12, pp. 77 » CAPS II, Chess Problems, Compositions and Studies

1980 ...

Floyd R. Kirk (1980). Bloss Flunks Test. Personal Computing, Vol. 4, No. 8, pp. 72 » Ratliff, Bloss
David E. Welsh (1980). Rating Boris 2.5. Personal Computing, Vol. 4, No. 10, pp. 78 » Boris 2.5
W. W. Foster (1980). Six Test Problems for Chess Playing Machines. Personal Computing, Vol. 4, No. 10, pp. 81
David E. Welsh (1980). Rating Boris 2.5 - Part II. Personal Computing, Vol. 4, No. 11, pp. 87
Ken Thompson (1982). Computer Chess Strength. Advances in Computer Chess 3
Ivan Bratko, Danny Kopec (1982). A Test for Comparison of Human and Computer Performance in Chess. Advances in Computer Chess 3
Danny Kopec, Ivan Bratko (1982). The Bratko-Kopec experiment: a comparison of human and computer performance in chess. Advances in Computer Chess 3
Danny Kopec, Enrique Irazoqui, Ivan Bratko (1983). The Updated Bratko-Kopec Test. Computer Chess Digest Annual pp. 45-63
Alexander Szabo (1984). Computer-Chess Tactics and Strategy. M.Sc. Thesis, University of British Columbia, pdf
Göran Grottling (1984). The Truth about their Strength. ICCA Journal, Vol. 7, No. 4
Göran Grottling (1985). How to Bracket the Truth in Playing Strength. ICCA Journal, Vol. 8, No. 1
Monroe Newborn (1985). A Hypothesis Concerning the Strength of Chess Programs. ICCA Journal, Vol. 8, No. 4
Peter W. Frey (1986). Algorithmic Strategies for Improving the Performance of Game-Playing Programs. In J. Doyne Farmer, Alan Lapedes, Norman Packard, Burton Wendroff (Ed.) (1986). Evolution, Games and Learning: Models for Adaptation in Machines and Nature. Proceedings of the Fifth Annual International Conference of the Center, Elsevier
Alexander Szabo, Barbara Szabo (1988). The Technology Curve Revisited. ICCA Journal, Vol. 11, No. 1
Hans Berliner, Gordon Goetsch, Murray Campbell, Carl Ebeling (1989). Measuring the Performance Potential of Chess Programs. Advances in Computer Chess 5
Eric Hallsworth (1989). Playing Levels. Computer Chess News Sheet 23, pp 2, pdf hosted by Mike Watters

1990 ...

Hans Berliner, Gordon Goetsch, Murray Campbell, Carl Ebeling (1990). Measuring the Performance Potential of Chess Programs. Artificial Intelligence, Vol. 43, No. 1
Eric Hallsworth (1990). Speed, Processors and Ratings. Computer Chess News Sheet 25, pp 6, pdf hosted by Mike Watters
Hans Berliner, Danny Kopec, Ed Northam (1991). A taxonomy of concepts for evaluating chess strength: examples from two difficult categories. Advances in Computer Chess 6, pdf
Steve Maughan (1992). Are You Sure It's Better? Selective Search 40, pp. 21, pdf hosted by Mike Watters
Warren D. Smith (1993). Rating Systems for Gameplayers, and Learning. ps
Luis Antunes, Luis Moniz, Carlos Azevedo (1993). Rb+: the Dynamic Estimation of the Opponent's Strength. INESC-ID, Lisbon, Portugal » RB
Mark E. Glickman (1993). Paired Comparison Models with Time Varying Parameters. Ph.D. thesis, Harvard University, advisor Hal Stern, pdf
Mark E. Glickman (1995). A Comprehensive Guide To Chess Ratings. pdf
Mark E. Glickman, Christopher Chabris (1996). Using Chess Ratings as Data in Psychological Research. pdf
Robert Hyatt, Monroe Newborn (1997). CRAFTY Goes Deep. ICCA Journal, Vol. 20, No. 2 » Crafty
Mark E. Glickman, Albyn C. Jones (1999). Rating the Chess Rating System. pdf