Francis HEYLIGHEN
PO, Free University of Brussels, Pleinlaan 2, B-1050 Brussels, Belgium
Phone: +32-2-629 25 25, Fax: +32-2-629 24 89
E-mail: fheyligh@vnet3.vub.ac.be
ABSTRACT. A list of the most relevant publications on complex, evolving systems is produced by counting the number of times each publication is cited in a collection of texts on the domain. The importance of these books and papers is summarized and put into its historical context by noting the main contribution to the field of their authors, categorized by the research tradition they originated from. These include biology, physics, chemistry, mathematics, cybernetics, systems theory, economy and complex adaptive systems.
Keywords: complexity, evolution, systems theory, classic publications, authors.
Number of pages: 18
Figures and tables: none
It would be very useful to provide newcomers with a selection of the most important publications. The traditional way to determine what is most important in a scientific discipline is by counting the number of citations: the more often a particular paper or author is referred to by others, the more likely it is that the corresponding work is both relevant to the field and of high quality. In the case of complex, evolving systems, the problem is where to start counting. There are not as yet any long standing journals exclusively devoted to the field, from which we can gather a sufficient number of citations. Including citations from related domains (say, thermodynamics, neural networks or evolutionary biology) is likely to produce a bias towards work in these more established disciplines, where there are many more publications and therefore many more citations, than in the young discipline we are trying to circumscribe.
Luckily, I had a good opportunity to collect relevant material when I organized a symposium on the subject. This symposium, devoted to "The Evolution of Complexity", took place at the Free University of Brussels in 1995, as part of the large interdisciplinary congress "Einstein meets Magritte". The Call for Papers emphasized the role that the newly developed insights around evolution, complexity and systems could play in bridging disciplines and providing a unified picture of the universe, from atoms and molecules to cells, organisms and societies. Over 40 proposals, including an abstract and a list of basic references for the approach, were submitted by authors from diverse scientific and cultural backgrounds. I combined these lists of references, with the more detailed references in the about 20 papers that were eventually accepted for the Proceedings [1]. I finally added a number of bibliographies from books and papers which I personally selected as most relevant to the domain. Adding all references together produced a file with over 1500 citations.
To process the file, I first eliminated all authors cited by only one author, in order to diminish personal biases (such as self-citation). I then counted how many times each of the remaining authors and publications were included. Some researchers manage to condense most of their contributions into a single book, their "magnum opus". Such a book is a good candidate to become a citation classic. Examples include Kauffman's "The Origins of Order" (1993), which with 9 citations is the most cited work in this survey, and Holland's "Adaptation in Natural and Artificial Systems" (1992). Other authors, such as Brian Arthur or Heinz von Foerster, produce dozens of different, but equally interesting papers, which only get one or two citations each. This diminishes the importance accorded to any individual publication, but boosts the total number of citations an author gets, since the citing author will generally include several smaller publications. Therefore, I decided on a formula to compute overall importance which adds the total number of times a work is cited with a quarter of the number of times other works of the same author are cited.
This produced a rough ordering of all works according to their importance for the field. The highest scoring publications are included in the bibliography below, with the number of asterisks denoting their relative score. For the borderline cases, I used my own judgment to select those which seemed most relevant. This general procedure is biased towards older, "classic" works, since very recent publications have not yet had the time to be read and cited by many authors. I have tried to compensate for that by preferring the newest editions of classic books, by adding a few recent reviews by established authors (e.g. Holland, 1996; Casti, 1994), and by selecting collections that contain both older classic papers and more recent work of the same author (e.g. Arthur, 1994; Wolfram, 1996). The formulation of the Call for Papers and my personal selection also privilege more general and more introductory publications, as opposed to specialised, technical work. Where possible, I have included both the original technical references (e.g. Bak et al., 1988; Kauffman, 1993), and later reviews addressed to a wider audience, such as articles in "Scientific American" (e.g. Bak and Chen, 1991; Kauffman, 1995).
Together, I believe, this produces excellent material for students and people entering the field, as well as for active researchers interested in broadening their horizon or exploring the historical roots of their domain. This is especially important since many researchers working on these recently fashionable topics are unaware that subjects like complexity, self-organization, networks and adaptive systems have already been extensively studied in the 1940's and 1950's, by researchers like Wiener, Ashby, von Neumann and von Foerster, and in discussion forums like the famous Josiah Macy meetings on cybernetics [2]. Some recent popularizing books on "the sciences of complexity" (e.g. Waldrop, 1992) seem to ignore this fact, creating the false impression that work on complex adaptive systems only started in earnest with the creation of the Santa Fe Institute in the 1980's.
The following sections will attempt to situate the most important authors in the historical development of their domain, and indicate their main contributions to the field. Thus I hope to sketch a panorama of the most important concepts and approaches. I have grouped them in three categories, denoting the broad tradition from which they originate: the natural sciences (including biology, chemistry, physics and mathematics), cybernetics and general systems theory, and complex adaptive systems.
More modern and eminently readable overviews of evolutionary theory in biology can be found in the books of Richard Dawkins, who emphasizes the role of replicators, including genes and memes (units of culture), and the importance of game-theoretic models of co-evolution. Often in debate with Dawkins, the paleontologist Stephen Jay Gould is another well-known popularizer of evolutionary thought. Together with Niles Eldredge, he has proposed the theory of punctuated equilibrium, according to which evolution is a largely chaotic, unpredictable process, characterized by long periods of stasis interspersed by sudden bursts of change.
Another important strand of work leading to the analysis of complex evolution is thermodynamics. Ilya Prigogine received the Nobel prize for his work, in collaboration with other members of the "Brussels School", showing that physical and chemical systems far from thermodynamical equilibrium tend to self-organize by exporting entropy and thus to form dissipative structures. Both his philosophical musings (Prigogine & Stengers, 1984) about the new world view implied by self-organization and irreversible change, and his scientific work (Nicolis & Prigogine, 1977, 1989; Prigogine, 1980) on bifurcations and order through fluctuations remain classics, cited in the most diverse contexts. Inspired by Prigogine's theories, Erich Jantsch has made an ambitious attempt to synthesize everything that was known at the time (1979) about self-organizing processes, from the Big Bang to the evolution of society, into an encompassing world view.
The physicist Hermann Haken (1978) has suggested the label of synergetics for the field that studies the collective patterns emerging from many interacting components, as they are found in chemical reactions, crystal formations or lasers. Another Nobel laureate, Manfred Eigen (1992), has focused on the origin of life, the domain where chemical self-organization and biological evolution meet. He has introduced the concepts of hypercycle, an autocatalytic cycle of chemical reactions containing other cycles, and of quasispecies, the fuzzy distribution of genotypes characterizing a population of quickly mutating organisms or molecules (1979).
The modelling of non-linear systems in physics has led to the concept of chaos, a deterministic process characterized by extreme sensitivity to its initial conditions (Crutchfield, Farmer, Packard & Shaw, 1986). Although chaotic dynamics is not strictly a form of evolution, it is an important aspect of the behavior of complex systems. The science journalist James Gleick has written a popular history of, and introduction to, the field. Cellular automata, mathematical models of distributed dynamical processes characterized by a discrete space and time, have been widely used to study phenomena such as chaos, attractors and the analogy between dynamics and computation through computer simulation. Stephen Wolfram has made a fundamental classification of their types of behavior. Catastrophe theory proposes a mathematical classification of the critical behavior of continuous mappings. It was developed by René Thom (1975) in order to model the (continuous) development of (discontinuous) forms in organisms, thus extending the much older work by the biologist D' Arcy Thompson (1917).
Another French mathematician, Benoit Mandelbrot (1983), has founded the field of fractal geometry, which models the recurrence of similar patterns at different scales which characterizes most natural systems. Such self-similar structures exhibit power laws, like the famous Zipf's law governing the frequency of words. By studying processes such as avalanches and earthquakes, Per Bak (1988, 1991) has shown that many complex systems will spontaneously evolve to the critical edge between order (stability) and chaos, where the size of disturbances obeys a power law, large disturbances being less frequent than small ones. This phenomenon, which he called self-organized criticality, may also provide an explanation for the punctuated equilibrium dynamics seen in biological evolution.
The seminal books by the respective founders of these two fields, Norbert Wiener and Ludwig von Bertalanffy, are still widely cited as introductory overviews. As collections of papers that first appeared during the 1930's and 1940's they are perhaps not very coherent. W. Ross Ashby's "Introduction to Cybernetics", though difficult to find, provides a very clear and more systematic development of the main concepts and principles. Ashby's insights into adaptive systems (Ashby, 1960) are surprisingly modern, and his main contributions, such as the law of requisite variety, the principle that every dynamic system will self-organize, and the requirement that every regulator of a system must also be a model of a that system, still need to be assimilated by many researchers in complex systems. In a spirit similar to Ashby, Arvid Aulin has proposed a law of requisite hierarchy governing control systems and societies. Another cybernetician, the anthropologist Gregory Bateson, is famous for his stimulating and insightful essays on the parallels between mind and natural evolution.
Heinz von Foerster is one of the most cited cyberneticians and founding fathers of the domain. His main contribution is the "second-order" cybernetics, or the cybernetics of observing systems (von Foerster, 1981, 1996). More directly linked to the study of complex evolution, perhaps, are his classic 1960 paper on self-organization where he formulated the order from noise principle, and the 1962 book he edited on the same subject. His emphasis on circular, self-referential processes has been elaborated in Humberto Maturana and Francisco Varela's work on autopoietic systems. Autopoiesis (self-production) denotes the fact that organisms produce their own components. In that sense they are autonomous or "organizationally closed": for them the environment is merely a source of perturbations that need to be compensated in order to maintain the system's organization (Varela, 1979; Zeleny, 1981). Such a view has profound implications for the nature of knowledge as characterizing all living organisms (Maturana & Varela, 1992).
The theme of circularity also returns in the analysis of causal networks that contain cycles, a common characteristic of complex systems. Magoroh Maruyama has written a classic paper emphasizing self-reinforcing cycles. Jay Forrester has founded a whole new modelling approach, systems dynamics, which is based on the analysis of the different positive and negative feedback cycles in networks of many interacting variables. This approach is at the base of the famous world model (Forrester, 1973) presented in a report to the Club of Rome. One of the founding fathers of general systems theory, the economist Kenneth Boulding, has proposed a related, flow-based model of the evolution of society, integrating ecology and economics.
Though they do not strictly belong to the cybernetics and systems movement, the following authors have strongly influenced--and been influenced by--that tradition. Claude Shannon is the creator of the mathematical theory of information, which is implicitly or explicitly used in most models of complexity. Though relatively little known, the mathematician John von Neumann is perhaps the greatest scientific genius of the 20th century. He has made essential contributions to the most diverse domains: ergodic theory, lattice theory, the foundations of quantum mechanics, the theory of games, the development of the atomic bomb, the architecture of digital computers, etc. Perhaps most directly relevant to the study of complexity is his investigation of self-reproducing automata (for which he created the cellular automata models mentioned earlier). Nobel laureate Herbert A. Simon is another universal mind, with foundational contributions ranging from economics, psychology, management, and philosophy to artificial intelligence, a domain he helped to create. He has studied the problem solving techniques that adaptive systems (people, organizations, computers, ...) use to cope with complexity. "The Sciences of the Artificial" provides a good introduction to his ideas, including his classic essay on "The Architecture of Complexity", where he proposes an evolutionary explanation for hierarchical organization. The great methodologist of the social sciences, Donald T. Campbell, has made an in-depth analysis of the blind-variation-and-selective-retention mechanisms that underly the evolution of knowledge (thus founding the domain of evolutionary epistemology), society, and hierarchical systems in general (for which he introduced the concept of downward causation).
Two popular science books, one by the science writer Mitchell Waldrop and one by the Nobel laureate and co-founder of the Santa Fe Institute Murray Gell-Mann, offer good reviews of the main ideas underlying the CAS approach. Another Santa Fe collaborator, the systems analyst John Casti, has written several popular science books, discussing different issues in the modelling of complex systems, while integrating insights from the CAS approach with the two older traditions.
John Holland is the founder of the domain of genetic algorithms. These are parallel, computational representations of the processes of variation, recombination and selection on the basis of fitness that underly most processes of evolution and adaptation (Holland, 1992). They have been successfully applied to general problem solving, control and optimization tasks, inductive learning (classifier systems, Holland et al., 1986), and the modelling of ecological systems (the ECHO model, Holland, 1996). The biologist Stuart Kauffman has tried to understand how networks of mutually activating or inhibiting genes can give rise to the differentiation of organs and tissues during embryological development. This led him to investigate the properties of Boolean networks of different sizes and degrees of connectedness. Through a reasoning reminiscent of Ashby, he proposes that the self-organization exhibited by such networks of genes or chemical reactions is an essential factor in evolution, complementary to Darwinian selection by the environment.
Holland's and Kauffman's work, together with Dawkins' simulations of evolution and Varela's models of autopoietic systems, provide essential inspiration for the new discipline of artificial life, This approach, initiated by Chris Langton (1989, 1992), tries to develop technological systems (computer programs and autonomous robots) that exhibit lifelike properties, such as reproduction, sexuality, swarming, and co-evolution. Tom Ray's Tierra program proposes perhaps the best example of a complex, evolving ecosystem, with different species of "predators", "parasites" and "prey", that exists only in a computer.
Backed by Kauffman's work on co-evolution, Wolfram's cellular automata studies, and Bak's investigations of self-organized criticality, Langton (1990) has proposed the general thesis that complex systems emerge and maintain on the edge of chaos, the narrow domain between frozen constancy and chaotic turbulence. The "edge of chaos" idea is another step towards an elusive general definition of complexity. Another widely cited attempt at a definition in computational terms was proposed by Charles Bennett.
Another investigation which has strongly influenced the artificial life community is Robert Axelrod's game theoretic simulation of the evolution of cooperation. By letting different strategies compete in a repeated Prisoner's Dilemma game, Axelrod (1984) showed that mutually cooperating, "tit-for-tat"-like strategies tend to dominate purely selfish ones in the long run. This transition from biological evolution to social exchanges naturally leads into the modelling of economic processes (Anderson, Arrow & Pines, 1988). W. Brian Arthur has systematically investigated self-reinforcing processes in the economy, where the traditional law of decreasing returns is replaced by a law of increasing returns, leading to the path-dependence and lock-in of contingent developments. More recently (1994), he has simulated the seemingly chaotic behavior of stock exchange-like systems by programming agents that are continuously trying to guess the future behavior of the system to which they belong, and use these predictions as basis for their actions. The conclusion is that the different predictive strategies cancel each other out, so that the long term behavior of the system becomes intrinsically unpredictable. This result leads back to von Foerster's second-order cybernetics, according to which models of social systems change the very systems they intend to model.
Anderson P. W., K. J. Arrow, and D. Pines (Eds.). The Economy as an Evolving Complex System, Addison-Wesley, Redwood City CA, 1988. **
Arthur, W. B.: Competing Technologies, Increasing Returns, and Lock-in by Historical Events, The Economic Journal 99: 1989, pp. 106-131. *
Arthur, W. B.: Positive Feedbacks in the Economy, Scientific American, February 1990, pp. 92-99. *
Arthur W. B. Increasing Returns and Path Dependence in the Economy, University of Michigan Press, Ann Arbor, 1994.
Arthur W. B.: Bounded Rationality and Inductive Behavior (the El Farol Problem), American Economic Review 84, pp. 406-411, 1994.
Ashby W. R. An Introduction to Cybernetics, Methuen, London, 1964. **
Ashby W. R. Mechanisms of Intelligence: Writings of Ross Ashby, Intersystems, Salinas CA, 1981.
Ashby, W. R. Design for a Brain - The Origin of Adaptive Behaviour. Chapman and Hall, London, 1960.
Aulin A. The Cybernetic Laws of Social Progress, Pergamon, Oxford, 1982 *
Axelrod R. M. The Evolution of Cooperation, Basic Books, New York, 1984. *
Bak P. and Chen K.: Self-Organized Criticality, Scientific American: January 1991, pp. 46-53.
Bak P., Tang C., & Weisenfeld K.: Self-Organized Criticality. Physical Review A 38: 1988, pp. 364-374. *
Bennett C. H. Dissipation, Information, Computational Complexity and the Definition of Organization. Emerging Syntheses in Science, Pines D. (ed.), Addison-Wesley, Redwood City CA, 1985, pp. 215-233. *
Boulding K. E. Ecodynamics: a new theory of societal evolution. Sage, London, 1978.
Campbell, D. T. Evolutionary epistemology. Evolutionary epistemology, rationality, and the sociology of knowledge, G. Radnitzky and W. W. Bartley (eds.), Open Court, La Salle IL, 1987, pp. 47-89.
Campbell, D. T. "Downward Causation" in Hierarchically Organized Biological Systems. Studies in the Philosophy of Biology, F.J. Ayala and T. Dobzhansky (eds), Macmillan, New York, 1974 .
Casti J.L. Complexification: explaining a paradoxical world through the science of surprise, HarperCollins, 1994.
Crutchfield, J., Farmer, J.D., Packard, N., and Shaw, R.: Chaos, Scientific American, 255 (6): December 1986, pp. 46-57.
Darwin C. The origin of species by means of natural selection or the preservation of favoured races in the struggle for life. (Edited with and introduction by J W Burrow). Penguin classics, 1985. (First published by John Murray, 1859) *
Dawkins R. The selfish gene (2nd edition), Oxford University Press, Oxford, 1989. **
Dawkins R. The Extended Phenotype: The Gene as a Unit of Selection, Oxford University Press, Oxford, 1983. *
Dawkins R. The Blind Watchmaker, Longman, London, 1986. *
Eigen M. and P. Schuster. The Hypercycle: A principle of natural self- organization, Springer, Berlin, 1979 **
Eigen M., and R. Winkler-Oswatitsch. Steps Towards Life: A Perspective on Evolution. Oxford University Press, New York, 1992. *
Fisher R. A. The Genetical Theory of Natural Selection, 2nd edition, Dover Publications, New York, 1958.
Forrester, J. Industrial Dynamics, MIT Press, Cambridge, MA, 1961.
Forrester, J. W. World Dynamics (2nd ed.), Wright-Allen Press, Cambridge, MA, 1973.
Gell-Mann, M., The Quark and the Jaguar: Adventures in the Simple and the Complex, W.H. Freeman, San Francisco, 1994. *
Gleick, J. 1987. Chaos: Making a New Science, Penguin Books, New York. *
Gould S.J., and N. Eldredge. 1977: Punctuated equilibria: the tempo and mode of evolution reconsidered. Paleobiology 3, pp. 115-151.
Haken H. Synergetics, Springer, Berlin, 1978.
Holland J. H. 1992. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence, MIT Press, Cambridge MA. ***
Holland J.H. Hidden Order: How adaptation builds complexity, Addison-Wesley 1996.
Holland J. H., Holyoak K. J., Nisbett R. E. & Thagard P. R. 1986 Induction: processes of inference, learning and discovery, MIT Press, Cambridge MA. *
Jantsch, E., The Self-Organizing Universe: Scientific and Human Implications of the Emerging Paradigm of Evolution, Oxford, Pergamon Press, 1979.*
Kauffman S. A.: Antichaos and Adaptation, Scientific American: August 1991, pp. 78-84 **
Kauffman S. A. The Origins of Order: Self-Organization and Selection in Evolution Oxford University Press, New York, 1993 ****
Kauffman S. A. At Home in the Universe: The Search for Laws of Self-Organization and Complexity, Oxford University Press, Oxford, 1995.
Langton C. G.: Computation at the Edge of Chaos: phase transitions and emergent computation, Physica D, 42, 1-3, pp. 12-37, 1990. *
Langton, C. G. (Ed.). Artificial Life: The Proceedings of an Interdisciplinary Workshop on the Synthesis and Simulation of Living Systems, Addison-Wesley, Redwood City CA, 1989. **
Langton, C. G., Taylor, C., Farmer, J.D., and Rasmussen, S. (Eds.). Artificial Life II: Proceedings of the Second Artificial Life Workshop, Addison-Wesley, Redwood City CA, 1992. *
Mandelbrot B. B. The Fractal Geometry of Nature, Freeman, New York, 1983.
Maruyama M.: The Second Cybernetics: Deviation-Amplifying Mutual Causal Processes, American Scientist 51, No. 2: 1963, pp. 164-179.
Maturana H. R., & Varela F. J. The Tree of Knowledge: The Biological Roots of Understanding, (rev. ed.), Shambhala, Boston, 1992. ***
Monod, J. Chance and Necessity, Collins, London, 1972.
Nicolis, G, and Prigogine, I. Self-Organization in Non-Equilibrium Systems, Wiley, New York, 1977. **
Nicolis, G. and I. Prigogine. Exploring Complexity, Freeman, New York, 1989.
Prigogine, I. and Stengers, I. Order out of Chaos, Bantam Books, New York, 1984 ***
Prigogine, I. From Being to Becoming: Time and complexity in the physical sciences, Freeman, San Francisco, 1980.
Ray, T. S. An Approach to the Synthesis of Life. Artificial Life II, C. G. Langton et al. (Eds.), Addison-Wesley, Redwood City CA, 1992, pp. 371-408.
Shannon, C. E., and W. Weaver. The Mathematical Theory of Communication (5th ed.). University of Illinois Press, Chicago, 1963.
Simon, H. A. The Sciences of the Artificial (2nd. edition) MIT Press, Cambridge MA, 1981. **
Thom, R. Structural Stability and Morphogenesis, Benjamin, Reading MA, 1975.
Thompson, D. On Growth and Form, Cambridge University Press, Cambridge, 1917.
Varela, F., Principles of Biological Autonomy, North Holland, New York, 1979.*
von Bertalanffy L. General System Theory (Revised Edition), George Braziller, New York, 1973. *
von Foerster H. On self-organising systems and their environments. Self-Organising Systems, M.C. Yovits and S. Cameron (Eds.), Pergamon Press, London, 1960, pp. 30-50. *
von Foerster H. and Zopf, G. (Eds.) Principles of Self-Organization, Pergamon, New York, 1962. *
von Foerster H. Observing Systems: Selected papers of Heinz von Foerster. Intersystems, Seaside, CA, 1981.
von Foerster H. Cybernetics of Cybernetics (2nd edition). Future Systems, Minneapolis, 1996. **
von Neumann J. Theory of Self-Reproducing Automata. (Ed. by A. W. Burks), Univ. of Illinois Press, Champaign, 1966. *
Waldrop M. M. Complexity: The Emerging Science at the Edge of Order and Chaos, Simon & Schuster, New York, 1992. **
Wiener N. Cybernetics: Or Control and Communication in the Animal and the Machine, M.I.T. Press, New York, 1961. **
Wolfram S. Cellular Automata and Complexity: Collected Papers, Addison-Wesley, Reading MA, 1994.
Zeleny M. (Ed.) 1981, Autopoiesis: A Theory of Living Organization, North Holland, New York.
2. S. Heims. The Cybernetics Group. MIT Press, Cambridge MA, 1991.