[Science] The beginning of biological game theory.

Note: This is the first of what I hope will be a long series of posts on peer-reviewed research. Most of the blogging of this kind that I’ve seen has focused on new research hot off the presses. I love that sort of interpretation, but I think that the past is important too – despite academia’s focus on the newest and greatest data, there are some really important papers that deserve their own interpretation for the lay public who are interested in science. So, to start, I’m going to dig into the archives and tap a few of the big papers in my own little corner of the world.

And I’ll start with a favourite of mine, which launched an entire school of thought that touched many corners of biology:

J. Maynard Smith and G. R. Price. The logic of animal conflict. Nature, 246(1):1518, 1973. DOI: 10.1038/246015a0

As a trivial introduction, a few words on game theory: game theory is the study of optimal behaviour in strategic situations, where the optimal choice that one individual makes depends on the choices of the other individuals involved. A brief, concrete example may help. Imagine a foraging situation in which a solitary bird chooses amongst a field of potential food patches. This situation is not best analyzed using game theory, because the bird’s choice is simple: select the patch which maximizes his food intake. However, if we now add in the rest of the flock, the situation changes and game theory is the best way we have of analyzing the new problem. At this point, our focal bird’s choice must necessarily depend not only on the patch that they would prefer, but on the choices of all the other birds simultaneously. If every bird in the flock chooses the same patch, it may now be better for our focal bird to switch to another, less rich patch, because even though their preferred patch is richer they may not actually gain anything from competing to exploit it. (This is a simple example and an even simpler definition, of course, and I refer reader interested in the mechanics of game theory to the references below). An important term to learn is frequency dependence / independent. In the foraging situation above, the bird foraging alone is experiencing frequency independent payoffs, because his payoffs depend only upon his own actions. When the rest of the flock shows up and begins to forage as well, the payoffs become frequency dependent because now the payoff to a strategy depends on the strategy adopted by every other member of the flock present. Strategies which are increasingly common (more frequent) will experience a change in payoff, negative or positive as per the specific payoffs.

Game theory as a field began to acquire a shape and form when Oskar Morgenstern and John von Neumann published a book called Theory of Games and Economic Behaviour (1944), and the applications to economics and other fields like political science were immediate and highly influential. A flood of research followed, with scholars like John Nash, Anatol Rapoport, and Reinhard Selten leading a charge into the new and complex formal mathematics that underlies game theory. However, penetration into my field, biology, was limited until the 1970s, when a small paper in the journal Nature appeared, with John Maynard Smith and George R. Price as the authors (Maynard Smith & Price, 1973) .

The paper was a model of simplicity and clarity. It posed a central question: why aren’t more animal fights fatal? This was a question of some concern, because animals have large repertoires of nasty weapons with which to pursue violent conflicts, but as Maynard Smith and Price pointed out, animals rarely fight to the death. At the time, one of the more popular explanations was a group selection argument, which stated that animals kept fatalities to a minimum to benefit the group or the species. But this didn’t cut it any more, because at the time, group selection in biology was undergoing a violent death of its own. So, for the subject of this paper, the question remained: absent the feel-good explanation afforded by group selection, how could this behaviour be explained? Could it be adaptive on an individual level?

To answer this question, Maynard Smith and Price put forth a novel concept: the Evolutionarily Stable Strategy (ESS). As the authors put it:

Roughly, an ESS is a strategy such that, if most of the members of a population adopt it, there is no ”mutant” strategy that would give higher reproductive fitness (p. 15).

The ESS concept is a difficult one for many to grasp, and I’m not going to go into a full report of the subtleties here. The important points to know are:

• An ESS strategy guarantees that a population which adopts that strategy is uninvadable by a small proportion of mutants using another (single) strategy. If the population is using the ESS strategy A and a small percentage (say 1%) mutates to strategy B, ESS theory guarantees that the B mutants will not be able to invade and will die out.
• For those in the audience who like to see a little math, a slightly more formal definition of an ESS is as such: in an infinite asexually reproducing population of individuals who each adopt a strategy in a frequency-dependent payoff situation and encounter each other in pairs, the expected payoff E to an individual playing strategy I against an individual playing strategy J is E(I , J ). With this (incomplete) notation in place, an ESS is a strategy which satisfies the following conditions:
  • , or
  • and .
• In English, this states that for a strategy to be stable, individuals adopting the ESS must receive a higher payoff against other ESS players than mutant individ- uals do against ESS individuals. Failing that, if mutants receive the same payoff against ESS players as ESS players do against themselves, then ESS players must receive a higher payoff against mutants than mutants do against themselves. The first condition states that mutants can’t invade; the second condition says that if they can invade (by drift), that they will still be selected against if they become appreciably common in the population.

To this point, I’ve presented things in a backwards fashion, because the formal definition of an ESS was actually the last thing that Maynard Smith and Price actually did in the paper. To begin, they set the stage by presenting computer simulation results on a set of fictional strategies repeatedly playing a game against each other. The game was a variation of what is now famous as Hawk / Dove, though interestingly Price insisted that Dove be known as ”Mouse” for religious reasons. The game involved repeated interactions between two players who could adopt one of two tactics: ”(D)angerous”, which would be likely to cause serious harm, or ”(C)onventional” , which would be unlikely to cause serious harm. At any point, either player could choose ”(R)etreat”, after which the game would be over. These tactics, D / C / R, along with probabilities for playing each in any round of the game, constituted a strategy. If a player played D, there was a chance that their opponent would be seriously injured, after which the opponent would be obliged to retreat. Payoffs were given for short combats.

In the simulation, five different strategies were tested against each other: a ”total war” strategy known as Hawk, three ”limited war” strategies called Mouse, Retaliator, and Prober-Retaliator, and a final strategy called Bully. Hawk would always play D, until seriously injured or the opponent retreated. Mouse would never play D, and if the opponent played D would retreat immediately. The other three strategies used some combination of C and D in different ways.

The details after that are only of interest to those who would like to read the paper, because the main point to come out of the computer simulations is that t the limited war strategies did better than the total war Hawk. Thus, Maynard Smith and Price were able to demonstrate that under the assumptions of the Hawk / Dove game, it could indeed be adaptive to forego fatal conflict as a result of individual selection. By following this up with a formal investigation of the properties of ESSs, they were able to show that this was not just a fluke but that in the game they used Hawk could not be an ESS.

Maynard Smith was already a well known figure in biology at this point, and he has retained much of the credit for the subsequent explosion of biological game theory and the notion of the ESS. However, he himself freely admitted that Price was a major driver of the initial work on ESS theory. As noted in Ullica Christina’s book Defenders of the Truth: The Sociobiology debate (Christina, 2000), Maynard Smith actually introduced the idea of the ESS first in a book published in 1972, in which he stated in the acknowledgements:

The essay on ’Game Theory and the evolution of fighting’ was especially written for this book. I would probably not have had the idea for this essay if I had not seen an unpublished manuscript on the evolution of fighting by Dr. George Price, now working in the Galton Laboratory at University College London. Unfortunately, Dr. Price is better at having ideas than at publishing them. The best I can do therefore is to acknowledge that if there is anything to the idea, the credit should go to Dr. Price and not to me (Maynard Smith, 1972, p. viii).

(Price’s connection with game theory, his accomplishments in biology, and his tragic character are noted further in Christina’s book.) Further, the ideas developed in this paper and in other work by Maynard Smith were clearly influenced by the work of both Hamilton’s work on ”uninvadable strategies” (Hamilton, 1967) and Fischer’s work on sex ratios in the 1930s (Fisher, 1930). In fact, I remember seeing that Maynard Smith had tried to read Morgenstern and von Neumann’s book, but had failed to get far enough through the dense mathematics to find out that M. & v-N. had effectively beaten him to the punch on ESS theory by several decades (though the source of this escapes me – does anyone know what it is, or know if it’s apocryphal?)

This being the case, though, it was the Nature paper which really brought biological game theory to light, leading to the seminal book on the sub ject by Maynard Smith some nine years later (Maynard Smith, 1982). Those works led to the widespread adoption of a new way of thinking about frequency-dependent problems and optimization in biology, an approach that continues to bear fruit even today. For anyone looking to get a better handle on the intricacies of game theory or to see where its application in biology really started, I recommend that you read this paper. (As a bonus, it’s short – 4 pages – and generally quite readable….)

References:

• Christina, U. (2000). Defenders of the Truth: The sociobiology debate. Oxford: Oxford
  University Press.
• Fisher, R. A. (1930). The Genetic Theory of Natural Selection. Oxford: Clarendon Press.
• Hamilton, W. D. (1967). Extraordinary sex ratios. Science, 156 (3774), 477–488.
• Maynard Smith, J. (1972). Game theory and the evolution of fighting. In J. Maynard Smith (Ed.) On Evolution. Edinburgh University Press.
• Maynard Smith, J. (1982). Evolution and the Theory of Games. New York, NY:Cambridge University Press.
• Maynard Smith, J., & Price, G. R. (1973). The logic of animal conflict. Nature, 246(1), 15–18.
• von Neumann, J., & Morgenstern, O. (1944). Theory of Games and Economic Behaviour. New Jersey: Princeton University Press.