Theorists have posited that pure tit-for-tat strategies in iterative prisoners dilemma games were invulnerable. Is this correct? The authors seek to answer this question by examining the ability of various prisoners dilemma gaming strategies to withstand invasion by other competing strategies.

Bender and Swistak examine a gaming strategy universe that includes the strategies:

• Tit for tat - a player will initially cooperate and then in future rounds mimic the behavior of their opponent.
• Tit for 2 Tats - a player will cooperate for the first two rounds and then defect in rounds where their opponent defected in the previous two.
• Suspicious Tit for Tat - a player will initially defect and then will mimic their opponent in future rounds.
• Always Defect
• Always Cooperate
• Grim Trigger - Begin by cooperating, if opponent defects then always defect afterward.

These strategies were examined in pure conditions where only one existed, and then competing strategies were introduced. If a given strategy could withstand incursions by competing strategies it was deemed "stable".

Stability proved to be a continuum. All strategies proved to have points of equilibrium. At this point, a strategy can withstand its maximum level of incursion. That point is that strategy's maximum stability.

Chapter 1, The Problem of Cooperation. Why do people (or other actors) cooperate? "The objective of this enterprise is to develop a theory of cooperation that can be used to discover what is necessary for cooperation to emerge." It uses the Prisoner's Dilemma as a framework for testing theories about balancing self-interest and competition.

"In the Prisoners' Dilemma, the strategy that works best depends directly on what strategy the other player is using and, in particular, on whether this strategy leaves room for the development of mutual cooperation."

Chapter 2, TIT FOR TAT. "The iterated Prisoners' Dilemma has become the E. Coli of social psychology," yet people have not paid much attention to how to play the game well. Axelrod organized a computer tournament to which people familiar with PD submitted programs encoding different strategies. The winner was one of the simplest, TIT FOR TAT.

Axelrod then constructed an environment in which different programs competed, and the losing programs were eliminated: this was an ecology that rewarded high scoring programs, and punished others. "This process simulates survival of the fittest. A rule that is successful on average with the current distribution of rules in the population will become an even larger proportion of the environment of the other rules in the next generation. At first, a rule that is successful with all sorts of rules will proliferate, but later as the unsuccessful rules disappear, success requires good performance with other successful rules." In other words, the competition gets tougher.

"The analysis of the tournament results indicate that there is a lot to be learned about coping in an environment of mutual power. Even expert strategists from political science, sociology, economics, psychology, and mathematics made the systematic errors of being too competitive for their own good, not being forgiving enough, and being too pessimistic about the responsiveness of the other side."

The tournaments reveal that "there is a single property which distinguishes the relatively high-scoring entries from the relatively low-scoring entries. This is the property of being nice, which is to say never being the first to defect."

TIT FOR TAT's rules for success:

• Be nice. Don't be the first to go on the attack. This demonstrates good will, and avoids provoking others.
• Retaliate. If others attack, retaliate. Not doing so encourages bad behavior and gives niceness a bad reputation.
• Be forgiving. If others defect but then go back to cooperating, accept the opportunity to move back to a cooperative mode.
• Be clear. Others can predict what you'll do, be certain that their moves will have definite outcomes. "There is an important contrast between a zero-sum game like chess and a non-zero-sum game like the iterated PD. In chess, it is useful to keep the other player guessing about your intentions. The more the other player is in doubt, the less efficient will be his or her strategy. But in a non-zero-sum setting it does not always pay to be so clever. In the iterate PD, you benefit from the other player's cooperation."

Chapter 4, Trench Warfare. During World War I, "live and let live" arrangements emerged spontaneously between opposing units on the Western Front. Cooperation could take hold because "the same small units faced each other in immobile sectors for extended periods of time." Consequently, they had a more sustained relationship than in mobile warfare, and could develop commonly-understood rules, reciprocity and restraint in attacks, displays of strength (e.g., snipers shooting at hard targets)as well as ethics (recognition that there was an arrangement and violating it was immoral) and rituals (e.g., regular artillery firing).

"Cooperation first emerged spontaneously in a variety of contexts, such as restraint in attacking the distribution of enemy rations, a pause during the first Christmas in the trenches, and a slow resumption of fighting after bad weather made sustained combat almost impossible. These restraints quickly evolved into clear patterns of mutually understood behavior, such as two-for-one or three-for-one retaliation for actions that were taken to be unacceptable."

Chapter 6, How to Choose Effectively. Four suggestions about how to do well in PD:

• Don't be envious. In a PD, "envy is self-destructive. Asking how well you are doing compared to how well the other player is doing is not a good standard unless your goal is to destroy the other player." However, in an iterated prisoner's dilemma, you can't do better than the other player, unless they're always suckers. "In a non-zero-sum world you do not have to do better than the other player to do well for yourself. The other's success is virtually a prerequisite of your doing well for yourself."
• Don't be the first to defect (be nice). "It pays to cooperate as long as the other player is cooperating." In a short game, defection can make sense; but in a relationship, taking advantage of the other person is self-defeating.
• Reciprocate both cooperation and defection. TIT FOR TAT "does not destroy the basis of its own success. On the contrary, it thrives on interactions with other successful rules." However, the right level of forgiveness depends on the context, and the other players' strategies.
• Don't be too clever. "In a zero-sum game, such as chess it pays for us to be as sophisticated and as complex in our analysis as we can. Non-zero-sum games are not like this. The other player can respond to your own choices. And unlike the chess opponent, the other player in a PD should not be regarded as someone who is out to defeat you." "There is an important contrast between a zero-sum game like chess and a non-zero-sum game like the iterated PD. In chess, it is useful to keep the other player guessing about your intentions. The more the other player is in doubt, the less efficient will be his or her strategy. But in a non-zero-sum setting it does not always pay to be so clever. In the iterate PD, you benefit from the other player's cooperation."

Chapter 7, How to Promote Cooperation. Promoting cooperation can be thought of as an exercise in tinkering with the variables in a PD. "As long as the interaction is not iterated, cooperation is very difficult. That is why an important way to promote cooperation is to arrange that the same two individuals will meet each other again, be able to recognize each other from the past, and to recall how the other has behaved until now."

• Enlarge the shadow of the future. For cooperation to emerge, players must be in a continuing relationship, with the expectation that it will continue in the future. "Mutual cooperation can be stable if the future is sufficiently important relative to the past." "There are two basic ways of doing this: by making the interactions more durable, and by making them more frequent. [P]rolonged interaction allows patterns of cooperation which are based on reciprocity to be worth trying and allows them to become established," Making interactions more frequent makes "the next interaction occur sooner, and hence the next move looms larger than it otherwise would." You might do this by enforcing isolation, or constructing hierarchies or organizations, which are "especially effective at concentrating the interactions between specific individuals."
• Change the payoffs. Make defection less attractive, by enforcing laws, or growing the value of long-term incentives.
• Teach people to care about each other.
• Teach reciprocity. Reciprocity "actually helps not only oneself, but others as well. It helps others by making it hard for exploitative strategies to survive."
• Improve recognition abilities. "The ability to recognize the other player from past interactions, and to remember the relevant features of those interactions, is necessary to sustain cooperation. Without these abilities, a player could not use any form of reciprocity and hence could not encourage the other to cooperate."

Chapter 8, The Social Structure of Cooperation.
The social structure of cooperation involves labels, reputation, regulation, and territoriality.

• Labels are fixed characteristics of an agent that are observable by other agents. Labels affect reciprocity and retaliation via assumptions of group similarity and stereotypes.
• Reputation is others' belief about the strategies an agent will employ. Reputation may be based on past behavior or on rumours, i.e. reputation can be accurate or merely believed. Reputation affects whether or not other agents will cooperate or defect with you.
• Regulation involves setting the stringency of a standard of behavior "high enough to get most of the social benefits of regulation, and not so high as to prevent the evolution of a stable pattern of voluntary compliance from almost all of the companies" (or regulated agents).
• Territoriality refers to both physical and conceptual spaces that can be 'invaded' by agents of differing strategies. Territoriality establishes boundaries within which behaviors will be reinforced or retaliated against depending on prevailing norms. Also, the boundary provides an 'inside' for agents that comply with the norms, and an 'outside' to which they can be expelled if they do not comply.

Chapter 9, The Robustness of Reciprocity.

• Cooperation can get started by even a small cluster of individuals who are willing to reciprocate cooperation, even in a world where no one else will cooperate.
• Once cooperation is establish, it protects itself from invasion by non-cooperative strategies.
• The foundation of cooperation is the durability of the relationship, which allows agents to learn about each other in order to cooperate.