Bmc Evolutionary Biology the Evolution of Competition and Policing: Opposing Selection within and among Groups

Background: Although selection favors exploitative competition within groups, a group of hypercompetitive individuals may be less productive than a cooperative group. When competition is costly for group fitness, among-group selection can favor groups with 'policing' individuals who reduce within-group competition at a cost to their own fitness, or groups of individuals who restrain their competitive intensity ('self policing'). We examine these possibilities in a series of explicit population-genetic models.


Background
A major impediment to the evolution of productive social interactions is what Hardin [1] termed "The Tragedy of the Commons" (TOC). Because individual self-interest favors unfettered exploitation of a resource, whenever two or more individuals share a common environment and make simultaneous demands on its resources, conditions are ripe for the TOC. Opportunities for the TOC include groups of microbes living in the same host, offspring within the same family, and genes in the same organism [2]. Wright ([3] p. 129), Wynne-Edwards ( [4] p. 623) and Hamilton [5] viewed the TOC as the problem of control-ling cheaters, lest they lead to the destruction of the group (see also [6][7][8]).
Resolution of this tension between the interests of the group and the individual allows major evolutionary transitions, wherein communities of individuals merge into a single functional unit [8][9][10][11]. The processes proposed for resolving this tension are kin selection for altruism and enforced cooperation via policing. These are viewed as alternatives, since the former is facilitated by high levels of genetic relatedness among group members, while the latter is favored by low levels of relatedness. Although group selection works best with high levels of relatedness, "contrary to simple expectation" ( [12] p. 42), policing theory [13,14] and data from vespid wasps [15,16] indicate that the highest levels of policing occur in groups with the lowest levels of genetic relatedness. Hence, policing is viewed as a mechanism for integrating groups of unrelated individuals, while altruism is seen a mechanism for cohesion in related groups [2,[12][13][14]17,18].
In this paper, we use explicit population genetic models to show that costly policing is favored by selection among groups and opposed by selection within groups in proportion to relatedness among group members in the same manner as kin-selected altruism: although altruistic and enforced cooperation may be alternative mechanisms for group cohesion, they evolve by the same evolutionary process. We examine the evolution of individual competitive ability and policing in the first two-trait model of individual and group selection. Individual selection within groups favors enhanced competitive ability but opposes costly policing. Conversely, group selection opposes the evolution of competitive ability but favors the evolution of policing.
Unlike previous optimization treatments of policing, our explicit genetic model reveals stable interior equilibria as well as the rate of progress toward them. Owing to the multiplicative nature of the group-mean fitness function (a pres [13]), many combinations of competition and policing have equal fitnesses. Although mutation analysis reveals that these interior equilibria are invasible and can move toward the global optima found by Frank [13], evolutionary progress toward the global equilibrium is exceedingly slow, on the order of the square of the inverse of the mutation rate, because it depends upon the order of occurrence of competition and policing mutations. These global optima may exist but few species will endure long enough to guarantee arrival at them.
Ratnieks [19] coined the term, 'policing' in the context of social insect colonies where worker females lay haploid, male eggs that are often eaten by other workers. Policing behaviors can mitigate the individually beneficial and the communally detrimental effects of individual selfishness ( [2,13,14,20]). When individuals engage in mutual policing at some cost to themselves, the group as a whole functions as an evolutionary unit, as in the social insects, where "Worker policing ... may be selected due to the colony-level benefit of conflict suppression" ( [21] p. 169). Additionally, policing is thought to be particularly important in the maintenance of sociality in higher primates. Alexander [22] argued that socially imposed rules and traditions favor group-level efficiency in humans, and Flack and colleagues have found evidence for the importance of policing in the coherence of groups of pigtailed macaques [20,23].
Frank [2,13,14,17,18] was the first to formally examine the co-evolutionary interplay between competitive and policing behaviors. In the simplest version of his model, Frank [13] envisions that each individual has two quantitative traits, policing and competition. The fitness, w ij , of the j th individual in the i th group is given by the function where z ij is the competitive ability of the j th individual and z i. is the mean competitive ability of individuals in group i. Similarly, a ij is the amount of policing by the j th individual in the i th group and a i. is the average level of policing within group i. The constant, c, represents the cost of policing to the individual and it is assumed to be the same for all individuals. Individual fitness increases as z ij increases and is favored by selection within groups. However, mean group fitness is a declining function of mean z i and is opposed by among group selection.
Analyzing the optima of eq. [1], Frank found that: (1) mean competitive ability decreases with increasing relatedness; and, (2) mean level of policing increases with increased competition. We use Frank's fitness model in our investigation of the evolution of competition and policing in populations composed of either full or half sib families. We examine these different levels of relatedness in order to explore directly how relatedness affects the evolution of policing and competition. We partition total selection into within and among group components [24] to make the levels of selection explicit and show how the major findings of Frank [13] emerge from the conflicting forces of selection within and between groups. Unlike Frank [11] we find many stable interior equilibria, in which extreme competitors coexist with less competitive genotypes, and effective police coexist with civilians; suggesting that a population need not necessarily arrive at the monomorphic state predicted by Frank [13]. Moreover, we find that like multiplicative selection functions of twolocus individual selection, joint selection on competitive and policing alleles does not generate linkage-disequilibrium, and thus the hyper-competitive genotypes are equally likely to be police or civilians.

General Approach
In our model, policing and competition only occur within sibling groups. This can be visualized as competition over a resource proximal to the location where offspring are raised, or competition over maternal resources. After sib- lings mature, they leave this locale, mix with other members of the population and mate randomly, with no further selection on levels of competition or policing.
To further simplify analysis, we assume a large population in Hardy-Weinberg equilibrium, and we ignore the effects of mutation, migration, and random genetic drift. For each model, we identify all parental mating types and their respective frequencies, the offspring genotypes from each mating type, and the change in genotype frequencies within each family that result from the fitness costs and benefits of competition and policing. Next, we calculate mean absolute fitness of each family. From these values, we calculate the changes in gene frequency due to selection within sib groups, selection among sib groups, and the total change in gene frequency per generation. We complement our technical treatments by concluding each model with a qualitative summary of our findings.

Model 1: The Evolution of Competitive Ability
We envision a population with no policing where an individual's level of competition is controlled by one locus with two alleles with additive affects. Let allele A occur in frequency p and the alternative allele, a, in frequency q = (1 -p), so that, after random mating, AA, Aa, and aa individuals are in frequencies p 2 , 2pq, and q 2 , respectively. Let AA individuals have a baseline competitive ability denoted by z 0 . Adding an a allele to the genotype, increases competitive ability by an amount, z a . Thus, Aa individuals compete at the level z 0 + z a , while aa individuals compete at the level, z 0 + 2z a . Mean competitive ability of the population equals z 0 + 2qz a .

Model 1A: Competition Without Policing
Population-genetic dynamics Setting a i. = a ij = 0 in eq. [1] (to reflect no policing), the fitness of an individual of genotype j in family-type i is mean fitness in family i equals and mean fitness across all families is We find the frequency of allele a after within family selection by taking the product of the frequency of each family type (f i ), the change in frequency of allele a due to selection within this family (∆q i ), and the mean fitness of this family (w i ), summing across all family types and dividing by (see [25] for procedure). Assuming weak selection as in Wade [24,25], we approximate f i from Hardy-Weinberg expectations, and thus solutions in text are approximations. In Additional file 1, we present the exact solutions, which differ only slightly from the approximate solutions as also found by Wade [25] for a very different fitness function.
We subtract the frequency of allele a before selection to find the change in q due to selection among and within groups [26][27][28] Total change in frequency of allele a (∆q total ) is the sum of eqs. [5] and [6]. Values of f i. , ∆q i. , q i. , and w i. for full and half sib families are presented in Tables 1 and 2, respectively. Table 3 shows mean fitness, as well as ∆q among groups, within groups, and total (discussed below).

(2)
w z = − 1 (4) w ∆q w f q w qw . Thus, the difference in the strength of within group selection depends on allele frequency, as well as the level of competition. Specifically, when q < 0.5 and z 2hs < 1.0, within-group selection favoring competition will be stronger for half-sib families than it is for full-sib families; however, as q rises above 1/2 and z 2hs rises above 1, within-group selection favoring competition can be stronger for full-sib families than it is for half-sib families. Since z 2HS cannot exceed 1 in this model, the force of within-group selection can only be stronger in a population of full-sib than half-sib families if the competitive allele is very common.
The formula for total selection on competitive ability is similarly complicated ( Table 3, Row 5). Subtracting ∆q total full-sib from ∆q total half-sib yields . Since selection on competition is dependent on z 0 , z a and q, it is difficult to interpret the effect of family structure on evolution from this result.

Solving for optima
By solving for the equilibrium value of q* (setting ∆q total = 0), and comparing results for half and full-sib families

Half sib Full sib
Mean fitness ( ) Furthermore, with a few simplifying assumptions we show that this approach provides the same results as Frank [13,14]. We begin with a population fixed for allele p, so that z .. = z 0 . We then introduce a rare mutant carrying allele q and find ∆q total for both half and full sib families. Figures 2a and 2b display the sign of ∆q total (when q is rare) as a function of z 0 and z a for half and full sib families, respectively. These figures are similar to the pairwise-invasibility-plots of adaptive dynamics [29]; however, rather than presenting z resident and z invader we present z 0 (which can be thought of as z resident ), and z a (z invader -z resident ).
We can then obtain the stable level of competition, z*, by setting ∆q = 0 and solving for z 0 as z a approaches zero. Let z* represent the invasion equilibrium, the value of z 0 at which an invader with small levels of z a does not change in frequency. This equilibrium equals 3/4 for half sib fam-ilies and 1/2 for full sib families, coincident with Frank's solution that z* = 1 -r. For both family types, z* is both evolutionarily and continuously stable, meaning that z* cannot be invaded by a rare allele with any nonzero value of z a and, reciprocally, that a rare allele entering any monomorphic population will increase in frequency if the allele increases the level of competition and z 0 is less than z 0 *, or if the allele decreases the level of competition and z 0 is greater than z 0 *. If the mutant not only invades, but is driven to fixation, then the population becomes monomorphic for a new level of competition, z 0 ' which equals z 0 + z a . Now if a new mutant allele, z a , has a small effect on the level of competition and z 0 + z a individuals are closer to z 0 * than z 0 ' this new value of z a can spread and now the sum of z 0 ' + z a becomes the new z 0 . A continuation of this process will ultimately lead to the stable level of competition, z 0 , described above.
Population-genetic constrains However, we are not guaranteed that the evolution of competitive behavior will follow the particular steps out-Equilibrium frequency (q*) of a 'competitive' allele Figure 1 Equilibrium frequency (q*) of a 'competitive' allele. q* as a function of the base-line level of competition (z 0 ) and the additive effect of the competitive allele (z a ). The equilibrium frequency, q* varies continuously from fixed to lost. Lines mark a change in q* of 0.2, and are labeled by numbers to their right.
lined above. Of special interest is the observation that, for many combinations of z 0 and z a , there exist a stable level of q and p such that a dimorphism of competitive behavior is maintained ( Figures 1A and 1B display equilibrium values of q for half and full sib families, respectively). We can show that such a stable equilibrium is invasible by a mutant with competitive ability closer to z* and resistant to invasion by mutants with competitive ability further from z* (in Additional file 3 we show the derivation of this result for the full-sib model). However, we have not explored the possibility of a new stable intermediate equilibrium, involving 2 or more alleles, in between these boundary analyses. Additionally, it is widely known that complex genetic systems, with dominance, epistasis and pleiotropy can act as constraints, preventing traits from approaching their optima; we have not formally addressed the influence of these factors in the evolution of competitive intensity.
The evolutionary dynamics of competitive ability may also deviate from the description above if competitive ability interacts with another trait that affects the spread of competition. Frank [2,13,14] added policing behavior to the model of competition. Below, we also add a fixed level of policing to this model and examine the evolutionary interaction between these two traits.

Model 1B: Competition with a Fixed, Nonzero Level of Policing
Population-genetic dynamics With a fixed level of policing (such that a i. = a ij = a in eq. [1]), the fitness of an individual of genotype j in familytype i changes from eq. [2] to become mean fitness in family i equals and mean fitness across all families is Invasibility of a 'competitive' allele Figure 2 Invasibility of a 'competitive' allele. Invasibility of competitive allele with additive effect, z a , on its competitive intensity, in a population fixed for z 0 competitive intensity. The sign, + denotes regions in which the competitive allele can invade when rare, the sign, -denotes regions in which it cannot.
When policing is absent, (i.e., a = 0) eqs. [6][7] reduce to eqs. [2][3][4]. Tables 4 and 5 show family characteristics including frequencies, fitnesses and allele frequency change for within full and half-sib families, respectively. In Table 6, we show mean fitness, as well as the ∆q approximate solutions for half and full-sib families.

Solving for optima
From the ∆q total expressions in Table 6 we derived z 0 * for half and full-sib families. Frank ([13] - figure 1B) shows that, when selection favors any level of policing, (1) policing effort (a) will approach 1, and (2) the level of competition will evolve to . Here we address prediction 2 by inserting the values, r = 1/2 and 3/4 for full and half sib families, respectively, into Frank's prediction and comparing it to our own derivation of z 0 *. As a → 1, we find that the two approaches provide equivalent solutions, namely, and .
Intriguingly, both solutions result in values of z* > 1, which, in the absence of policing, would result in negative fitnesses (see eqs. [2][3][4]). Values of z* in excess of 1 are only possible with some level of policing already present in the population, so that no families or individuals suffer from w < 0. Frank ([14] p. 1166) explained this result as follows: "The high competitiveness in a policing situation is no different from high internal pressure in a fish that lives at great depth. The fish brought to the surface explodes; intense competition and avoidance of repressive policing cause chaos when the same amount of energy is devoted to competition in the absence of repressive policing. " Although Frank's and our approaches provide equivalent solutions for the level of competition in half and full-sib structured populations with complete policing, some information is lost in this comparison. Frank's approach allows for the derivation of z* with any quantitative value of r, while we have only solved for two specific values of relatedness, r = 1/2 and r = 1/4. By contrast, our model provides a prediction for the optimal level of competition when the level of policing (a) lies between the two stable solutions (1 and 0). Thus, the two derivations of z* provide complimentary solutions to the evolution of competitive ability in a police state.

Summary of Model 1: The Evolution of Competitive Ability
We apply a family selection approach to the problem of the evolution of interference competition. In the long term, without 'policing,' we find that mean competitive ability will approach a value of 1/2 for full-sib and 1/4 for half-sib families, in agreement with previous research [13]. However, we also discovered a plane of stable equlibria between competitive specialists and a less competitive class. When a fixed level of policing behavior is added to this model, the regions of stable equilibria are reduced and a sharp increase in competitive ability is favored. In some sense, although policing limits the expression of competitive ability, the innate tendency to compete is enhanced beyond that able to evolve in the absence of policing: policing permits the evolution of more extreme interference competition.

Model 2: The Evolution of Policing with a Fixed Level of Competition
We now examine the evolution of policing behavior in a population with a fixed level of competition, assuming that an individual's level of policing is controlled by one locus with two alleles with additive affects. Let allele B occur in frequency t and the alternative allele, b, in frequency u = (1 -t), so that, after random mating, BB, Bb, and bb individuals are in frequencies t 2 , 2tu, and u 2 , respectively. Let BB individuals have a baseline level of

Population-genetic dynamics
In this model, policing decreases the benefit of competition to within-group fitness, and decreases the deleterious effect of competition on between-group fitness. Following [1], and setting z = z ij = z i. = z .. to represent a fixed level of competition, the fitness of the j th individual in the i th group equals the mean fitness of group i equals and population mean fitness equals where c and a represent the cost and degree of policing, respectively, σ 2 a_AF represents the among-family variance in policing effort. For both half and full sib families, we find the frequency of allele b after selection among and within families following eqs. [5] and [6] respectively, maintaining the same major assumptions, but substituting u for q. Values of f i. , ∆u i. , u i. , and w i. for full and halfsib family-structured populations are presented in Tables  7 and 8 , which is zc times the difference in among-family variance in policing effort for half and full-sib families (Table 9).
Clearly, policing decreases the benefit of competition to within-group fitness, and decreases the deleterious effect of competition on between-group fitness; however, this group-level benefit is mediated by the cost of policing. The direction and magnitude of among-group selection on policing effort ( . _ σ (12) w w a i.   (1-a)) pqz a (1-a)/(2z 1HS (1-ac)) Aa 2*p*q p/2 1/2 q/2 (q/2+1/4) z 0 +za(q+1/2) = z 2HS (1-ca)(1-z 2HS (1-a)) ((za(1-a))/(z 2HS (1-ac)))(1/8+pq/2) aa q^2 ---p q (q/2+1/2) z 0 +za(q+1) = z 3HS (1-ca)(1-z 3HS (1-a)) pqz a (1-a)/(2z 3HS (1-ac))   While among-group selection can favor increased policing, within group selection always favors reduced levels of policing (Table 9) Note that the claim that costless policing does not involve kin selection [30] is not substantiated by our model: when the cost of policing (c) is set to 0, there is no selection within groups and policing evolves solely by among-group selection. The difference between ∆u withinFS and ∆u withinHS equals It can be shown that ∆u withinFS is greater than ∆u withinHS when . Since within-group selection decreases the frequency of a policing allele, and since z cannot be greater than one unless there is already a significant level of policing in the population, selection against policing is initially stronger in populations consisting of half-sib families than it is in populations of full-sib families, but as the level of policing increases this difference diminishes. This result partially explains why half-sib families can sustain high levels of policing -within-group selection against high levels of policing is weaker in half-sib based populations than in full-sib based populations.
The difference between populations structured as full and half-sib groups in the total change in policing-allele frequency is the sum the differences between them from selection within and between groups. Under most parameter values, policing is more strongly favored in full-sib than in half-sib structured populations ( Figure 3); again, this is contrary to the beliefs that reduced relatedness favors the evolution of policing and that the evolution of policing is a process distinct from kin selection.
Inserting values of z* found in the previous section (1/2 for full sib and 3/4 for half sib families) for z, an allele which increases policing effort can increase in frequency when or for full and half sib based populations, respectively. Beginning with no policing (a 0 = 0), a rare mutant (u is small) with a small policing effort (a a is small and positive) will only increase when c < 1/2 for full sib, or c < 3/4 for half sib structured populations. This result is consistent with Frank's first equilibrium prediction -when r > 1 -c, policing will not invade and the system will be stable with z ij = z i. = z .. = z* = 1 -r, and a ij = a i. = a .. = a* = 0.
We can also solve for the equilibrium value of a 0 * with a

Summary of Model 2: The Evolution of Policing Effort
We apply a family selection approach to the problem of the evolution of policing effort. Selection among families generally increases the frequency of an allele that increases policing effort, while selection within families always decreases the frequency of a policing allele. Under identical parameters, selection among half sib groups favoring a policing allele is twice as strong as the same selection among a population of half-sib groups. By inserting equilibrium values of z* from section 1A into our ∆u expressions, we show that a nonzero level of policing can evolve only when c < 1/2 and 3/4 for full and half-sib families, respectively.

Model 3: The Co-evolution of Policing and Competition
Population-genetic dynamics In this model we examine the co-evolution of competition and policing behavior. In this two-locus model we retain the naming conventions in models 1 and 2. The fitness of the j th in the i th group is given in eq. [1]. The mean fitness of the i th family equals and the mean population fitness equals Family tables are presented in Additional file 2.
The results of this model parallel results from models 1B and 2. Mean population fitness, ∆u within families, among families and total are equivalent to values in Table  9 with replacing z. Similarly, ∆q within and among families as well as ∆q total are equivalent to values listed in 6, with replacing a. No linkage disequilibrium is generated by selection either within or among groups, therefore the dynamics of the co-evolution of policing and competition do not depend on whether they are jointly polymorphic, or if the invasion of a new type occurs while the other locus is monomorphic. As in models 1 and 2, genetic constraints such as mutational step size, and dominance can prevent populations from attaining these equilibrium values.
Solving for optima As in model 2, and as Frank [13,14] found, policing will only evolve when r < 1 -c. When policing is favored by selection, policing effort will rise above one, the biological cap on the level of policing. By constraining the system to avoid negative individual fitnesses (by keeping a < 1), when policing is favored it will approach one and the level of competition will approach that found in model 1B (above).

Population-genetic constraints
Notably, although this two-gene, two-trait model makes equivalent 'long-term' predictions as the single-trait based approaches, this model places additional constraints on the order and size of effect of mutations to competition and policing, which may prohibit a population from achieving the ultimate equilibria. That is large mutational many mutations at one locus, that place a population closer to long-term equilibria may be lost because the appropriate trait values at the other locus do not yet exist.

Discussion and Conclusion
Conflict between the levels of selection is a classical problem of evolution theory [6,8,25,[31][32][33][34]. Here, in formal evolutionary genetic models, we investigated the evolution of two traits, interference competition and policing, that, not only, interact to affect fitness but also are affected by selection within and between groups, albeit in opposite ways. The first trait, interference competition, increases the fitness of individuals within groups but does so at a cost to group fitness. The second trait, policing, increases group mean fitness but at a cost to the fitness of policing individuals within groups. By partitioning total selection into within and among group components, we showed how each level of selection contributes to the evolutionary dynamics and long-term evolutionary outcome of selection on policing and competition. We noted that costless policing involves only selection among groups.
We found that competition is opposed by among-group selection and decreases with higher levels of genetic relatedness: over all values of cost and for any given level of policing, there is a negative relationship between competition and relatedness. That is, under identical parameters, the equilibrium values of competition are always higher for half-sib than for full-sib structured populations (Fig.  4a). As has been shown before [13,14,27], kin selection opposes competition within kin groups. We also find that interference competition with policing can evolve to a level that exceeds its maximum value (z* = 1) without policing, a finding similar to that of Frank [13,14]: a high level of policing allows a high level of interference competition.
One of our novel findings is that policing is favoured by among-group selection and it reaches higher values with higher genetic relatedness within groups as we illustrate graphically (Fig. 4). Thus, kin selection favors the evolution of policing: across a range of costs of policing and for a given level of competition, the equilibrium level of policing is higher for full-sib than for half-sib structured populations (Fig. 4b). As well, the rate of increase of a . _ σ (16) z a policing allele increases with genetic relatedness (Fig. 3). We conclude that, although policing may be a different mechanism than altruism for ensuring group cooperation, it evolves by the same evolutionary process as altruism, namely, kin selection.
We also find that, owing to the multiplicative nature of the group-mean fitness function, the mean level of competition has a strong influence on the equilibrium level of policing and vice versa (Figs. 4c,d). However, the influences are not strictly reciprocal. The equilibrium level of policing cannot rise above one (where it renders all competition ineffective) and remains at zero until high values of competition exist, while the equilibrium level of competition can get very large and grow very quickly, especially when policing is costly. Together, these trait associations result in a negative relationship between the equilibrium values of policing and relatedness -the high levels of competition favored in populations with low relatedness create a strong selective pressure favoring policing behavior and overcome the weakness of kin selection in less genetically structured groups. Thus, the joint equilibrium relationship between relatedness and policing is the opposite of the dynamical one. It is this difference between the dynamic and equilibrium relationship to relatedness that has led some to interpret policing as an alternative to kin selection. Our dynamic model reveals the source of this confusion.
Our evolutionary genetic approach also reveals the existence of planes of stable, interior equilibria. For a fixed level of policing, there exists a plane of stable equilibria Equilibrium values of competition and policing with competitive specialists and a less competitive, selfpolicing class (Figures 3 and 4). Similarly, the evolution of policing with a fixed level of competition gives rise to planes of stable gene frequency equilibria with policing and non-policing individuals (Brandvain unpublished). These non-policing individuals are cheaters or free-riders by virtue of the fact that they bear none of the costs of policing but reap the reward of reduced within-group resource competition. Together, these findings indicate that neither trait may reach the ultimate evolutionary outcome characterized by optimizing the mean fitness function. Our further analyses of the one-trait models, adding mutant alleles causing increased competition or reduced policing, respectively, result in the population moving from one region of stable equilibria to another, toward the ultimate outcomes predicted by Frank's model. Thus, our regions of stable equilibria are invasible by mutations and the ESS coincides with that predicted by Frank and his collaborators. However, rates of evolution can be very slow and depend upon the fortuitous input of a continuing series of mutations with effects of a particular kind occurring in the right order with respect to the equilibrium. Thus, when both traits are evolving simultaneously, there is reason to question whether the ESS can be achieved.
We used a two-locus model to investigate the simultaneous evolution of competition and policing. Like individual selection models with multiplicative fitnesses, selection in this model does not generate linkage disequilibrium between the competition and policing loci. As a result, the evolution of each trait is affected by the genotypic mean value of the other (playing the role of the fixed levels in the one locus model). Hence, we find similar regions of stable two-locus equilibria, with mixed levels of both competition and policing behaviors. Similarly, we find that mutant alleles at either locus can invade these stable equilibria. However, depending on the order and magnitude of the mutational effects introduced, the population can move either toward or away from the ultimate evolutionary outcome. Thus, achieving the joint ESS depends not only on the occurrence of mutations for policing and competitive ability but also on the mutations occurring in the right order, a much more onerous requirement than in the one-locus models. As a result, a population's sojourn away from the ESS may be extremely long and we should expect diversity among species (or among populations of the same species) in levels of competition and policing.

Biological Implications
In some circumstances, different genes can experience different values of relatedness. For example, maternally inherited mitochondrial genes can be more closely related than diploid nuclear genes and the mating system can change both the absolute and relative values [35]. In genomic imprinting (the differential expression of a gene based on its parent of origin), a similar situation arises within the broods of multiply mated females: siblings are more closely related for maternally derived alleles than they are for paternally derived alleles. High relatedness of paternally derived alleles in monogamous or inbred sibcohorts selects for self restraint of paternally derived competitive alleles, while low relatedness of siblings from multiple mated mothers will select for increased levels of competitive effort of paternally derived alleles [36]. Because our model allows for the separation of probability of ibd by parent of origin and because we show that policing is favored by selection between groups, we predict that the high competition of paternally-derived alleles will result in increased policing by maternally derived alleles.
Because genes in separate species are not closely related, policing has been invoked in this way: "the study of interspecific mutualistic associations offers the opportunity to explore the mechanisms ...that maintain cooperative behavior even in the absence of kin selection" ( [37] p. 254; see also [13] p. 520). This view of policing has motivated the use of non-genetic, economic-optimum models of co-evolution to understand the evolution of symbioses. The evolution of conflict reduction without relatedness appears to provide an important alternative explanation for major transitions in evolution. In this view, policing (sometimes called sanctioning) evolves under very low relatedness and is an alternative to kin selection, which also favors decreased competitive ability. We have shown that policing is not an alternative to kin-group selection, thus, although policing could be an important force in the evolution of symbioses [38], especially in cases of coinheritance [7,35,39], it does not offer a new class of evolutionary explanation.
While the population genetic models of policing and competition presented in this paper highlight both the dynamics of the evolution and the conflicting forces of selection within and between groups, Frank's models are more analytically tractable and both models make the same predictions about the long term evolutionary outcome of natural selection. Thus, these complementary approaches provided a clearer view of the evolutionary forces involved in competition and policing behaviors.

Authors' contributions
YB and MJW conceived of the research and co-wrote the paper. YB conducted the theoretical research, and produced the figures and tables.