 Research Article
 Open Access
 Published:
Inbreeding depression under mixed outcrossing, selffertilization and sibmating
BMC Evolutionary Biology volume 16, Article number: 105 (2016)
Abstract
Background
Biparental inbreeding, mating between two relatives, occurs at a low frequency in many natural plant populations, which also often have substantial rates of selffertilization. Although biparental inbreeding is likely to influence the dynamics of inbreeding depression and the evolution of selfing rates, it has received limited theoretical attention in comparison to selfing. The only previous model suggested that biparental inbreeding can favour the maintenance of stable intermediate selfing rates, but made unrealistic assumptions about the genetic basis of inbreeding depression. Here we extend a genetic model of inbreeding depression, describing nearly recessive lethal mutations at a very large number of loci, to incorporate sibmating. We also include a constant component of inbreeding depression modelling the effects of mildly deleterious, nearly additive alleles. We analyze how observed rates of sibmating influence the mean number of heterozygous lethals alleles and inbreeding depression in a population reproducing by a mixture of selffertilization, sibmating and outcrossing. We finally use the ensuing relationship between equilibrium inbreeding depression and population selfing rate to infer the evolutionarily stable selfing rates expected under such a mixed mating system.
Results
We show that for a given rate of inbreeding, sibmating is more efficient at purging inbreeding depression than selfing, because homozygosity of lethals increases more gradually through sibmating than through selfing. Because sibmating promotes the purging of inbreeding depression and the evolution of selfing, our genetic model of inbreeding depression also predicts that sibmating is unlikely to maintain stable intermediate selfing rates.
Conclusions
Our results imply that even low rates of sibmating affect plant mating system evolution, by facilitating the evolution of selfing via more efficient purging of inbreeding depression. Alternative mechanisms, such as pollination ecology, are necessary to explain stable mixed selfing and outcrossing.
Background
Inbreeding plays a central role in the evolution of many plant and animal populations, affecting for example effective population size [32], the speed of adaptation [15], or the accumulation of deleterious mutations and the resulting inbreeding depression [27, 33]. Much attention so far has focused on selffertilization, the most extreme form of inbreeding, which is widespread in plant populations [16], and also occurs in some hermaphroditic animals [21]. However, biparental inbreeding (BI), or mating between two relatives, is also likely to occur frequently in natural populations due to small local population size [46, 48], limited dispersal causing finescale genetic structure [26, 47, 49] or social structure in animals [41]. After Ritland [38] developed a relatively simple method to estimate BI using genotypes for molecular markers in progeny arrays, by comparing multilocus vs. singlelocus estimates of outcrossing rates, large datasets on BI have accumulated in plant populations. Since the first analysis of such data [9] BI has been demonstrated to occur frequently, albeit at low rates.
In selfcompatible hermaphroditic plant populations, BI acts along with selfing to diminish the mean number of recessive deleterious mutations and the inbreeding depression maintained by a balance between mutation and selection, as it increases the inbreeding coefficient of the population, exposing recessive deleterious mutations to selection in homozygous form. Several experimental observations confirm that BI can contribute to purging inbreeding depression [6, 19, 22, 39, 44]. The only previous theoretical treatment of the joint influence of selfing and BI on deleterious mutations [42] showed that such purging by BI could facilitate the evolution of increased selfing rates. Uyenoyama [42] also suggested that BI could promote the maintenance of stable mixed mating systems with intermediate selfing rates when inbreeding depression is low (<0.5), which would provide a general answer to the “enigma" of mixed mating systems [16]. However, this prediction may strongly depend on a number of simplifying assumptions regarding inbreeding depression, its genetic basis and its expression in individuals produced by BI. First, when inbreeding depression was allowed to evolve with the mating system in [42], it was modelled assuming completely recessive mildly deleterious mutations. However, the distribution of fitness effects of mutations is known to be strongly bimodal (e.g. [10]), such that inbreeding depression is often caused by a combination of nearly recessive highly deleterious (lethal and semilethal) mutations and moderately recessive or nearly additive mildly deleterious mutations (reviewed in [5]). These two classes of mutations exhibit contrasting responses to natural selection in an inbred population [3]. Second, stable mixed mating systems in Uyenoyama’s model were only studied assuming zero inbreeding depression in offspring produced by BI and constant inbreeding depression associated with selfing, neither of which is biologically plausible based on the available empirical data (see above and [45]).
Here, we develop a more realistic genetic model to analyze how sibmating, a relatively common form of BI, influences inbreeding depression in a population reproducing by a mixture of selffertilization and outcrossing, a necessary first step to make predictions on the evolution of selfing rates. Our model differs from [42] in the genetic basis of inbreeding depression and includes the two aforementioned components. The component of inbreeding depression due to moderately recessive mildly deleterious mutations is unlikely to be purged much by inbreeding because exposure to selection depends little on inbreeding for nearly additive mutations. It was modelled as a constant, background component of inbreeding depression (as in [33]). The component of inbreeding depression due to highly deleterious mutations was modelled using the approach of Kondrashov [25] to describe evolution of deleterious mutations at a very large number of unlinked loci. We employed a simplified version of the Kondrashov model for nearly recessive lethal mutations [28]. The Kondrashov model accounts for zygotic disequilibrium (nonrandom associations of diploid genotypes among loci) caused by a mixture of selfing and outcrossing. These models assume infinite population size and individually rare deleterious mutations, which can become homozygous only through selfing; in their original form, they therefore cannot deal with sibmating. We extend the Kondrashov model for nearly recessive lethals [28] by tracking three consecutive generations (grandparents, parents and offspring) to incorporate sibmating.
This model allows us to study the effect of realistic levels of sibmating on the equilibrium inbreeding depression in plant populations reproducing by a mixture of selfing and outcrossing. The resulting relationship between inbreeding depression and the population selfing rate is then used to infer the evolutionarily stable selfing rates expected in a population practising sibmating, when inbreeding depression is allowed to evolve with the mating system. This is the second major difference from the approach of Uyenoyama [42], who assumed constant inbreeding depression upon selfing, independent of the selfing rate in the population. We show below that for a given rate of inbreeding sibmating is more efficient than selfing in purging inbreeding depression; sibmating is therefore expected to promote the evolution of selfing and is unlikely to maintain stable intermediate selfing rates.
Results and discussion
In a population reproducing by a mixture of selffertilization, outcrossing between unrelated individuals and sibmating, the model calculates the mean number of nearly recessive heteroygous lethals, with dominance coefficient h=0.02, maintained per diploid genome at mutationselection equilibrium under a genomic mutation to lethals U. In the following, \(\bar {s}\) is the population primary selfing rate, at fertilization. The total inbreeding depression δ is defined as the decrease in mean fitness of selfed offspring vs. those produced by outcrossing between unrelated parents. The distribution of heterozygous lethals, combined with a constant (nonevolving) background inbreeding depression d=0.25 for selfing, produces the total inbreeding depression upon selfing. Individuals produced by different types of mating are denoted using subscripts without brackets: o, s, oo, os, and ss, respectively indicating outcrossing to an unrelated individual, selffertilization, and sibmating involving two outcrossed parents, one outcrossed and one selfed parent, or two selfed parents (Fig. 1). For more details, see Methods.
Rates of BI in natural plant populations
We first tested our assumption of a low frequency of BI (below 0.1), which was required to neglect the probability of BI occurring in two successive generations in any given lineage in the population. We merged two existing databases compiling multilocus estimates of outcrossing rates (t _{ m }) in natural plant populations [9, 16], retaining only species for which both multilocus and mean singlelocus (t _{ s }) estimates were available (276 species), and we estimated the frequency of BI from t _{ m }−t _{ s } [38]. As expected, BI was generally rare in natural plant populations (Fig. 2), with an average value of 0.033 (90 % range [−0.038,0.144]). Less than 10 % of species have estimated rates of BI above 0.1, and many of these are likely due to sampling error, as obviously are all the negative estimates; the highest estimate of 0.218 [30] is from an agricultural population (seed orchard).
Influence of sibmating on mean number of lethals and total inbreeding depression
In a mixedmating population without sibmating, the mean number of deleterious mutations at equilibrium, and hence inbreeding depression, decreases with increased selfing rate (Fig. 3) due to increased homozygosity exposing recessive deleterious mutations to selection [27]. The decrease in mean lethals and inbreeding depression is relatively smooth under small or moderate genomic mutation rates to lethals (U=0.02 and U=0.2, Fig. 3 ad). At a high genomic mutation rate to lethals (U=1, Fig. 3 ef) there appears a fairly sharp purging threshold due to zygotic disequilibria and selective interference among lethals causing the secondary selfing rate to remain close to zero for primary selfing rates below the purging threshold [28]. The shape of the relationship between population selfing rate and the mean number of heterozygous lethals or total inbreeding depression at equilibrium remains qualitatively the same in the absence of background inbreeding depression (cf. Figure 5 in Appendix). Background inbreeding depression increases total inbreeding depression at equilibrium, and shifts the threshold selfing rates for purging recessive lethals towards higher values.
With sibmating, the general pattern of decreased equilibrium number of lethals and inbreeding depression with increased selfing rate remains the same, but sibmating can have a significant effect on the inbreeding depression and the purging threshold (Fig. 3). In a completely outcrossing population, sibmating decreases the mean number of lethals to the greatest extent for small to moderate genomic mutation rates to lethals, U; in such populations with U=0.02, 0.2 or 1 the proportional decrease in the equilibrium number of heterozygous lethals caused by 10 % sibmating is 0.34, 0.27 or 0.05 respectively. This pattern remains qualitatively the same without background inbreeding depression (Figure 5 in Appendix).
For a given rate of inbreeding, sibmating is more efficient than selfing at purging inbreeding depression. This can be illustrated by comparing populations with identical expected inbreeding coefficient prior to selection, for example an outcrossing population with b=10 % halfsib (oo) mating vs. a predominantly outcrossing population with a selfing rate of 0.025 (without sibmating); the equilibrium number of heterozygous lethals is always lower in the population with sibmating than in the partly selfing population, and the difference increases at higher genomic mutation rates to lethals (0.65 vs. 0.70, 7.11 vs. 9.3 or 46.8 vs. 49 respectively for U=0.02, 0.2 or 1, Fig. 3 a, c & e).
For the same rate of inbreeding in a population (as in the foregoing example), sibmating is more efficient than selfing at purging nearly recessive lethal mutations because the probability of producing homozygous lethals in sibmating is smaller than under selfing, which reduces selective interference among loci [28]. This is reflected in the mean fitnesses of the different types of progeny. Figure 4 shows that the progeny of all sibmating types generally have mean fitnesses intermediate between the mean fitness of selfed and outcrossed individuals. The only exceptions are for the types of mating os and ss under a high genomic mutation rate to lethals (U=1, Fig. 4 c) at selfing rates near the purging threshold (see below for an explanation).
Mean fitness at equilibrium also varies among types of sibmating. In general the more inbred offspring have lower mean fitness, due mostly to the constant background component of inbreeding depression. However, with a high genomic mutation rate to lethals the difference among different kinds of sibmating is reduced, and sometimes reversed (see Fig. 4 c, where inbred individuals with two selfed parents have the highest mean fitness at selfing rates below the purging threshold). Without background inbreeding depression, this becomes the rule: the most inbred sibmating types always have the highest mean fitness (cf. Figure 6 in Appendix). Because selfed parents have purged a fraction of their deleterious mutations, the mean fitness of individuals produced by sibmating is then higher when their parents are selfed vs. outcrossed. The difference is particularly large under a high genomic mutation rate to lethals (U=1, Fig. 4 c) at selfing rates near the purging threshold (\(\bar {s}=0.38\)), where biparentally inbred individuals with at least one selfed parent have higher mean fitness than outcrossed individuals, despite being inbred. Near the purging threshold, an appreciable fraction of selfed zygotes survive (i.e. the secondary selfing rate moves away from zero), and purging occurs in a subset of the population. In this situation, the difference in the number of lethals between outcrossed individuals and surviving selfed (purged) individuals is such that the overall heterozygous effects of lethals in outcrosses between unrelated individuals exceeds the (homozygous) lethal effects in os and ss sibmatings.
With respect to purging recessive lethal mutations, the results of our model of inbreeding depression in mixedmating populations practising a combination of outcrossing to unrelated individuals, selfing, and sibmating are consistent with those of [42], although this earlier study was based on a simpler, less realistic model for inbreeding depression. In both models, BI favours the purging of recessive deleterious mutations in predominantly outcrossing populations, and may therefore facilitate the evolution of selfing. In contrast to Uyenoyama’s model, we incorporated zygotic disequilibrium and selective interference among loci, which are magnified under high inbreeding depression. We show that with high genomic mutation rate to recessive lethals, sibmating lowers the threshold for purging recessive lethal mutations, thereby potentially favouring evolution of high selfing rates.
Could the lower purging threshold caused by sibmating be observed in experimental data? Winn et al. [45] observed a purging threshold for total inbreeding depression, with substantial inbreeding depression maintained for selfing rates up to 0.8, but the amongspecies variation in the rate of BI (Fig. 1 and Duminil et al. 2009) may be too small to allow empirical tests of whether BI affects the purging threshold for recessive lethals.
Evolutionarily stable selfing rates in a population practicing sibmating
Following Lande and Schemske [27], who showed that complete outcrossing and complete selfing were the only two evolutionarily stable equilibria maintained by the two main genetic forces driving the evolution of selfing (Fisher’s automatic advantage of selfing and inbreeding depression), numerous models were proposed to explain the maintenance of stable mixed mating (reviewed in [16]). Uyenoyama [42] suggested that BI could be one mechanism maintaining stable intermediate selfing rates under moderate inbreeding depression by generating frequencydependent selection. At higher selfing rates, the automatic advantage of selfing decreases and BI further decreases the automatic advantage of selfing via greater transmission of the genome in the BI portion of the outcrossed matings. However, Uyenoyama [42] assumed no inbreeding depression associated with BI but constant inbreeding depression associated with selfing, to observe stable mixed mating.
We can use the analytical approximation of Lande and Schemske [27] to predict the evolutionarily stable selfing rates expected in our more realistic model that includes inbreeding depression for progeny of sibmating and that allows inbreeding depression to evolve as a function of the mating system. The approximation states that there is selection for increased selfing if the inbreeding depression is less than 0.5 (the automatic advantage of selfing), and vice versa. Here, the approximation predicts that the only evolutionarily stable selfing rates are complete outcrossing (δ>0.5, Fig. 3 d, f) or complete selfing (δ<0.5, Fig. 3 b, d). Porcher and Lande [36] showed that this approximation predicts evolutionarily stable selfing rates accurately with a low to moderate genomic mutation rate to lethals (U=0.02−0.2). A full analysis of evolution of selfing is required under high mutation rate to lethals (U=1), when the approximation does not apply. However, in general, allowing purging of inbreeding depression generates a strong positive feedback on the evolution of selfing rates, which can outweigh other mechanisms that promote stable intermediate selfing rates in the absence of purging (e.g. [35] for an example with fluctuating inbreeding depression).
Scope and limitations of the model
To keep the model tractable, we have assumed that the component of inbreeding depression due to nearly additive, mildly deleterious mutations could not be purged. Combining two components of inbreeding depression represents a significant improvement over most theoretical studies about mating system evolution, which generally assume constant total inbreeding depression, inbreeding depression caused by a single locus, or at best model a single component. Assuming a constant (nonevolving) background inbreeding depression is a reasonable first approximation. The contributions of mutations to inbreeding depression and their sensitivity to purging upon selfing depend primarily on their deleterious effect and dominance coefficient [4]. Furthermore, in predominantly outcrossing species, the total inbreeding depression resulting from the classical bimodal distribution of inbred fitness is composed of comparable amounts due to nearly recessive lethal and semilethal mutations versus more nearly additive mildly deleterious mutations [8, 20]. For these reasons, nearlyrecessive lethals are expected to be most readily purged by selection with inbreeding, and to play the most important role in the coevolution of inbreeding depression and selfing rate. Data from natural populations show that earlyacting recessive lethals contribute a substantial fraction of inbreeding depression in outcrossing species but are mostly purged in highly selfing species [20]. In contrast, the component of inbreeding depression due to lateacting more nearly additive mildly deleterious mutations remains nearly constant across population selfing rates [20].
Estimates of the mean dominance coefficient of mildly deleterious mutations in natural populations are generally in the range of 0.2<h<0.5 (e.g. [17]) and theoretical analyses show that for such mutations the contribution to inbreeding depression depends only weakly on the selfing rate [4, 36]. However, a small fraction of mildly deleterious mutations may have smaller dominance coefficients and thereby be liable to purging. In addition, we have shown that slow inbreeding, as produced here by sibmating, is more efficient than selfing for purging nearly recessive lethals. This may also be true for mildly deleterious mutations [12]. A full model of inbreeding depression, including all types of unconditionally deleterious mutations (with a range of selection and dominance coefficients), as well as inbreeding depression due to stabilizing selection on quantitative characters [29], is therefore needed for a comprehensive understanding of the joint dynamics of inbreeding depression and mating systems. The net effect of purging mildly deleterious alleles should nevertheless remain small compared to the purging of nearlyrecessive, highly deleterious mutations. Incorporating the purging of mildly deleterious mutations therefore should not alter the main conclusion of our model, which is that sibmating contributes to purging inbreeding depression, thereby reinforcing the positive feedback on the evolution of selfing rate. In other words, BI, as modelled here by sibmating, cannot produce a stable mixedmating system.
Our model also assumes infinite population size and hence cannot address the effect of sibmating caused by small population size alone. In small populations, demography and drift may interact with mutation, selection and the mating system to alter the dynamics of inbreeding depression and the purging process. For example, fixation of mildly deleterious mutations decreases inbreeding depression [1]. Yet our model should be valid for other mechanisms causing BI in large populations, such as geographic structuring by limited dispersal of seed and pollen (isolation by distance), which is likely to be common in plants [43]. The main result of our model, that sibmating promotes increased selfing via purging of inbreeding depression, requires that selfing efficiently purges recessive lethal mutations in finite populations. The few theoretical studies examining the joint effects of selfing rates and population size on the dynamics of inbreeding depression all show that for populations of 100 or more individuals, purging by selffertilization occurs, particularly with highly deleterious mutations with a small dominance coefficient, such as recessive lethals (e.g. [14] for an analytical treatment with a single locus or [4] for individualbased simulations with a Kondrashovlike model). Thus the predictions from infinite population models are consistent with those from simulations of finite populations (compare e.g. [34] vs. [13] for the evolution of selfincompatibility).
Conclusions
Using a realistic model of inbreeding depression incorporating the bimodal distribution of deleterious mutations and the joint evolution of mating system and inbreeding depression, we have shown that sibmating promotes the purging of inbreeding depression, but is unlikely to maintain stable intermediate selfing rates. These results, combined with previous work, suggest that genetic mechanisms alone are therefore unlikely to provide general explanations for the maintenance of mixed mating. Factors involved in pollination ecology, including pollen discounting, and pollinator behaviour, interacting with genetic mechanisms of evolving inbreeding depression, provide a more general explanation of the widespread maintenance of intermediate selfing rates in plant populations (e.g. [7, 33, 36]).
Methods
Model assumptions and notation
The assumptions are the same as for the simplified [25] model of mixed selfing and random mating for partially recessive lethals developed in [28], except that we want to allow other forms of inbreeding in addition to selfing. The different types of mating are referred to using subscripts without brackets: o, s, oo, os, and ss describe respectively individuals produced by outcrossing to an unrelated individual, by selffertilization, and by sibmating involving two outcrossed parents, one outcrossed and one selfed parent, or two selfed parents (Fig. 1). In the simplified Kondrashov model, inbreeding depression is due to nearly recessive lethal mutations occurring at an infinite number of unlinked loci in an infinite population. The genomic mutation rate to lethals is U and their dominance coefficient h. Lethal mutations are never homozygous in adults, segregate independently and have identical effects on fitness, such that a diploid population with discrete nonoverlapping generations is fully described by the distribution of number of heterozygous lethals per diploid genome in adults. The frequency of adults in generation {t} carrying x heterozygous lethal mutations is p _{{t}}(x). The life cycle begins with reproduction and is followed by mutation, then selection.
The distribution of heterozygous lethals causes inbreeding depression upon selfing, which is defined as the decrease in mean fitness of selfed offspring \(\bar {w}_{s}\) relative to those produced by outcrossing between unrelated parents \(\bar {w}_{o}\) so that \(\delta = 1\bar {w}_{s}/\bar {w}_{o}\). We included a constant (nonevolving) component of inbreeding depression due to mildly deleterious nearly additive mutations via a background inbreeding depression d=0.25 for selfing, as in [33]. In the present model the background inbreeding depression reduces the mean fitness of the different types of mating in proportion to their (neutral) inbreeding coefficients (Eq. 17), F _{ o }=0, \(F_{s}=\frac {1}{2}\), \(F_{oo}=\frac {1}{8}\), \(F_{os}=\frac {1}{4}\) and \(F_{ss}=\frac {1}{2}\).
For simplicity, a low frequency of sibmating below 0.1 is assumed, embedded in a background pedigree of mixed mating (selfing and outcrossing), which is consistent with empirical observations. Hence we can neglect the probability of sibmating occurring in two successive generations in any given lineage in the population. Finally, we also assume that sibmating occurs only between individuals from the same maternal family, because the most likely cause of sibmating in many plant populations is finescale genetic structure, most of which is attributable to limited seed dispersal in comparison to pollen flow [31]. This excludes any contributions from fullsib mating, except when the maternal parent of two sibs is selfed (sibmating type ss).
We first describe the production of juvenile heterozygous lethal genotypes in generation {t} after mutation but before selection on heterozygotes produced by each type of mating. We then complete the life cycle by appropriately weighting the different types of matings and performing the operations of selection on heterozygous viability and normalization of genotype frequencies in the adults.
Random mating
A proportion of zygotes are produced by random mating (outcrossing to an unrelated individual) as in the Kondrashov model for partially recessive lethal mutations [28].
In the parental generation {t−1} the probability that a mature plant with y heterozygous lethals produces a gamete with x (≤y) lethals (before mutation) is \(\binom {y}{x}(1/2)^{y} \). Thus in the entire parental population the probability of producing a gamete with x lethals (before mutation) is
With an infinite number of loci every new mutation is unique and outcrossing to an unrelated individual in an infinite population never produces homozygosity of lethal alleles. The probability that random mating produces a zygote with x heterozygous lethals (before mutation) is then
and after mutation (but before selection on heterozygotes) the fraction of randomly outcrossed juvenile offspring with x heterozygous lethals is
Selffertilization
A proportion of individuals are produced by selffertilization, as in [28]. In the parental generation {t−1} the probability that a mature plant with y heterozygous lethals produces by selffertilization a seed with x (≤y) heterozygous lethals (before mutation) is \(\binom {y}{x}(1/2)^{x} (1/4)^{yx} = \binom {y}{x}(1/2)^{2yx} \). Thus in the entire parental population the probability of producing a juvenile offspring with x lethals (before mutation) is
After mutation (but before selection on heterozygotes) the fraction of outcrossed juvenile offspring in generation {t} with x heterozygous lethals is
Outcrossing between relatives
Transmission of lethals from grandparents to offspring
To derive the number of homozygous lethals produced by sibmating we extend the Kondrashov model to describe the transmission of mutations from grandparents to offspring. Depending on the type of sibmating (Fig. 1), we distinguish four types of grandparents (from single to quadruple), differing in the number of genome copies they contribute to the parental and offspring generation. Matings in the grandparental generation {t−2} that produce halfsibs in generation {t−1} cause no homozygosity until the offspring generation {t} and therefore have the same distribution of number of heterozygous lethals in the parental generation {t−1} as under mixed outcrossing and selfing (o and s). The probabilities \(g^{{~}^{\prime }}(x)\), \(g^{{~}^{\prime \prime }}(x)\), \(g^{{~}^{\prime \prime \prime }}(x)\), and \(g^{{~}^{\prime \prime \prime \prime }}(x)\), that in the entire population at generation t−2, a single, double, triple or quadruple grandparent, respectively, produces a juvenile offspring in generation t with x heterozygous lethals, are derived in the Appendix. These relative probabilities sum to less than 1, either because parents that die due to heterozygous lethals make no contribution to the offspring generation or because offspring die due to homozygous lethals produced by sibmating. Thus these formulae account for selection on heterozygous lethals in parents, and selection on homozygous lethals in offspring, but do not include any selection on heterozygous lethals in offspring, which we deal with afterwards.
Convolution of grandparental contributions
In a halfsib mating involving two outcrossed individuals (oo), three unrelated grandparents may contribute heterozygous lethals to the offspring of each halfsib mating. The three contributions result from independent events involving genetic segregation and recombination. Each grandparent outcrosses to an unrelated individual, with the double grandparent outcrossing to two different unrelated individuals. The total contribution to heterozygous lethals in juvenile offspring from the three grandparents is simply the sum from the three grandparents, so the unnormalized distribution of heterozygous lethals from these three sources is simply the convolution of the distributions from these three sources. This can be expressed in two concatenated convolutions, first convoluting the two single grandparent contributions,
and then convoluting this with the double grandparent distribution to get the unnormalized distribution of total number of heterozygous lethals contributed by the grandparents,
In a sibmating involving one selfed and one outcrossed individuals, only two unrelated grandparents (one single, one triple) may contribute heterozygous lethals to the offspring of each halfsib mating. The distribution of heterozygous lethals in the offspring is therefore a convolution from these two grandparents:
Finally, in a sibmating involving a quadruple grandparent, the distribution of heterozygous lethals in the offspring is given directly by Eq. (21)
New mutations in parents
Now we must account for the extra generation of mutation and selection on the new mutations produced during reproduction by grandparents and passed to juveniles of the parental generation. New mutations expressed in juvenile parents in generation {t−1} are inherited and selected only in heterozygotes and are therefore unaffected by inbreeding (from all grandparental sources). On average, U new mutations arise in each of two parents, a fraction \(\frac {1h}{2}\) of which are transmitted to the offspring, accounting for selection on heterozygous lethals and genetic segregation. Hence, the total mutation rate is U(1−h) and the fitness of each parent is reduced by a factor e ^{−Uh}, regardless of their mutation load. For sibmating type ij (where i j=oo,os or ss)
New mutations in the offspring
For all types of mating, the relative frequency of juvenile offspring produced by halfsib mating containing x heterozygous lethal mutations after mutation (but before selection) is
Normalization to account for selection in parents
The sum over x of \(p_{ij}^{**}(x)\) frequencies calculated above is less than 1, due to selection on homozygous and heterozygous lethals in the parental generation, and selection on homozygous lethals in the offpsring generation. A renormalization is therefore necessary to correct for selection in the parental generation, and combine the frequencies of individuals produced by sibmating (calculated over two generations) with those of outcrossed and selfed individuals (calculated over a single generation). Because crosses do not occur at random, the normalization factor for each type of mating ij is not a simple product of population mean fitnesses, but instead the mean fitness of all possible pairs of sib parents within the type of mating
where q _{ i{t−1}}(x,y) is the probability that a grandparent carrying x heterozygous lethals produces a parent with y heterozygous lethals via outcrossing (i=o) or selfing (i=s). The factor e ^{−2Uh} allows mutation decreasing the fitness of the parental generation. It can be shown that
Using Eqs. (13) in Eq. (12), we obtain the following normalization factors
and we have
Note that the constant background inbreeding depression, d, does not appear in the normalization factors, although it does affect the mean fitness of types of mating involving at least one selfed parent, reducing it by a multiplicative factor (1−d) for each selfed parent. This same multiplicative factor also alters the probability of transmission of lethal alleles from grandparents to offspring detailed in the Appendix, such that they cancel out in the normalization step and are not shown.
Selection and combination of types of mating in generation t
Relative frequencies of the different types of mating
Assuming that the frequencies of different types of sibmatings are not genetically determined in the population, then the relative frequencies of types oo, os and ss sibmatings respectively are proportional to the secondary rates (after selection) of random mating and selfing (respectively \(1\bar {s}^{*}\) and \(\bar {s}^{*}\)). Assuming that the primary selfing rate (at fertilization), \(\bar {s}\), is the principal evolutionary variable, so that if \(\bar {s}\) increases the other rates must decrease proportionally, all other rates are proportional to \(1\bar {s}\). In addition, we assume a metapopulationlike dispersal model, with short distance seed dispersal generating family “islands”, and both short (withinfamily, rate b) and longdistance (between families, rate 1−b) pollen dispersal. Hence, halfsib mating rate is controlled by the constant parameter b, i.e. the probability that a plant is pollinated by local pollen produced by a sib. Each family consists of a fraction \(\bar {s}^{*}\) of selfed adult plants and a fraction \(1\bar {s}^{*}\) of outcrossed adult plants. The relative probabilities of each type of mating are thus

Selfing rate, \(f_{s} = \bar {s}_{\{t\}}\)

Outcrossing to unrelated individual, \(f_{o} = (1\bar {s}_{\{t\}})(1b)\)

Halfsib type oo, \(f_{oo} = \left (1\bar {s}_{\{t\}}\right)b\left (1\bar {s}^{*}_{\{t1\}}\right)^{2}\)

Halfsib type os, \( f_{os} = 2\left (1\bar {s}_{\{t\}}\right)b\bar {s}^{*}_{\{t1\}}\left (1\bar {s}^{*}_{\{t1\}}\right)\)

Fullsib type ss, \( f_{ss} = \left (1\bar {s}_{\{t\}}\right)b\bar {s}^{*2}_{\{t1\}}\)
where \(\bar {s}^{*}_{\{t1\}}=\bar {s}\bar {w}_{s\{t1\}}/\bar {w}_{\{t1\}}\) is the secondary selfing rate in the parental generation (see below for definitions of \(\bar {w}_{s\{t1\}}\) and \(\bar {w}_{\{t1\}}\)). This gives f _{ s }+f _{ oo }+f _{ os }+f _{ ss }+f _{ o }=1. To conform to the assumptions, we must restrict b<<1.
Selection and final step of normalization
The normalization of the total frequency of recessive lethals in mature offspring, incorporating selection on heterozygous lethals and the constant, lateacting component of inbreeding depression due to mildly deleterious alleles, d, is
where the sum includes all the mating types r, with F _{ r } being the neutral inbreeding coefficient of each mating type (given at beginning of Methods) and the mean fitness \(\bar {w}_{\{t\}}=\sum _{r} f_{r}\bar {w}_{r\{t\}}\) is the weighted average of the mean fitnesses of progenies produced by the different matings,
Numerical analysis
The variables are the same as in the Kondrashov model for selfing, but to perform the iterations we must keep track of the last two generations of p(x), instead of just the single previous generation. Numerical iterations start with one generation of mixed selfing and outcrossing, thereafter allowing also a low rate of sibmating.
Parameter values are as follows. The genomic mutation rate to lethals is varied between U=0.02 and U=1, to encompass the limited number of experimental estimates. Data from Drosophila [2] and annual plants [23] indicate genomic mutation rates to lethals on the order of U=0.01−0.03. Few estimates exist for large perennial or partially asexual plants, in which the mutation rate to lethals may be much higher due to accumulation of somatic mutations. For example, Lande et al. [28] extrapolated mutation rates to (lethal) embryonic chlorophyll deficiency to the whole genome to obtain a genomic mutation rate of U=0.2 in a longlived mangrove tree. Similarly, high genomic mutation rates to lethal (U=0.2 and above) are needed to explain inbreeding depression close to 1 in large gymnosperm trees [18, 37]. Although U=1 may exceed the maximum actual genomic mutation rate to lethals, this value illustrates the impact of selective interference in purging partially recessive deleterious mutations caused by high inbreeding depression; selective interference has been demonstrated to occur even with mildly deleterious mutations [24].
The dominance coefficient of lethals is always set to h=0.02 [40]. The rate of sibmating is varied between b=0 and b=0.1, conforming to assumptions and experimental estimates (Fig. 2). Finally, the constant background component of inbreeding depression is set to d=0.25, which is close to the average level of lateacting inbreeding depression that is not purged in highly selfing populations [20].
Inference of evolutionarily stable selfing rates
To examine the effect of sibmating on the evolution of selfing rates, we used an approximation first proposed by Lande and Schemske [27] and later generalized by Porcher and Lande [36] to include ecological mechanisms. This approximation assumes that the mating system evolves by small infrequent mutational steps and allows the equilibrium inbreeding depression to evolve as a function of the mating system. It can be used to find the joint equilibria of selfing rate and inbreeding depression by examining the indirect selection gradient on small changes in the selfing rate. Equilibrium selfing rates occur when the selection gradient is 0, when total inbreeding depression exactly counterbalances all other constraints on the evolution of selfing. With selfing and outcrossing alone, the other constraint is the automatic genetic advantage of inbreeding: 50 % for selfing [11], so that an unstable evolutionary equilibrium of the selfing rate occurs at δ=0.5. The inclusion of sibmating diminishes the automatic advantage of selfing by increasing gene transmission in other types of mating and also modifies the total inbreeding depression. Because we assume a small rate of sibmating it does not qualitatively change the stability of the unstable evolutionary equilibrium for the selfing rate, and it produces only a small change in the position of the unstable equilibrium (see Appendix).
Appendix
Transmission of lethal alleles from grandparents to offspring
Single grandparent
For each heterozygous lethal allele in a single grandparent the relative probability that the juvenile offspring of halfsib matings inherits a heterozygous lethal from them is \(\frac {1h}{4}\). The relative probability that the juvenile offspring does not inherit a heterozygous lethal from one of its single grandparents is \(\frac {1}{2} +\frac {1h}{4} = \frac {3h}{4}\). Hence, if the single grandparent carries y heterozygous lethals, the relative probability of the juvenile offspring of halfsib mating inheriting x of these in heterozygous form is \(\binom {y}{x}\left (\frac {1h}{4}\right)^{x} \left (\frac {3h}{4}\right)^{yx}\). Averaging over all possible single grandparents the relative probability that a single grandparent contributes x heterozygous lethals to the juvenile offspring of halfsib mating is
where the subscript {t−2} refers to the grandparental generation.
Double grandparent
For each lethal heterozygote in a double grandparent, the juvenile offspring of halfsib mating is heterozygous for the lethal allele with relative probability \(2\left (\frac {1h}{4}\right)\left (\frac {3h}{4}\right)\), and is homozygous for nonlethal allele(s) with relative probability \(\left (\frac {3h}{4}\right)^{2}\). Thus, given a double grandparent with y heterozygous lethals, the relative probability that the juvenile offspring of a halfsib mating inherits x (≤y) heterozygous lethals from the double grandparent is \(\binom {y}{x}\left (\frac {1h}{2}\right)^{x} \left (\frac {3h}{4}\right)^{2yx}\). Averaging over all common grandparental contributions to halfsib matings, the unnormalized distribution of number of heterozygous lethals inherited from double grandparents is
Triple grandparent
Given a triple grandparent in generation {t−2} with a heterozygous lethal allele at a particular locus, when producing an outcrossed (halfsib) parent in generation {t−1}, the parent is heterozygous for this allele with probability \(\frac {1h}{2}\), and does not contain this allele with probability \(\frac {1}{2}\). Similarly, the relative probability that a selfed (halfsib) parent in generation {t−1} is heterozygous for this allele is \(2\left (\frac {1h}{2}\right)\frac {1}{2} =\frac {1h}{2}\), and the relative probability that the selfed parent does not contain this allele is \(\frac {1}{4}\).
Pairing one selfed and one outcrossed halfsib parents, the relative probability that in generation {t} the juvenile offspring of type os halfsib mating carry a heterozygous lethal at this locus is therefore \( P=\frac {2h}{4}\frac {1h}{4}+\frac {3h}{4}\frac {1h}{4} = \frac {(1h)(52h)}{16}. \) Similarly, the relative probability that the juvenile offspring in generation {t} lack the lethal allele is \(Q= \frac {(2h)(3h)}{16}.\) The relative probability that a triple grandparent containing y heterozygous lethals produces juvenile offspring with x (≤y) heterozygous lethals is \(\binom {y}{x}P^{x} Q^{yx}\). Therefore in the entire population in generation {t−2} the probability that a triple grandparent produces a juvenile offspring in generation {t} with x heterozygous lethals (before mutation) is
Quadruple grandparent
Given one quadruple grandparent in generation {t−2} with a heterozygous lethal allele at a particular locus, the relative probability that a selfed (halfsib) parent in generation {t−1} is heterozygous for this allele is \(\frac {1h}{2}\), and the relative probability that the selfed parent does not contain this allele is \(\frac {1}{4}\).
Thus the relative probability that in generation {t} the juvenile offspring of type ss fullsib mating carry a heterozygous lethal at this locus is \( P'=2\frac {2h}{4}\frac {1h}{4}=\frac {(1h)(2h)}{8} \) and the relative probability that the juvenile offspring in generation {t} lack the lethal allele is \( Q'= \frac {(2h)^{2}}{16}. \) The relative probability that a quadruple grandparent containing y heterozygous lethals produces juvenile offspring with x (≤y) heterozygous lethals is \(\binom {y}{x}P'^{x} Q'^{yx}\). Therefore in the entire population in generation {t−2} the probability that a quadruple grandparent produces a juvenile offspring in generation {t} with x heterozygous lethals (before mutation) is
Approximation for evolution of selfing under low rates of sibmating
The approximation assumes an infinite population with primary selfing rate \(\bar {s}\) in which an initially rare modifier with selfing rate s appears [27]. Assuming that all types of mating produce the same amount of pollen, and that all ovules are fertilized, the expected fitness of individuals with the selfing rate s, without sibmating, is:
Selfed offspring are weighted twice as much as outcrossed offspring to incorporate the automatic genetic advantage of selfing. The intensity of selection on a modifier with a small effect on the selfing rate is approximately proportional to the selection gradient:
Evolutionary equilibrium selfing rates occur when the selection gradient is zero, which, without sibmating, amounts to comparing inbreeding depression with the automatic advantage of selfing. Because the equilibrium inbreeding depression δ is a decreasing function of the population selfing rate, provided that δ>1/2 in an outcrossing population and δ<1/2 in a completely selfing population, an unstable evolutionary equilibrium selfing rate occurs at an intermediate value, \(0<\bar {s}<1\) (see Fig. 3).
With sibmating, the weighting of outcrossed vs. selfed offspring changes, because individuals reproducing via BI transmit more than one copy of their genome. The fraction of the genome of an individual mating with a relative that is found in its offspring is 5/8, 11/16 and 7/8, respectively for sibmating types oo, os and ss, respectively (vs. 1/2 for outcrossing between unrelated individuals and 1 for selfing). Therefore,the expected fitness of individuals with the selfing rate s becomes:
where \(\bar {s}^{*}\) is the secondary selfing rate after selection in the population and \(P(\bar {s},\bar {s}^{*})\) is the siring success of individuals with selfing rate s which depends solely on the population selfing rate (\(\bar {s}\) and \(\bar {s}^{*}\)) and disappears when fitness is differentiated with respect to s. The selection gradient is:
This can be rewritten :
where δ _{ oo }, δ _{ os } and δ _{ ss } are the inbreeding depressions of sibmating types oo, os and ss, defined as 1 minus the ratio of their mean fitness over the mean fitness of offspring produced by outcrossing between unrelated parents. The two parts of the right side can be compared with those of Eq. (23). The first part (first two terms) corresponds to the automatic transmission advantage of selfing, now decreased by sibmating; the increased transmission of genes by sibmating favours outcrossing genes, which are more concentrated in sibmatings than in selfed matings. The second part (third term) is the total inbreeding depression in the population, modified by sibmating, in a complicated way because sibmating also contributes to the purging of lethals. With rates of sibmating below 10 %, all effects remain small, on the order of a few percent (compare Figure 7 in Appendix with Fig. 3 d). Most importantly, incorporating sibmating only displaces slightly the location of the unstable equilibrium selfing rate, when it exists, but never creates intermediate stable selfing rates, because total inbreeding depression remains a decreasing function of the primary selfing rate, always decreasing faster than the automatic advantage.
References
 1
Bataillon T, Kirkpatrick M. Inbreeding depression due to mildly deleterious mutations in finite populations: size does matter. Genet Res. 2000; 75(1):75–81.
 2
Charlesworth B, Borthwick H, Bartolome C, Pignatelli P. Estimates of the genomic mutation rate for detrimental alleles in Drosophila melanogaster. Genetics. 2004; 167(2):815–26.
 3
Charlesworth D, Morgan MT, Charlesworth B. Inbreeding depression, genetic load, and the evolution of outcrossing rates in a multilocus system with no linkage. Evolution. 1990; 44(6):1469–89.
 4
Charlesworth D, Morgan M, Charlesworth B. Mutation accumulation in finite outbreeding and inbreeding populations. Genet Res. 1993; 61(1):39–56.
 5
Charlesworth D, Willis JH. The genetics of inbreeding depression. Nat Rev Genet. 2009; 10(11):783–96.
 6
Collin CL, Penet L, Shykoff JA. Early inbreeding depression in the sexually polymorphic plant Dianthus sylvestris (Caryophyllaceae): effects of selfing and biparental inbreeding among sex morphs. Am J Bot. 2009; 96(12):2279–87.
 7
Devaux C, Lande R, Porcher E. Pollination ecology and inbreeding depression control individual flowering phenologies and mixed mating. Evolution. 2014; 68(11):3051–65.
 8
Dobzhansky T. Genetics of the evolutionary process. New York: Columbia University Press; 1970.
 9
Duminil J, Hardy OJ, Petit RJ. Plant traits correlated with generation time directly affect inbreeding depression and mating system and indirectly genetic structure. BMC Evol Biol. 2009; 9:177.
 10
EyreWalker A, Keightley PD. The distribution of fitness effects of new mutations. Nat Rev Genet. 2007; 8(8):610–8.
 11
Fisher RA. Average excess and average effect of a gene substitution. Ann Eugenics. 1941; 11:53–63.
 12
GarciaDorado A. On the consequences of ignoring purging on genetic recommendations for minimum viable population rules. Heredity. 2015; 115(3):185–7.
 13
Gervais C, Abu Awad D, Roze D, Castric V, Billiard S. Genetic architecture of inbreeding depression and the maintenance of gametophytic selfincompatibility. Evolution. 2014; 68(11):3317–24.
 14
Glemin S. How are deleterious mutations purged? Drift versus nonrandom mating. Evolution. 2003; 57(12):2678–87.
 15
Glemin S, Ronfort J. Adaptation and maladaptation in selfing and outcrossing species: new mutations versus standing variation. Evolution. 2013; 67(1):225–40.
 16
Goodwillie C, Kalisz S, Eckert CG. The evolutionary enigma of mixed mating systems in plants: occurrence, theoretical explanations, and empirical evidence. Ann Rev Ecol Evol Syst. 2005; 36:47–79.
 17
Halligan DL, Keightley PD. Spontaneous mutation accumulation studies in evolutionary genetics. Ann Rev Ecol Evol Syst. 2009; 40:151–72.
 18
Hedrick PW, Savolainen O, Karkkainen K. Factors influencing the extent of inbreeding depression: an example from Scots pine. Heredity. 1999; 82:441–50.
 19
Hirao AS. Kinship between parents reduces offspring fitness in a natural population of Rhododendron brachycarpum. Ann Bot. 2010; 105(4):637–46.
 20
Husband BC, Schemske DW. Evolution of the magnitude and timing of inbreeding depression in plants. Evolution. 1996; 50(1):54–70.
 21
Jarne P, Auld JR. Animals mix it up too: the distribution of selffertilization among hermaphroditic animals. Evolution. 2006; 60(9):1816–24.
 22
Jones FA, Comita LS. Neighbourhood density and genetic relatedness interact to determine fruit set and abortion rates in a continuous tropical tree population. Proc R Soc BBiological Sci. 2008; 275(1652):2759–67.
 23
Jürgens G, Mayer U, Torres Ruiz RA, Berleth T, Miséra S. Genetic analysis of pattern formation in the Arabidopsis embryo. Dev Suppl. 1991; 1:27–38.
 24
Kelly JK. Mutationselection balance in mixed mating populations. J Theor Biol. 2007; 246(2):355–65.
 25
Kondrashov AS. Deleterious mutations as an evolutionary factor. 2. Facultative apomixis and selfing. Genetics. 1985; 111(3):635–53.
 26
Lande R. Isolation by distance in a quantitative trait. Genetics. 1991; 128(2):443–52.
 27
Lande R, Schemske DW. The evolution of selffertilization and inbreeding depression in plants. 1. Genetic models. Evolution. 1985; 39(1):24–40.
 28
Lande R, Schemske DW, Schultz ST. High inbreeding depression, selective interference among loci, and the threshold selfing rate for purging recessive lethal mutations. Evolution. 1994; 48(4):965–78.
 29
Lande R, Porcher E. Maintenance of quantitative genetic variance under partial selffertilization, with implications for evolution of selfing. Genetics. 2015; 200(3):891–906.
 30
Muluvi GM, Sprent JI, Odee D, Powell W. Estimates of outcrossing rates in Moringa oleifera using amplified fragment length polymorphism (AFLP). Afr J Biotechnol. 2004; 3(2):145–51.
 31
Petit RJ, Duminil J, Fineschi S, Hampe A, Salvini D, Vendramin GG. Comparative organization of chloroplast, mitochondrial and nuclear diversity in plant populations. Mol Ecol. 2005; 14(3):689–701.
 32
Pollak E. On the theory of partially inbreeding finite populations. 1. Partial selfing. Genetics. 1987; 117(2):353–60.
 33
Porcher E, Lande R. The evolution of selffertilization and inbreeding depression under pollen discounting and pollen limitation. J Evol Biol. 2005; 18(3):497–508.
 34
Porcher E, Lande R. Loss of gametophytic selfincompatibility with evolution of inbreeding depression. Evolution. 2005; 59(1):46–60.
 35
Porcher E, Kelly JK, Cheptou PO, Eckert CG, Johnston MO, Kalisz S. The genetic consequences of fluctuating inbreeding depression and the evolution of plant selfing rates. J Evol Biol. 2009; 22(4):708–17.
 36
Porcher E, Lande R. Evaluating a simple approximation to modeling the joint evolution of selffertilization and inbreeding depression. Evolution. 2013; 67(12):3628–35.
 37
Remington DL, O’Malley DM. Wholegenome characterization of embryonic stage inbreeding depression in a selfed loblolly pine family. Genetics. 2000; 155(1):337–48.
 38
Ritland K. Extensions of models for the estimation of mating systems using n independent loci. Heredity. 2002; 88:221–8.
 39
Seltmann P, Hensen I, Renison D, Wesche K, Ploch S, Duenas JR, Cocucci A, Jung K. Biparental inbreeding depression, genetic relatedness and progeny vigour in a windpollinated treeline species in Argentina. Plant Ecol. 2009; 205(1):155–64.
 40
Simmons MJ, Crow JF. Mutations affecting fitness in Drosophila populations. Annu Rev Genet. 1977; 11:49–78.
 41
Storz JF. Genetic consequences of mammalian social structure. J Mammal. 1999; 80(2):553–69.
 42
Uyenoyama M. Inbreeding and the cost of meiosis: the evolution of selfing in populations practicing biparental inbreeding. Evolution. 1986; 40(2):388–404.
 43
Vekemans X, Hardy OJ. New insights from finescale spatial genetic structure analyses in plant populations. Mol Ecol. 2004; 13(4):921–35.
 44
Wagenius S, Hangelbroek HH, Ridley CE, Shaw RG. Biparental inbreeding and interremnant mating in a perennial prairie plant: fitness consequences for progeny in their first eight years. Evolution. 2010; 64(3):761–71.
 45
Winn AA, Elle E, Kalisz S, Cheptou PO, Eckert CG, Goodwillie C, Johnston MO, Moeller DA, Ree RH, Sargent RD, VallejoMarin M. Analysis of inbreeding depression in mixedmating plants provides evidence for selective interference and stable mixed mating. Evolution. 2011; 65(12):3339–59.
 46
Wright S. Evolution in Mendelian populations. Genetics. 1931; 16(2):97–159.
 47
Wright S. Isolation by distance. Genetics. 1943; 28(2):114–38.
 48
Wright S. Evolution and the genetics of populations. vol. 2. Theory of gene frequencies. Chicago, USA: University of Chicago Press; 1969.
 49
Zhao R, Xia H, Lu BR. Finescale genetic structure enhances biparental inbreeding by promoting mating events between more related individuals in wild soybean (Glycine soja; Fabaceae) populations. Am J Bot. 2009; 96(6):1138–47.
Acknowledgements
We thank J. Duminil, O.J. Hardy, R.J. Petit and C. Goodwillie, S. Kalisz, C.G. Eckert for sharing databases on plant outcrossing rates. Support for this work was provided by the CNRS program PICS grant 5273 to E.P., a Royal Society Research Professorship and a grant from the Balzan Foundation to R.L., and time on the computing cluster (UMS 2700 OMSI) at MNHN.
Author information
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
Both authors were involved in all aspects of the project. Both authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Received
Accepted
Published
DOI
Keywords
 Mixed mating
 Selffertilization
 Biparental inbreeding
 Nearly recessive lethals
 Mutation
 Inbreeding depression