Background and aims: As a major evolutionary transition in seed plants, the evolution of plant mating systems has been much debated in evolutionary ecology. Over the last 10 years, well-established patterns of evolution have emerged. On the one hand, experimental studies have shown that self-fertilization is likely to evolve in a few generations (microevolution) as a response to rapid environmental change (e.g. pollinator decline), eventually rescuing a population. On the other, phylogenetic studies have demonstrated that repeated evolution towards self-fertilization (macroevolution) leads to a higher risk of lineage extinction and is thus likely to be disadvantageous in the long term.

Scope: In either case - the short-term or long-term evolution of self-fertilization (selfing) - these findings indicate that a mating system is not neutral with respect to population or lineage persistence. They also suggest that selfing can have contrasting effects depending on time scale. This raises the question of whether mating system evolution can rescue populations facing environmental change. In this review, empirical and theoretical evidence of the direct and indirect effects of mating systems on population demography and lineage persistence were analysed. A simple theoretical evolutionary rescue model was also developed to investigate the potential for evolutionary rescue through selfing.

Key findings: Demographic studies consistently show a short-term advantage of selfing provided by reproductive assurance, but a long-term disadvantage for selfing lineages, suggesting indirect genomic consequences of selfing (e.g. mutation load and lower adaptability). However, our theoretical evolutionary rescue model found that even in the short term, while mating system evolution can lead to evolutionary rescue, it can also lead to evolutionary suicide, due to the inherent frequency-dependent selection of mating system traits.

Conclusions: These findings point to the importance of analysing the demographic consequences of self-fertilization in order to predict the effect of selfing on population persistence as well as take into account the indirect genomic consequences of selfing. The pace at which processes such as inbreeding depression, purging, reproductive assurance and genomic rearrangements occur after the selfing transition is the key to clarifying whether or not selfing will result in evolutionary rescue.