Markov decision evolutionary games with expected average fitness

E. Altman, Y. Hayel, H. Tembine, R. El-Azouzi

Research output: Contribution to journalArticlepeer-review


Aim: To model and characterize evolutionary games where individuals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly selected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov decision processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of evolutionarily stable strategies based on the concept of 'occupation measures'. We relate this set to the concept of evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show the existence of an occupation measure ESS (OMESS).

Original languageEnglish (US)
Pages (from-to)677-689
Number of pages13
JournalEvolutionary Ecology Research
Issue number4
StatePublished - May 2009


  • Energy control in wireless networks
  • Evolutionarily stable strategy
  • Evolutionary games
  • Markov decision process
  • Occupation measure

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics


Dive into the research topics of 'Markov decision evolutionary games with expected average fitness'. Together they form a unique fingerprint.

Cite this