Model-Free Neural Counterfactual Regret Minimization With Bootstrap Learning

Weiming Liu, Bin Li, Julian Togelius

    Research output: Contribution to journalArticlepeer-review


    Counterfactual regret minimization (CFR) has achieved many fascinating results in solving large-scale imperfect information games (IIGs). Neural network approximation CFR (neural CFR) is one of the promising techniques that can reduce computation and memory consumption by generalizing decision information between similar states. Current neural CFR algorithms have to approximate cumulative regrets. However, efficient and accurate approximation in a large-scale IIG is still a tough challenge. In this article, a new CFR variant, recursive CFR (ReCFR), is proposed. In ReCFR, recursive substitute values (RSVs) are learned and used to replace cumulative regrets. It is proven that ReCFR can converge to a Nash equilibrium at a rate of O1T. Based on ReCFR, a new model-free neural CFR with bootstrap learning, neural ReCFR-B, is proposed. Due to the recursive and noncumulative nature of RSVs, neural ReCFR-B has lower variance training targets than other neural CFRs. Experimental results show that neural ReCFR-B is competitive with the state-of-the-art neural CFR algorithms at a much lower training cost.

    Original languageEnglish (US)
    Pages (from-to)315-325
    Number of pages11
    JournalIEEE Transactions on Games
    Issue number3
    StatePublished - Sep 1 2023


    • Counterfactual regret minimization (CFR)
    • game theory
    • imperfect information games (IIG)
    • neural networks

    ASJC Scopus subject areas

    • Software
    • Artificial Intelligence
    • Electrical and Electronic Engineering
    • Control and Systems Engineering


    Dive into the research topics of 'Model-Free Neural Counterfactual Regret Minimization With Bootstrap Learning'. Together they form a unique fingerprint.

    Cite this