Optimal No-Regret Learning in Repeated First-Price Auctions

We study online learning in repeated first-price auctions where a bidder, only observing the winning bid at the end of each auction, learns to adaptively bid to maximize the cumulative payoff. To achieve this goal, the bidder faces censored feedback: If the bidder wins the bid, then the bidder is not able to observe the highest bid of the other bidders, which we assume is i.i.d. drawn from an unknown distribution. In this paper, we develop ffiffiffi the first learning algorithm that achieves a near-optimal Õ(^√T) regret bound, by exploiting two structural properties of first-price auctions, that is, the specific feedback structure and payoff function. We first formulate the feedback structure in first-price auctions as partially ordered contextual bandits, a combination of the graph feedback across actions (bids), the cross-learning across contexts (private values), and a partial order over the contexts. We establish both strengths and weaknesses of this framework by showing a curious separation that a regret nearly independent of the action/context sizes is possible under stochastic contexts but is impossible under adversarial contexts. In particular, this framework leads to an O(^√ffiffiffi T log^2:5T) regret for first-price auctions when the bidder’s private values are independent and identically distributed. Despite the limitation of this framework, we further exploit the special payoff function of first-price auctions to develop a sample-efficient algorithm even in the presence of adversarially generated private values. We establish an O(^√ffiffiffi T log³T) regret bound for this algorithm, hence providing a complete characterization of optimal learning guarantees for first-price auctions.