A hybrid exact algorithm for complete set partitioning

In the Complete Set Partitioning problem we are given a finite set of elements where every subset is associated with a value, and the goal is to partition this set into disjoint subsets so as to maximise the sum of subset values. This abstract problem captures the Coalition Structure Generation problem in cooperative games in characteristic function form, where each subset, or coalition, of agents can make a profit when working together, and the goal is to partition the set of agents into coalitions to maximise the total profit. It also captures the special case of the Winner Determination problem in combinatorial auctions, where bidders place bids on every possible bundle of goods, and the goal is to find an allocation of goods to bidders that maximises the profit of the auctioneer. The main contribution of this article is an extensive theoretical analysis of the search space of the Complete Set Partitioning problem, which reveals that two fundamentally different exact algorithms can be significantly improved upon in terms of actual runtime. These are (1) a dynamic programming algorithm called "DP" [48,36] and (2) a tree-search algorithm called "IP" [32]. We start by drawing a link between DP and a certain graph describing the structure of the search space. This link reveals that many of DP's operations are in fact redundant. Consequently, we develop ODP - an optimal version of DP that avoids all of its redundant operations. Since ODP and IP are based on different design paradigms, each has its own strengths and weaknesses compared to the other. Thus, one has to trade off the advantages of one algorithm for the advantages of the other. This raises the following question: Is this trade-off inevitable? To answer this question, we develop a new representation of the search space, which links both algorithms, and allows for contrasting the workings of the two. This reveals that ODP and IP can actually be combined, leading to the development of ODP-IP - a hybrid algorithm that avoids the limitations of its constituent parts, while retaining and significantly improving upon the advantages of each part. We benchmark our algorithm against that of Björklund et al. (2009) [8], which runs in O(²ⁿ) time given n agents. We observe that the algorithm of Björklund et al. relies on performing arithmetic operations with very large integers, and assumes that any such operation has unit cost. In practice, however, working with large integers on a modern PC is costly. Consequently, when implemented, our O(³ⁿ) algorithm outperforms that of Björklund et al. by several orders of magnitude on every problem instance, making ours the fastest exact algorithm for complete set partitioning to date in practice.