Relative-error cur matrix decompositions

Petros Drineas, Michael W. Mahoney, S. Muthukrishnan

    Research output: Contribution to journalArticlepeer-review

    Abstract

    Many data analysis applications deal with large matrices and involve approximating the matrix using a small number of "components." Typically, these components are linear combinations of the rows and columns of the matrix, and are thus difficult to interpret in terms of the original features of the input data. In this paper, we propose and study matrix approximations that are explicitly expressed in terms of a small number of columns and/or rows of the data matrix, and thereby more amenable to interpretation in terms of the original data. Our main algorithmic results are two randomized algorithms which take as input an m × n matrix A and a rank parameter κ. In our first algorithm, C is chosen, and we let A'=CC +A, where C + is the Moore-Penrose generalized inverse of C. In our second algorithm C, U, R are chosen, and we let A'=CUR. (C and R are matrices that consist of actual columns and rows, respectively, of A, and U is a generalized inverse of their intersection.) For each algorithm, we show that with probability at least 1-δ, \\A-A'|| F ≤ (1 + ∈) ||A-A k|| f, where A k is the "best" rank-κ approximation provided by truncating the SVD of A, and where ||X|| f is the Frobenius norm of the matrix X. The number of columns of C and rows of R is a low-degree polynomial in κ, 1/∈, and log(1/δ). Both the Numerical Linear Algebra community and the Theoretical Computer Science community have studied variants of these matrix decompositions over the last ten years. However, our two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist. Both of our algorithms are simple and they take time of the order needed to approximately compute the top κ singular vectors of A. The technical crux of our analysis is a novel, intuitive sampling method we introduce in this paper called "subspace sampling." In subspace sampling, the sampling probabilities depend on the Euclidean norms of the rows of the top singular vectors. This allows us to obtain provable relative-error guarantees by deconvoluting "subspace" information and "size-of-A" information in the input matrix. This technique is likely to be useful for other matrix approximation and data analysis problems.

    Original languageEnglish (US)
    Pages (from-to)844-881
    Number of pages38
    JournalSIAM Journal on Matrix Analysis and Applications
    Volume30
    Issue number2
    DOIs
    StatePublished - 2008

    Keywords

    • Approximate least squares
    • CUR matrix decomposition
    • Data analysis
    • Random sampling algorithms

    ASJC Scopus subject areas

    • Analysis

    Fingerprint Dive into the research topics of 'Relative-error cur matrix decompositions'. Together they form a unique fingerprint.

    Cite this