Low rank approximation with entrywise l₁-norm error?

Abstract

We study the l1 -low rank approximation problem, where for a given n×d matrix A and approximation factor a ≥ 1, the goal is to output a rank-k matrix A for which (Equation presented), where for an n × d matrix C, we let ||C||1 = Σni=1 Σdj=1 |Ci, j|. This error measure is known to be more robust than the Frobenius norm in the presence of outliers and is indicated in models where Gaussian assumptions on the noise may not apply. The problem was shown to be NP-hard by Gillis and Vavasis and a number of heuristics have been proposed. It was asked in multiple places if there are any approximation algorithms. We give the first provable approximation algorithms for l1-low rank approximation, showing that it is possible to achieve approximation factor a = (log d) · poly(k) in nnz(A) + (n + d) poly(k) time, where nnz(A) denotes the number of non-zero entries of A If k is constant, we further improve the approximation ratio to O(1) with a poly(nd)-time algorithm. Under the Exponential Time Hypothesis, we show there is no poly(nd)-time algorithm achieving a (1 + 1/log1+γ(nd))-approximation, for γ > 0 an arbitrarily small constant, even when k = 1. We give a number of additional results for l1-low rank approximation: nearly tight upper and lower bounds for column subset selection, CUR decompositions, extensions to low rank approximation with respect to lp-norms for 1 ≤ p < 2 and earthmover distance, low-communication distributed protocols and low-memory streaming algorithms, algorithms with limited randomness, and bicriteria algorithms. We also give a preliminary empirical evaluation.