We study algorithms for computing the convolution of a private input x with a public input h, while satisfying the guarantees of (ε, δ)-differential privacy. Convolution is a fundamental operation, intimately related to Fourier Transforms. In our setting, the private input may represent a time series of sensitive events or a histogram of a database of confidential personal information. Convolution then captures important primitives including linear filtering, which is an essential tool in time series analysis, and aggregation queries on projections of the data. We give an algorithm for computing convolutions which satisfies (ε, δ)-differentially privacy and is nearly optimal for every public h, i.e. is instance optimal with respect to the public input. We prove optimality via spectral lower bounds on the hereditary discrepancy of convolution matrices. Our algorithm is very efficient - it is essentially no more computationally expensive than a Fast Fourier Transform.