Understanding Spatio-Temporal Urban Processes

Increasingly, decisions are based on insights and conclusions derived from the results of data analysis. Thus, determining the validity of these results is of paramount importance. In this paper, we take a step towards helping users identify potential issues in spatio-temporal data and thus gain trust in the results they derived from these data. We focus on processes that are captured by relationships among datasets that serve as the data exhaust for different components of urban environments. In this scenario, debugging data involves two important challenges: the inherent complexity of spatio-temporal data, and the number of possible relationships. We propose a framework for profiling spatio-temporal relationships that automatically identifies data slices that present a significant deviation from what is expected, and thus, helps focus a user's attention on slices of the data that may have quality issues and/or that may affect the conclusions derived from the analysis' results. We describe the profiling methodology and how it derives relationships, identifies candidate deviations, assesses their statistical significance, and measures their magnitude. We also present a series of cases studies using real datasets from New York City which demonstrate the usefulness of spatio-temporal profiling to build trust on data analysis' results.