On-Line Big-Data Processing for Visual Analytics with Argus-Panoptes

Panayiotis I. Vlantis, Alex Delis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Analyses with data mining and knowledge discovery techniques are not always successful as they occasionally yield no actionable results. This is especially true in the Big-Data context where we routinely deal with complex, heterogeneous, diverse and rapidly changing data. In this context, visual analytics play a key role in helping both experts and users to readily comprehend and better manage analyses carried on data stored in Infrastructure as a Service (IaaS) cloud services. To this end, humans should play a critical role in continually ascertaining the value of the processed information and are invariably deemed to be the instigators of actionable tasks. The latter is facilitated with the assistance of sophisticated tools that let humans interface with the data through vision and interaction. When working with Big-Data problems, both scale and nature of data undoubtedly present a barrier in implementing responsive applications. In this paper, we propose a software architecture that seeks to empower Big-Data analysts with visual analytics tools atop large-scale data stored in and processed by IaaS. Our key goal is to not only yield on-line analytic processing but also provide the facilities for the users to effectively interact with the underlying IaaS machinery. Although we focus on hierarchical and spatiotemporal datasets here, our proposed architecture is general and can be used to a wide number of application domains. The core design principles of our approach are: (a) On-line processing on cloud with Apache Spark. (b) Integration of interactive programming following the notebook paradigm through Apache Zeppelin. (c) Offering robust operation when data and/or schema change on the fly. Through experimentation with a prototype of our suggested architecture, we demonstrate not only the viability of our approach but also we show its value in a use-case involving publicly available crime data from United Kingdom.

Original languageEnglish (US)
Title of host publicationAlgorithmic Aspects of Cloud Computing - 4th International Symposium, ALGOCLOUD 2018, Revised Selected Papers
EditorsYann Disser, Vassilios S. Verykios
PublisherSpringer Verlag
Pages102-117
Number of pages16
ISBN (Print)9783030197582
DOIs
StatePublished - 2019
Event4th International Symposium on Algorithmic Aspects of Cloud Computing, ALGOCLOUD 2018 - Helsinki, Finland
Duration: Aug 20 2018Aug 21 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11409 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th International Symposium on Algorithmic Aspects of Cloud Computing, ALGOCLOUD 2018
CountryFinland
CityHelsinki
Period8/20/188/21/18

Keywords

  • Apache Spark
  • Big-Data processing
  • IaaS Infrastructures
  • Interactive programming
  • Visual analytics

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'On-Line Big-Data Processing for Visual Analytics with Argus-Panoptes'. Together they form a unique fingerprint.

Cite this