Distributed mean-field-type filters for big data assimilation

Jian Gao; Hamidou Tembine

doi:10.1109/HPCC-SmartCity-DSS.2016.0206

Distributed mean-field-type filters for big data assimilation

Jian Gao, Hamidou Tembine

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The combination of observed data and dynamical models of mean-field type over networked systems is a challenging problem because of non-linearity, high dimensionality and partial observations. In many networked systems, the effective extraction and utilization of the information contained in observed data should be accomplished by exploiting the availability of accurate predictive, proactive tools of mean-field type dynamical systems. Incorporating observed big data into dynamical models of mean-field type has two problems. One is the curse of dimensionality, and the other is the control of error accumulation. This paper presents a distributed mean-field filter (DMF) framework for large scale networked systems. The proposed filter exploits the topology of the network and decomposes it into highly independent components with respect to the marginal mean-field correlations. The upper bound of global filtering error can be estimated using mean-field-type game theory. Numerical experiments in two object tracking scenarios are carried out to illustrate the performance of our algorithm. Evaluation results show that DMF significantly outperforms the existing filtering algorithms.

Original language	English (US)
Title of host publication	Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
Editors	Laurence T. Yang, Jinjun Chen
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1446-1453
Number of pages	8
ISBN (Electronic)	9781509042968
DOIs	https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0206
State	Published - Jan 20 2017
Event	18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016 - Sydney, Australia Duration: Dec 12 2016 → Dec 14 2016

Publication series

Name	Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016

Other

Other	18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
Country/Territory	Australia
City	Sydney
Period	12/12/16 → 12/14/16

Keywords

Filtering
Mean-field
Vehicle tracking

ASJC Scopus subject areas

Computational Theory and Mathematics
Computer Networks and Communications
Computer Science Applications
Hardware and Architecture
Information Systems

Access to Document

10.1109/HPCC-SmartCity-DSS.2016.0206

Cite this

Gao, J., & Tembine, H. (2017). Distributed mean-field-type filters for big data assimilation. In L. T. Yang, & J. Chen (Eds.), Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016 (pp. 1446-1453). Article 7828547 (Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0206

Distributed mean-field-type filters for big data assimilation. / Gao, Jian; Tembine, Hamidou.
Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016. ed. / Laurence T. Yang; Jinjun Chen. Institute of Electrical and Electronics Engineers Inc., 2017. p. 1446-1453 7828547 (Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Gao, J & Tembine, H 2017, Distributed mean-field-type filters for big data assimilation. in LT Yang & J Chen (eds), Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016., 7828547, Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016, Institute of Electrical and Electronics Engineers Inc., pp. 1446-1453, 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016, Sydney, Australia, 12/12/16. https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0206

Gao J, Tembine H. Distributed mean-field-type filters for big data assimilation. In Yang LT, Chen J, editors, Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 1446-1453. 7828547. (Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016). doi: 10.1109/HPCC-SmartCity-DSS.2016.0206

Gao, Jian ; Tembine, Hamidou. / Distributed mean-field-type filters for big data assimilation. Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016. editor / Laurence T. Yang ; Jinjun Chen. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 1446-1453 (Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016).

@inproceedings{9363bc5436294da2bfeb80b1dc9a4d8b,

title = "Distributed mean-field-type filters for big data assimilation",

abstract = "The combination of observed data and dynamical models of mean-field type over networked systems is a challenging problem because of non-linearity, high dimensionality and partial observations. In many networked systems, the effective extraction and utilization of the information contained in observed data should be accomplished by exploiting the availability of accurate predictive, proactive tools of mean-field type dynamical systems. Incorporating observed big data into dynamical models of mean-field type has two problems. One is the curse of dimensionality, and the other is the control of error accumulation. This paper presents a distributed mean-field filter (DMF) framework for large scale networked systems. The proposed filter exploits the topology of the network and decomposes it into highly independent components with respect to the marginal mean-field correlations. The upper bound of global filtering error can be estimated using mean-field-type game theory. Numerical experiments in two object tracking scenarios are carried out to illustrate the performance of our algorithm. Evaluation results show that DMF significantly outperforms the existing filtering algorithms.",

keywords = "Filtering, Mean-field, Vehicle tracking",

author = "Jian Gao and Hamidou Tembine",

year = "2017",

month = jan,

day = "20",

doi = "10.1109/HPCC-SmartCity-DSS.2016.0206",

language = "English (US)",

series = "Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1446--1453",

editor = "Yang, {Laurence T.} and Jinjun Chen",

booktitle = "Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016",

note = "18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016 ; Conference date: 12-12-2016 Through 14-12-2016",

}

TY - GEN

T1 - Distributed mean-field-type filters for big data assimilation

AU - Gao, Jian

AU - Tembine, Hamidou

PY - 2017/1/20

Y1 - 2017/1/20

N2 - The combination of observed data and dynamical models of mean-field type over networked systems is a challenging problem because of non-linearity, high dimensionality and partial observations. In many networked systems, the effective extraction and utilization of the information contained in observed data should be accomplished by exploiting the availability of accurate predictive, proactive tools of mean-field type dynamical systems. Incorporating observed big data into dynamical models of mean-field type has two problems. One is the curse of dimensionality, and the other is the control of error accumulation. This paper presents a distributed mean-field filter (DMF) framework for large scale networked systems. The proposed filter exploits the topology of the network and decomposes it into highly independent components with respect to the marginal mean-field correlations. The upper bound of global filtering error can be estimated using mean-field-type game theory. Numerical experiments in two object tracking scenarios are carried out to illustrate the performance of our algorithm. Evaluation results show that DMF significantly outperforms the existing filtering algorithms.

AB - The combination of observed data and dynamical models of mean-field type over networked systems is a challenging problem because of non-linearity, high dimensionality and partial observations. In many networked systems, the effective extraction and utilization of the information contained in observed data should be accomplished by exploiting the availability of accurate predictive, proactive tools of mean-field type dynamical systems. Incorporating observed big data into dynamical models of mean-field type has two problems. One is the curse of dimensionality, and the other is the control of error accumulation. This paper presents a distributed mean-field filter (DMF) framework for large scale networked systems. The proposed filter exploits the topology of the network and decomposes it into highly independent components with respect to the marginal mean-field correlations. The upper bound of global filtering error can be estimated using mean-field-type game theory. Numerical experiments in two object tracking scenarios are carried out to illustrate the performance of our algorithm. Evaluation results show that DMF significantly outperforms the existing filtering algorithms.

KW - Filtering

KW - Mean-field

KW - Vehicle tracking

UR - http://www.scopus.com/inward/record.url?scp=85013663368&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85013663368&partnerID=8YFLogxK

U2 - 10.1109/HPCC-SmartCity-DSS.2016.0206

DO - 10.1109/HPCC-SmartCity-DSS.2016.0206

M3 - Conference contribution

AN - SCOPUS:85013663368

T3 - Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016

SP - 1446

EP - 1453

BT - Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016

A2 - Yang, Laurence T.

A2 - Chen, Jinjun

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016

Y2 - 12 December 2016 through 14 December 2016

ER -

Distributed mean-field-type filters for big data assimilation

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this