Introduction to Partially Observed MDPs

Quanyan Zhu, Zhiheng Xu

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

In our cross-layer design, we use different models to capture the properties of different layers. As stated in Chap. 8, we can use an MDP model to capture the dynamical movements of the cyber layer. However, in the scenario, we assume that the defender can observe the cyber state at each cyber time instant. In real applications, it is challenging to obtain the full information of the cyber state directly. Hence, the MDP cannot capture the incomplete knowledge of the cyber states. In this chapter, we will introduce a Partially Observed Markov Decision Process (POMDP) to capture the uncertainty of the cyber state. In a POMDP, instead of observing the states, we have an observation, whose distribution depends on the state. Therefore, we use this information to build a Hidden Markov Model (HMM) filter, which can construct a belief of the states. Based on the belief, we aim to find an optimal policy to minimize an expected cost.

Original languageEnglish (US)
Title of host publicationAdvances in Information Security
PublisherSpringer
Pages139-145
Number of pages7
DOIs
StatePublished - 2020

Publication series

NameAdvances in Information Security
Volume81
ISSN (Print)1568-2633

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Introduction to Partially Observed MDPs'. Together they form a unique fingerprint.

Cite this