Digital watermarking, traditionally modeled as communication with side information, is generally considered to have important potential applications in various scenarios such as digital rights managements. However, the current literature mainly focuses on robustness, capacity and imperceptibility. There lacks systematic formal approach in tackling secure issues of watermarking. One one hand, the threat models in many previous works are not sufficiently established, which result in somewhat superficial or even flawed security analysis. On the other hand, there lacks a rigorous model for watermarking in general that allows useful analysis in practice. There has been some efforts in clearing the threat models and formulate rigorous watermarking models. However, there are also many other cases where security issues are lightly or incorrectly treated. In this paper, we survey various security notions and models in previous work, and discuss possible future research directions.