Comparing the streaming of FGS encoded video at different aggregation levels: Frame, GoP, and scene

Fine granularity scalability (FGS), a new coding technique that has recently been added to the MPEG-4 video coding standard, allows for the flexible scaling of each individual video frame at very fine granularity. This flexibility makes FGS video very well suited for rate-distortion optimized streaming mechanisms, which minimize the distortion (i.e. maximize the quality) of the streamed video by transmitting the optimal number of bits for each individual frame. The per-frame optimization of the transmission schedule, however, puts a significant computational burden on video servers and intermediate streaming gateways. In this paper we investigate the rate-distortion optimized streaming at different video frame aggregation levels. We find that compared to the optimization for each individual video frame, optimization at the level of video scenes reduces the computational effort dramatically, while reducing the video quality only very slightly.