Improving Learned Video Codec Performance in Low-Latency Lossy Communication Scenarios

Learned video codecs based on conditional coding architectures, e.g., DCVC, have demonstrated rate-distortion performance competitive with state-of-the-art classical standards, yet their behavior in presence of data loss is still largely unknown. The tight inter-frame dependency introduced by deep temporal context models causes far more severe problems than in classical hybrid codecs in presence of data loss, questioning the possibility of their actual usage in real-world communication systems. This paper presents a feedback-based error control mechanism for a learned video codec capable of encoding both intra-coded (I) and inter-coded (P) frames. When the receiver detects a lost frame and notifies the encoder through a feedback channel, the encoder inserts an I-frame at the earliest opportunity, restoring decoder synchronization and halting decoding issues. The approach requires no modification to the codec’s training procedure. Experimental results on standard video sequences over simulated lossy channels show that the proposed mechanism substantially reduces quality degradation, while the learned codec retains its rate-distortion advantage over AVC and HEVC coded video even at substantially high loss rate. Moreover, it can achieve the same or higher quality at a significantly less total bitrate cost.