Concealment of whole-frame losses for wireless low bit-rate video based on multiframe optical flow estimation

In low bit-rate packet-based video communications, video frames may have very small size, so that each frame fills the payload of a single network packet; thus, packet losses correspond to whole-frame losses, to which the existing error concealment algorithms are badly suited and generally not applicable. In this paper we deal with the problem of concealment of whole frame-losses, and propose a novel technique which is capable of handling this very critical case. The proposed technique presents other two major innovations with respect to the state-of-the-art: i) it is based on optical flow estimation applied to error concealment, and ii) it performs multiframe estimation, thus optimally exploiting the multiple reference frame buffer featured by the most modern video coders such as H.263+ and H.264. If data partitioning is employed, by e.g. sending headers, motion vectors and coding modes in prioritized packets as can be done in the DiffServ network model, the algorithm is capable of exploiting the motion vectors to improve the error concealment results. The algorithm has been embedded in the H.264 test model software, and tested under both independent and correlated packet loss models with parameters typical of the wireless environment. Results show that the proposed algorithm significantly outperforms other techniques by several dBs in PSNR, provides good visual quality, and has a rather low complexity, which makes it possible to perform real-time operation with reasonable computational resources.