By means of multiple channel realizations that operate independently of each other we expect to predict the distortion incurred at the receiver. Figure 4 shows the usual coder at the top of the image together with one channel realization. The latter one is subdivided into two branches:
The intra branch encodes the entire input frame in intra mode and reconstructs the quantized frame which is distorted due to errors occurring on the simulated channel.
The inter branch encodes the difference signal as it would be done by the original encoder as well. In order to get the reconstructed frame, however, the previous reconstructed frame of this particular channel realization is added to the obtained difference signal after the channel errors have been applied. In other words, each channel realization has its own frame buffer. This measure is essential for the consideration of error propagation which is due to a deviation of the contents of the frame buffers at both ends of the transmission line.
The reconstructed frames of both branches serve to determine the mode to be chosen by the actual encoder. In the following some key properties of our channel realizations will be discussed.
Figure 4: Scheme of our coder
In practice the video sequence is transmitted as a bit stream. Modeling errors on the level of single bits, however, would unnecessarily complicate the analysis. If any bit is received in error the decoder cannot decode the respective macro-block any more which justifies that macro-blocks are considered the smallest unit for error treatment. As long as bit errors are uniformly distributed over the bit sequence we can also assign a certain error probability to a single macro-block. Errors of macro-blocks are dealt with according to three different schemes in our project (figure 5):
Cancel MB: Only the pertaining macro block is discarded.
Cancel GOB: The entire group of blocks in which the erroneous macro block occurred is discarded.
Cancel rest of GOB: All macro blocks beginning at the erroneous one and up to the beginning of the next GOB are discarded.
Figure 5: Concealment techniques
Scheme one (Cancel MB) is not very realistic because bit errors normally also cause a loss of synchronization at the receiver. Scheme two exaggerates as more macro-blocks than necessary are discarded. Both of these schemes have in common that the loss probability is uniformly distributed over each frame unlike for scheme three. For scheme three the probability of loss is higher on the right hand side of each frame than it is on the left hand side. Each GOB usually starts with a sync marker that allows for resynchronization at the receiver. This scheme is used both in practical applications as well as in major parts of our project.
Discarded macro blocks can not simply be set to a default value as this would heavily disturb human perception of the video. Instead the previous frame is used to predict the lost data. The easiest yet quite powerful method to achieve so-called error concealment is just copying the contents of the previously decoded frame into the canceled area. Advanced methods employ motion prediction, but have not been considered in this work.
Eventually, the outcomes of the inter/intra branches are evaluated jointly for all channel realizations in order to decide about the mode that the actual encoder has to employ.
Mode decision is accomplished on a macro block basis. Therefore for each macro block the total cost JINTRA for coding it in intra mode and the total cost JINTER for coding it in inter mode is determined. The mode with the lower cost eventually gets selected as illustrated in figure 6.
Figure 6: Mode selection mechanism
Again R denotes rate and D distortion. The rates RINTER and RINTRA measure the number of bits required for the actual encoder to apply the respective mode. In case of intra mode every channel realization performs exactly the same way as the actual encoder by just encoding the entire current frame. The rate RINTRA obtained in each channel realization is therefore the same as for the actual encoder. The rate RINTER, however, can not be determined that easily because it depends on the contents of the frame buffer based on which the difference signal is obtained. In each inter branch we therefore have to determine the difference between the current frame and the actual encoder’s frame buffer first to measure RINTER (which is then again the same for each channel realization) and then compute the reconstructed frame as the sum of the distorted difference signal and the frame buffer contents of each individual channel realization. Distortion is measured in both cases between the reconstructed frames and the original frames. The distortion measure is a slightly modified sum of squared differences scheme following these steps (also refer to figure 7):
For each macro-block and each single channel realization calculate the difference of the pixel values between the original frame and the reconstructed frame.
Sum up the squared differences over all N channel realizations and divide by N
Sum up the previously obtained average squared differences for all the pixels in a macro-block.
Figure 7: Distortion computation
Figure 8 illustrates the stages at which the measurements are performed. Unlike conventional rate-distortion optimized approaches our proposed scheme not only takes the quantization error into account, but also considers errors due to transmission loss and error propagation.
Figure 8: Experimental setup for measurements