PQA Blockiness

[Carolina1]

Hi,

I'm doing the blockiness test on PQA to a basic video stream (just frames one-color, one second has black frames, one second has white frames, one second green frames and so on). I made a test with provider AVI single and another test with provider digital AV (banc 8133 with modules 6545 and 2172 for HDMI). I get almost a similar behavior for two tests but numerical values are totally different. I want to know what is the ideal blockiness value for a perfect frame (no-blockiness at all) ? is it 1?

Carolina

[Paul-D]

Carolina1: The quick answer to your question is that the perfect score is 0, no difference in macroblock edges.  This can be demonstrated by loading attached AVI in PQA.

Blockiness:

1. First let's make sure you're using Blockiness to achieve what you want.  Blockiness is used to detect defects in the compression that occurs when a video is encoded.  To compress a video frame, it is broken up into macroblocks.  Each block is then compressed individually, and put back together as frame.  Depending on the codec being used and the level of compression, this can result in the appearance of blocks throughout the image, like this.  Note that the defects that Blockiness detects are introduced at encoding, not decoding a transmission, which can result in macroblock errors like this.
2. The blockiness processor requires that you enter the correct Macroblock size, so that it knows how to divide the frame back up, the same way it was divided up when it was encoded.  It defaults to 16 for 16x16 macroblock size, but 8x8 is also common.  If you don't pick the correct Macroblock size for your video, it can result in bad results.
3. The formula for the Blockiness processor is found in PQA Help at NI PQA Executive and The NI PQA Configuration Panel>>NI PQA Executive components>>NI PQA Tabs>>Processors Tab>>Video Metric Processors>>Blockiness Processor.  I've also pasted a copy below.
   
   
   
   
   
4. To better explain the formula, it first goes through and finds the intensity of each pixel in the frame.  It then goes through and compares the intensity of adjacent pixels on either side of the block, and computes their difference.  It does this in the horizontal direction and vertical direction.
5. In my visual example below, we can see a macroblock size of 3x3.
1. To calculate Horizontal Blockiness, it would compare the difference between (0,2) and (0,3), then (1,2) and (1,3), then (2,2) and (2,3), and would continue to do so for the rest of the macroblock borders.  If there is no difference between the 2, this would result in 0.
2. To calculate Vertical Blockiness it would compare the difference between (2,0) and (3,0) then (2,1) and (3,1), then (2,2) and (3,2), and would continue to do so for the rest of the macroblock borders.  If there is no difference between the 2, this would result in 0.
   
   
   
6. In a video where there is a single color for the entire frame, coded losslessly, we would get a result of 0 for each frame.  However with a lossy compression codec, depending on the algorithm of the codec, you could still see transition differences, resulting in a non-zero number.
7. If you had a video with an subject, not just solid color, you would never achieve a perfect score of 0, because even if there are no edges created by the macroblocking, as we divide up the frame, there are going to be subtle differences pixel to pixel.

Checkers:

1. If you're instead looking for artifacts and macroblocking errors from the transmission of the video, then the Checkers processor would be the one to use.  It effectively goes through pixel by pixel vertically and horizontally and looks for sharp changes in pixel intensity, which would typically be caused by a block like shown in the link above not matching the rest of the scene.  However, controlling the content that the Checkers processor sees is important too, because it will return false positives for things like boxes on the screen or on-screen text.

[Carolina1]

Hi,

I think for me it is better to use checkers processor because I have to test decoder devices (Set-top-box) instead encoder devices. Thank you for your answer and video. Do you have the same video but HD format 1080p? 

Carolina

[Paul-D]

About the AVI I created:

I built that video using the IMAQ AVI functionality by creating frames in Paint for the resolution I wanted, then built them up.  I am attaching the code and source that I used to make a 1080p video that you could expand and change up if you'd like.  The code isn't perfect, just a quick concept.

About Checkers:

Using the Checkers processor is most likely going to require some post processing work in LabVIEW.  At a minimum, it is going to require you analyzing the multi-point metrics from within PQA rather than the single-point.  Single-point averages out the entire frame, so if you have 1 bad block, its hard to tell that from other things that might be going on.  Multipoint breaks the frame up in the 'Block size' that you specify, and effectively returns that 2D array as a 1D array (all of row 0 data, followed by all of row 1 data, followed by all of row 2 data, etc) , so its results can be challenging to look at too.  Basically, you'll be looking for one or more multi-point block that had a spiked value, indicating that it ran into a hard edge.  

If you bring the Checkers data back into LabVIEW, you can re-compose it into a 2D array, and also use an intensity graph to better see what the returned data is indicating.  I have a demonstration of this that I can share, but would prefer not to post that to the public forum.  If you'd like to contact your local NI Support team and open a service request, I can share that internally through them.  It will be provided as-is as a demonstration, I can't fully support it.

One other tip, it's much more CPU intensive, but to get the most granular Checkers results, I would recommend reducing the block size to 4 (the lowest it can go).  This block size is different than the one used for Blockiness.

About SSIM:

Using Checkers is ultimately challenging unless you can control your video source, and use a source that doesn't contain hard unnatural lines in it (like on-screen text or borders).  If you can control your source, and you can control the replaying of it to your DUT, I would recommend the SSIM processor with a reference video.  Using this method you create a 'Golden' known-good reference, and then compare your future acquisitions to it.  SSIM is an advanced vision algoritihm that detects unnatural defects in pictures the way that humans perceive them.  Something like a macroblocking error will score very high because it is not natural in a scene.

[Carolina1]

Hi, 

Really I didn't know that checkers has a lot of work and I don't have Labview. But I can control my video, I'm just using images as you send me and I create avi videos as you. Then I use this video with the AVI single provider and then I send it to my set-top-box, finally I made the comparison between two videos. Actually I haven't worked with the SSIM or PSNR processors, but I will do it next week.

Carolina