trouble reading high resolution PNG file

[YoMetric]

I have an application where i am trying to read a black and white PNG file and count the number of black pixels.  It has been working fine for me in testing, but so far all of my testing was done on "normal" sized image files.  We are trying to eventually scan a high resolution image with the highest DPI we can on a ~4" square area.  My scanner maxes out at 50,000 x 50,000 pixels, so i was able to scan a 4" x 3.5" area at 12500 dpi.

So...I have a high resolution PNG image file which contains 50,000 x 43750 pixels.  The image is black/white so the color depth is 1, which greatly reduces the file size to only ~3MB.  The problem i am having is that when i attempt to read the file using Read PNG File.VI, the resulting data does not appear to be correct.  I tried viewing the data on a picture control using Draw Flattened Pixmap.VI, but nothing appears.  I also tried just viewing the Image Data cluster output from Read PNG File.VI, and looking at the size of the Image array within that cluster.  It only contains 273,437,500 elements, but i would expect there to be 2,187,500,000.  It seems like it was truncated or something.  I am curious if there is a maximum size limitation that i am running in to?

Does anyone have any insight to why the Read PNG File.VI isnt working, or any suggestions for how i could count the black pixels in this PNG file

Thanks,

-Chris

[johnsold]

Check your math.  50,000 * 50,000 = 2,500,000,000 = 2.5 GB.

Lynn

[CoastalMaineBird]

Did you check the error output of the READ PNG FILE ?  That's your first clue.

I read the file OK using Windows PicViewer:

Not real pleasant to look at, but it's all there.

then I noticed - my RAM meter was stuck at 91% ! ! ! And the Windows Performance index shows 14.5 GBytes used:

I closed that and tried to read it with LabVIEW - I got a NOT ENOUGH MEMORY TO COMPLETE THIS OPERATION error.

If you did not check for errors, then you shouldn't be asking us before you do.

In any case, My LabVIEW is 32 bit, meaning it's  limited to 3GB (or is it 4GB, I forget).

[CoastalMaineBird]

Hmmm.  5e4 x 5e4 is 25 e8 = 2,5 e9 = 2.5 Gpixels. 

What you need to understand is that the fact that it's 1-bit depth doesn't help you here.  If you look at the data type that READ PNG FILE produces, the image data comes out in an array of U8.

That's 8 bits for each pixel.

That's 2.5 Gbytes right there.

The array seems to be sized at 273,437,500, as you said.

I don't know about the innards of the READ PNG FILE vi, but it's possible there is wastage in copying and resizing internal buffers.

Though I would expect it to know that at the start and not waste time trying.

In any case, if you can read the file piecemeal (you'll have to know the PNG spec for that), then you can count them.

Or maybe you can use an estimate? Open it in PicViewer and take a snapshot, then count the pixels in that and scale it up.

[YoMetric]

Hi Steve,

Thanks for looking into this.

I had been getting a "not enough memory" error yesterday when i first tried, but i only had 4GB of RAM in my system and i was using Labview 2014 32-bit.  I know the 2-3GB limit with 32-bit, so I added another 4GB stick for a total of 8 (all we had lying around at the moment but i ordered more RAM that i should get soon), and i installed Labview 2014 64-bit.  After that i was no longer getting an error message when i tried to use Read PNG File.vi, but instead getting this truncated looking output.  That is where i left off yesterday when i posted this to the forums. 

I realized it could possibly just be an issue of a lack of RAM, but i was just curious if anyone has tried reading a PNG file with so many pixels and if there was a limitation to that VI?  I couldnt find anything in the forums or google to verify.  I'm glad to see you were at least able to open the file with 16GB of ram in your system, i cant even currently open the file in windows picture viewer.  It sounds like i will have to wait until i get 16GB of RAM in my system and i will try again in Labview 2014 64-bit so it doesnt have that 2-3GB limitation. 

Thanks,

-Chris

[YoMetric]

I don't know about the innards of the READ PNG FILE vi, but it's possible there is wastage in copying and resizing internal buffers.

      That is what i was thinking too, it's a shame i cant view the inner workings and create a similar VI with a smaller data type output specifically for 1-bit color depth files, and also try to minimize the number of copies in memory.

Though I would expect it to know that at the start and not waste time trying.

In any case, if you can read the file piecemeal (you'll have to know the PNG spec for that), then you can count them.

     That was my next step if i cant manage to get it working with more RAM and the built-in Read PNG File.vi.  This would also take considerably more time as i would have to learn the PNG spec and write somethign to read it 

     from scratch

Or maybe you can use an estimate? Open it in PicViewer and take a snapshot, then count the pixels in that and scale it up.

     Unfortunately I cant use an estimate.  The reason we are using such a high resolution is that we are trying to very accurately measure the area of the black pads on this print.  The reason we thought this might be a viable method is

      that this scanner can output up to 19200 dpi resolution, allowing us to measure down to ~50 um per pixel.

[CoastalMaineBird]

Can you connect the scanner directly to the computer?

Maybe LabVIEW could read a signal and count the things live, rather than going thru a file.

Just throwing out ideas...

[CoastalMaineBird]

You might have better luck on the Machine Vision forum.

NI makes vision products and the people who use them might have better tools and ideas.

[CoastalMaineBird]

Can your scanner output anything else besides PNG?

Perhaps RAW data?

A RAW file format might be easier to deal with  - you could open it up and read one line, count pixels, read another line, count pixels, etc.

I know that PNG is compressed - I don't know if it's lossy or not.  If it's lossy, you're not as accurate as you could be.

[Verywiseguy]

PNG's are lossless, but you can intentionally make them lossy if you care more about compression.  I imagine YoMetric chose PNG for its losslessness.

YoMetric would have a much easier time if the image was saved in a format he could read pixel by pixel or even line by line, as you suggested.

Is there any way to split a PNG image into smaller images without actually having to open it? Then you could just open the smaller images and add up the black pixels in each.