Looking to Write vi that will Grab Smaller Sub-images Out of a Gigapixel Photograph

[johnfrCO]

I am looking into developing a program that would be capable of opening a 1 to 6 Gigapixel image (stored either as a Photoshop .psb or a ,jpg format). Once the image is open I want to capture 8K video frame sized images out of the original image. I am a LV programmer with around 22 years of experience developing LV applications. I have never delved into working with images. I also am a photographer that specializes in shooting Gigapixel Wall Mural photos such as the one at this link: http://abbascreationsphotography.com/Cape_Royal_South_East_View.htm

I want to create virtual tours of this type of photo that can be edited in 8K video like a time lapse. 8K video has a frame size of 7680×4320 pixels. Therefore, I want to grab 7680×4320 sized blocks out of the original image and store them as individual frames in .jpg files that would become part of a "time lapse" sequence. Ultimately I would like to be able to draw a path onto an overlay of the photo that my LV program would follow grabbing a frame every n pixels as it traces the path through the original image.

I envision the following issues:

1. First challenge would be to import the full image into an array capable of holding it. The image in the link above is 51,928 x 116,112 pixels (6.02 Gigapixels). The jpg file size is 216 MB. Thus, I see me needing an array size of 51,928 x 116,112 with a bit depth of 24 bits for color jpg. The PC I intend to run this on is a 64 bit i7 Hex core CPU and it has 128GB of RAM. I am unsure if LV can handle an array of this size.
2. The example below that I found here on the forum readily handles 7952 x 5304 pixel image from my Sony A7Rii. However if I try it with a image that is 61368 x 24952 pixels it crashes LV.

So the question is, Am I attempting something that is completely out of LV's capability?

[GregSands]

If you use the 64-bit version of LabVIEW, you should be ok with arrays of that size.

[johnfrCO]

Think I have found a road block & answered my own question.

I tried to read a ,jpg file that is 10000h x 40000w pixels. When I use the Read JPEG.vi in the image data cluster output it reports bottom at the correct value of 10000. The right value is reported as -25536.
When I open a smaller 8111h x 20109w image the image data cluster reports the correct values of bottom=8111 and right as 20199.
Further investigation revealed that the Rectangle.right, Rectangle.bottom are I32 with a maximum of 32.767. Thus this limits the maximum width and height of an image to 32,767 x 32,767.

[GregSands]

That doesn't surprise me too much, though it's not a complete dead-end.  I wondered about using IMAQ Vision, however that has a different constraint in that the maximum memory size of an image is 2GB (kudo this idea and this one to tell NI you'd like larger images!)

You would be better to read the image file directly into an array, not a LabVIEW Image. I just found this implementation which looks like it will work on Windows with 64-bit LabVIEW.  Having the image in an array should be easy for you to downsample a display image, and to extract subimages. 

[johnfrCO]

Thanks Greg for pointing to that. Unfortunately I don't have IMAQ Vision. I have LV 2015 PDS as my latest which doesn't include IMAQ.

[Kevin_Price]

I suspect you may need to "go native".  It seems you've found a limitation in image size support from the built-in LabVIEW function "Read JPG.vi".   (BTW it'd be an i16 that's limited to a positive max of 32767, not an i32).   You may need to dive into dealing with the jpg format natively or (hopefully) finding some kind of .net or .dll library that does some of the dirty work for you.

Instead of reading the entire giga-picture into memory all at once only to process it one 8k frame worth of pixels at a time, I'm pretty sure you'll be better off in the long run if you can simply limit your reads to 8k frame-sized chunks.

I don't know the jpg format well enough to know how one would "address" a specific rectangular frame from a giga-sized jpg, but it seems there must be a way.  (Possibly, you have to address a slightly bigger frame, use the jpg algorithm to construct something like a 2D pixel array, and then extract only the exact portion you want.)

An approach like this can use a chain of producer-consumers to pipeline the work.  First store a binary copy of the 200MB+ jpg file in memory.  You can then break out the set of bytes needed by the jpg algorithm to reconstruct the desired 8k frame.  Pass these along via queue to the jpg reconstructor.  That loop will turn the parameters and descriptors into a set of pixels 7680×4320.   Then you can pass those along via queue to another loop that turns those pixels into the format needed for a frame of 8k video.  

Something like that, I don't really know the specifics of the jpg->pixels->video frame->video process.

-Kevin P

[GregSands]

---

@johnfrCO wrote:

Thanks Greg for pointing to that. Unfortunately I don't have IMAQ Vision. I have LV 2015 PDS as my latest which doesn't include IMAQ.

---

You can do what you want without Vision - the Windows functions return the image as an array which you can use directly - just ignore the Vision part of the example and use the output of the Decode Image Stream vi.

[John_Freeman]

Kevin_Price, yes the "Read JPEG.vi outputs the size values in the Rectangle cluster as I16 values. It is a protected file so I am unable to go into that vi to modify it. As soon a I attempt to open a image that has more than 32767 pixels the program hangs.

[cstorey]

You could look at using LabVIEW to automate ImageMagick.  It seems there's already a LabVIEW implementation of the API, but its rather old and may or may not still work.

https://forums.ni.com/t5/LabVIEW/LVOOP-ImageMagick-Interface-v1-0-1-Released/td-p/538526

Here's info on the image size limits that ImageMagick supports (9Exa-pixels on 64-bit Windows!) - http://www.imagemagick.org/script/architecture.php#tera-pixel

I've used ImageMagick in the past for automating a lot of the usual image processing tasks and its very well documented. 

Craig