3D Point cloud contruction from a series of imagings.

[tsa256]

Good day everyone,

A few peers and I have begun development of a 3D imaging system. We have managed to complete a substantial amount of this project but have run into a few obstacles regarding the 3D reconstruction. So far we have a system that tracks an imaging device in all x, y and z coordinates. Our plan is the location data (x,y,z) of this imaging device and extract a series of points from the image that will be used as coordinates on a point cloud.

Assuming (x,y,z) = location of imaging device
Assuming (X,Y,Z) = coordinates of 3D image

Our main problem lies within the procedure to calculate these points in the point cloud. Our original plan suggested that we should use an algorithm such as the KLT feature tracker to find suitable points on the image for tracking. These points would then be paired with the location of the imaging device to get X, Y and Z of the point. After numerous iterations our system would acquire enough X, Y, Z points to produce a point cloud. This point cloud could then be rendered as a 3D object.

So my main questions lies within the pairing of these two systems. Would the KLT feature tracking algorithm be sufficient, are there other perhaps better ways of implementing this system? What short of difficulties should we expect, and how should we avoid them. If anyone has prior experience with similar systems or suggestions that would help I would greatly appreciate it.

Thank you,
Taylor S. Amarel

[The_Chancenator]

Hi Taylor,

First, it sounds very interesting what you are doing.  What are you imaging?  Are you still using DICOM images?  What type of imaging system is it?  How are you using NI/LabVIEW in this endeavor? 

Are you trying to do more of an image registration, where you'd stack the images based upon a certain feature or position (my understanding at least), just to align them--or a full on reconstruction where you hope to be able to view the reconstructed image as a whole?

Second, while I'd be happy to help you with any sort of LabVIEW related questions, my knowledge of any sort of 3D image reconstruction or the algorithms that do this is quite limited.  You may try posting to the Biomedical Users Forum as you've done before and see if anyone there has any suggestions. 

At this point, I don't know a native way to do this in LabVIEW.  However, if you post back with any LabVIEW or Vision related questions I can be of some more assistance.

Cheers,

[tsa256]

Hi Marti, I appreciate the help.

Currently the system is being produced with the goal to image nerves within human tissue, and then display a 3D image of these nerves.

We are using LabVIEW to interface with two USB web cameras in order to track three-six indicators on the ultrasound probe. After successfully isolating these indicators with a region of interest algorithm we implement stereoscopic image triangulation to derive the physical position of the ultrasound probe (x,y,z). In addition to these images being received we are planning to receive a stream of DICOM images. ( For early development we use sample DICOM on an external storage median.) Then the DICOMs images will been imported into labVIEW via a conversion or proprietary work around. Once converted these images will undergo a procedure for determining 2D features that are easily tracked. (We are considering the KLT feature tracking algorithm but are not entirely committed due to insufficient knowledge, and I believe there are other options.) After the "easily track-able 2D features" are made they will be programmability pair with the x,y,z coordinates of the sensor at acquisition time of this specific 2D point. Then as we retrieve more ultrasound images we would like to compare the change in these "easily track-able points' and successfully create a depth-map that will allows us to combine the x,y of the feature with the depth, thus creating a series of 3D coordinates (X,Y,Z). These coordinates will then be exported into another program and rendered into a 3D image.


The link below shows a quick visual diagram of our project in a more physical setup. 

In the future we hope to implement this system in real-time, but lack of resources has required much work to be done without proper data streams from transducers.

The above paragraph described our project. We have a relatively firm grasp on everything up the blue text. Determining "easily track-able features" seams to be a roadblock.Do have any advice or documented manuals on this subject. I think once we get that down the rest will come together a bit easier.

Other quick questions,
We are still using DICOM images, but have interfaced with the command line of ImageMagick to convert to JPEG. This does provide results but we take a significant performance hit because the images are loaded into ImageMagick (via labVIEW console commands and directory locations), converted, saved, then the new saved file is imported back into labVIEW. In hopes of this system being implemented real-time I do not believe this is adequate. So our alternative is would it be possible to dissect a DICOM file using labVIEW. As a former c programmer I know it was possible to partition and separate a file relative by knowing the byte location. DICOM files have a 128-byte preamble, and a 794-byte data file and then the image. This image is either compressed into JPEG lossy, or the more common JPEG lossless, the file type is also indicated within the 794-byte data file. I'm not too familiar with labVIEW low level stuff like this, but would it be possible? If so, do you know of any help files related to this, I've searched to no avail.

Thank you, you help is very much appreciated.
Taylor S. Amarel

[The_Chancenator]

Hi Taylor,

I can certainly help with the file reading and writing questions.  The best way to go "byte by byte" would be to use the read binary file functions in LabVIEW.  With binary files, you can use that "set file position" function to specify where exactly in the file you'd like to read.  I'd recommend taking a look at what "Advanced File Functions" you have under the File I/O palette as these would most definitely be helpful.

Once you get an actual array of pixel data, its not hard to get it into an image: Converting a Binary File to an Image

What is tricky is that you must know the exact file structure that you are dealing with, which it sounds like DICOM is a widely published and available file format.

How Can I Read and Write Binary Files Using LabVIEW is also a helpful article.

In terms of tracking objects, I assume you are wanting to do this from one image to the next, and then determine the position from one image to the next.  Do you have the Vision Development Module?  If so, we could try using some of these functions to track objects in a pattern matching type algorithm.

Cheers,

[tsa256]

I looked into the Binary information and unfortunatley I ran into an unexpected obstacle.  I tried to read from the DICOM image at a the starting point of the JPEG but then released it required a previously used dummy datatype.To my newly found knowledge LabVIEW needs to write to a binary  file, and know the format before it can read it.  Thus arising the question, how would I take a regular DICOM file and convert it into a acceptable labVIEW datatype that could be saved to a binary file? If I know how the DICOM file is partitioned regarding the 128, 794, and JPEG image. I thought I could just cut off the first 922 bytes, and then read the file.  But this byte by byte method requires the file to be in a binary format already, right? Maybe I'm missunderstading something here, but is it possible to just discard the first 922 bytes without the first having the Image in Binary? If I could somehow extract the JPEG in binary, normal, or array then I could use it with IMAQ functions.

As for processing, I do have the developement  NI vision module, and have a fair understanding of the build in functions. My knowledge falls short on how to apply these functions for this system. I've been researching many ways of analysing the image, to create a 3D volumetric image. These techniques range from the feature tracking, to interpolation techniques, and although I'm familiar with how they work I am not sure I have the experiance to implement them successfully. I'm willing to try anything. If you have any suggestions for this part of the system I will be more than happy to try them out.

Thank you.
Taylor S. Amarel

[The_Chancenator]

Hey Taylor,

You are correct--you do need to know the exact format- byte structure and all, of the file to read it from binary format.  My understanding is that at their root, all files are essentially binary (that is, represented byte by byte in 0's and 1's).  If you are not familiar with LabVIEW or file structures, this would be a very tricky task.  Unfortunately, we at support do not have the resources to develop code like this for customers--but we do have alliance members that do this type of work (for a fee).

I also noticed that there has been a release of the Biomedical Startup Kit (v2.1)--could you check there to see if the Dicom functionality has been added?

As for the 3D image--how do you plan on this being formatted?  Are you wanting to also view this in LabVIEW?  I wish I could provide some more answers on the actual reconstruction and analysis--whether it be feature tracking or some other method, but I am by no means an imaging expert.

[KLI]

Hello TSA256,

Did you find a solution ?

I'm trying to accomplish the same.

I have 2D arrays that I'd like to create a point cloud of.

Is there any information you have that could help me?

Thank you in advance.

Best Regards

Kristof

[Tim_W.]

Hi KLA,

Are you looking to do a 3D plot or 2D? I found this resource on displaying 3D point cloud data: Display Point Cloud in 3D Picture Control. If you're only looking to do a 2D plot, there are several tools available including Compass Plots.

Tim W.

Applications Engineering

National Instruments

http://www.ni.com/support 

[KLI]

Hello Tim,

I'm looking for a 3D plot.

My problem is, I believe, I first need to create a point cloid with the information(arrays) I have available and I'm not sure on how to do that.

I have arrays of 120x90 'pixels'.

Each of this pixels contains the 'distance' information.

I can plot 2D images of this information, but I'd really like to plot it in 3D.

Any help is appreciated.

[Tim_W.]

I think this forum might be a good place to look: Best 3D Visualization

Tim W.

Applications Engineering

National Instruments

http://www.ni.com/support