07-19-2019 02:35 PM - edited 07-19-2019 02:38 PM
Question:
What transformations on inputs or settings, assuming equal-sized 2d "src" and "kernel", are required for the MASM::CrossCorrelation.vi to give output consistent with CpenCV::filter2d?
Details:
I have code that is slow that was originally written in Python with OpenCV.
At the heart of this is the OpenCV::filter2d function which, according to OpenCV is really a correlation.
It claims to be a correlation, not a convolution, and in fact gives guidance for how to turn it into a convolution.
How do I go about getting consistent results? OpenCV has results with signed integer-valued intensities. LabVIEW has a table with very small coefficients, relatively speaking (1e-7 vs integers [-255,255]).
References:
Update:
08-23-2019 02:49 PM
What do you think the resolution of the coefficients needs to be to match your previous codes performance?
08-23-2019 03:13 PM
Do you have some simple input matrices and expected result? What size output do you want? (same?, doublexdouble?)
@EngrStudent wrote:Update:
- I'm going to try setting the input to type int8, and see how that impacts this.
I8 seems ill advised, because I guaranteed you'll overflow unless all inputs are <<<11 (a single 11x11 is already close to the max (127) and then you do all the additions on top. Most advanced math functions are only implemented for DBL anyway.