Array of Cluster slow but faster when surrounded with a sequence structure (optimization)?

[Foreshadow20]

---

@JÞB

So, when do those shift registers get initial data and when are they flushed?  Proper init and shutdowns are really a great idea,  unless you want to get different results between runs and can't understand why.

---

Those uninitialized string shift registers can easily fool you when they have the last runs data in them rather than empty arrays. 

The data is re-initialized every time the software starts. I use the "Data: Initialize" of the JKI SM to overwrite the shift registers.

Prior to writing this thread, I tested that to confirm (both with stopping the software for code change and closing LV in between).

• 1st Run without Sequence: Slow
• 2nd Run with Sequence: Fast
• 3rd Run without Sequence: Slow
• 4th Run with Sequence: Fast

"Nesting string driven case structures 3 deep is a questionable practice."

The 2 main case structures allow to It simplifies the code to categorize the cases and avoid having to scroll and rearrange all the cases. The third is always for "function code". The JKI SM does have a lot of Cases in the main. One older software that we have here has ~150 cases to scroll through.

[mcduff]

I use the JKI State machine and I agree with Jeff, no need to nest case structures, makes it harder to debug also.

Try installing the JKI-State-Machine-Editor, makes going through multiple cases much easier.

https://github.com/JKISoftware/JKI-State-Machine-Editor

mcduff

[JÞB]

---

@Foreshadow20 wrote:

---

@JÞB

So, when do those shift registers get initial data and when are they flushed?  Proper init and shutdowns are really a great idea,  unless you want to get different results between runs and can't understand why.

---

Those uninitialized string shift registers can easily fool you 

• 1st Run without Sequence: Slow
• 2nd Run with Sequence: Fast
• 3rd Run without Sequence: Slow
• 4th Run with Sequence: Fast

"Nesting string driven case structures 3 deep is a questionable practice."

.

---

Yeah,  you just proved that you're reallocating a buffer on the fly poorly.  Can help more with code. Can't troubleshoot pictures.

---

[Foreshadow20]

Here's a video of what is happening (in attachment, may upload later at home when I have access to video sites)

[Foreshadow20]

---

@JÞB

---

---

Yeah,  you just proved that you're reallocating a buffer on the fly poorly.  Can help more with code. Can't troubleshoot pictures.

After the "Get TC" case:

I use an "Array Subset" to retrieve a 2D array of string from a larger table which is stored in "Obj Table" (was added later but visible on the video)

Some string operations are done within "Obj Table" using "Replace Array Subset".

I use "Replace Array Subset" to push "Obj Table" into an Array that has more rows.

No dynamically inserted rows/columns into arrays.

Still, even if there's poor re-allocation on the fly, it wouldn't explain that adding a Sequence Structure would increase speed by 10x.

[JÞB]

---

@Foreshadow20 wrote:

---

@JÞB

---

---

Yeah,  you just proved that you're reallocating a buffer on the fly poorly.  Can help more with code. Can't troubleshoot pictures.

After the "Get TC" case:

I use an "Array Subset" to retrieve a 2D array of string from a larger table which is stored in "Obj Table" (was added later but visible on the video)

Some string operations are done within "Obj Table" using "Replace Array Subset".

I use "Replace Array Subset" to push "Obj Table" into an Array that has more rows.

No dynamically inserted rows/columns into arrays.

Still, even if there's poor re-allocation on the fly, it wouldn't explain that adding a Sequence Structure would increase speed by 10x.

---

That's the thing.  It's not the sequence structure it's the buffer allocation of the shift register data.

[Foreshadow20]

Here's the operations that I do on the TC array.

I did try to have a full size initialization at TC (and disable the "TC >> Resize >> Init"), but that didn't change the behavior.

[Yamaeda]

The compiler is usually very smart, but this might somehow be a fringe case in which it gets fooled and always performs data copies, and with the sequence it realizes it doesn't have to.
For comparison you can try adding an In Place structure, it should also prevent data copies, it'd be interesting to compare the results.
This reminds of another thread a year or two ago, it also revolved around an array of clusters and performance and my suggestion of IPS increased the performance 100-fold. I wonder if we're on to something.

If there's anyone knowledgable about the inner workings of LV (AQ?), if an array of clusters including strings gets a data copy all the members get a copy and a separate malloc (or similar), right? (since each string is just a pointer in the cluster) So if a cluster has 5 strings you'll get 5 mallocs for the strings, per array element ... that adds up quickly.

/Y

[Yamaeda]

Looking at your first picture some more, since you have a Bundle you'll probably get a data copy, if you disable that temporarily and/or send that info to a new Event case i'm betting on good speed.

/Y