If you want to check memory usage of your system it's quite easy. You can just use LabVIEW System API. But if you want to see how much memory is used by your single process that you're running you have to use Windows System API.
An example likt this is attached below. Use this to check if your application is getting close to the maximum memory usage in your system or not.
This example is written for LabVIEW 32 bit but with easy modification you can use it also for 64 bit apps.

Example code from the Example Code Exchange in the NI Community is licensed with the MIT license.
How is it possible to get the memory usage of another process (*.exe)?
You can improve performance ~100,000x if you store the process reference:
I tried the example by nanocyte in a while loop but it does not appear to show updates.
Why is the output of this different than than any output of LabVIEW 2020's new Get Memory Status VI? And why are both different from all of task manager's indications?