Can VXImove be used asynchronously (DMA)?

[Mike Percy]

In other words, can VXImove (or other API function) provide a means to begin a transfer from local memory to VXI device such that the transfer progresses while the software continues to do other things. At some point, the status of the transfer can be checked to verify it completed.

It appears to me that VXImove, which may use DMA under-the-covers, blocks the program until the transfer completes (or times out). If there are long transfers, this may cost many burned CPU cycles that could have been doing other things.

If VXImove is truly blocking (in thread sense) and not mearly polling, then it can be handled in threads, but if there is an easy way to do it in the API, it'd be nicer.

Consider something like

int toggle
= 0;
unsigned buffer[2][MAX];
ASYNC_xfer xfer[2];

while (1)
{
populate_buffer(&buffer[toggle][0]);
status = VXImove_async_status(&xfer[toggle]);
assert(status = VXI_ASYNC_XFER_COMPLETE);
xfer[toggle] = VXImove_async(&buffer[toggle][0], 0, vxi_address..., );
toggle = !toggle;
}

Classic double-buffering, where one buffer is being filled while the other is being transferred. Under the current API, it seems that the CPU cost is time(populate) + time(VXImove). Async moves would reduce overall CPU cost due to overlapping of I/O (move) with CPU (populate).

[ScotSalmon]

Mike-

We recommend using the VISA API for new VXI development, and the behavior you describe is one reason why. NI-VISA does support asynchronous moves with its viMoveAsync operation and IO_COMPLETION events. The NI-VXI API always completes VXImove synchronously.

You're right that in many situations we are able to offload the move operation to a DMA controller on our hardware, so this can potentially allow you to use the CPU to perform other operations in parallel with the move operation.

If you must use the NI-VXI API (for example, because you're working with legacy code -- although converting to NI-VISA may still be a good idea!), you can do the VXImove in your own separate thread, as you noted. This will let you do additional computing in other t
hreads while the DMA is going on.

One note: depending on your software version, operating system, and your specific model of VXI controller, you may find that the CPU is already kept busy during a move operation. In some situations, especially on older drivers, NI-VXI does poll for move completion. Also, for optimal move performance we sometimes switch to programmed I/O which requires CPU intervention but has less total overhead for some types of moves (this is configurable if it's happening and you don't want it). Finally, our VXI-1394 controller has unavoidable CPU overhead for communicating with the FireWire bus. As a result of these variables, your benefit from making the move asynchronous will vary depending on your environment.

Let me know if you have any further questions.

-Scot (VXI Senior Software Engineer)

[Mike Percy]

I will definitely have to check out the VISA API then. This is indeed some mostly legacy code, well, just a few years old, which has used VXI API since the beginning. The VXI calls are relatively limited in scope, so converting to VISA is not likely to be a problem. Since the transfer time is not negligible in our overall timeline, asynchrounous, CPU-free transfers would be a boon.

We're using Linux [NI-VXI/VISA Software for Linux (NI-VXI v2.0.0 and v1.6.4)] and PCI-MXI-2. Is there somewhere in the documentation that I have overlooked "optimal move performance" techniques?

[ScotSalmon]

Mike-

Sorry for not replying to this earlier. I was expecting to find a reference to some simply optimization techniques in our PCI-MXI-2 manual for Linux, but the documentation I find only has that information in the Windows version, even though it applies to Linux as well. I have noted a suggestion to enhance the Linux manual with something along the lines of the performance note on page 3-10 of http://www.ni.com/pdf/manuals/321712b.pdf. Note that that section refers to the NI-VXI API operations VXImove and VXImemAlloc but the same applies to the VISA operations viMove and viMemAlloc.

-Scot

[Mike Percy]

After several months of doing more pressing work, I've finally been able to revist this issue with some test code. The main test function looks like the code below. The issue here is that the viMoveAsync always returns VI_SUCCESS_SYNC, so it seems that something is preventing the asnyc move from happening. Unless I can determine what needs to change to allow async moves, the normal viMove() is less trouble. Is it a Linux issue, some additional source/dest space definition issue, soem other init function that needs to be added, should the source space be allocated with viMalloc, locked, whatever. Assuming everything can be made to work async, there seems to be more overhead, so that the must be some message length below which it is more advantageous to use viMove(), and some where it is more useful to do viMoveAsync(). If there are no guideline son this, I will have to do some timing experiments.

Also, viMoveAsync() returns a jobID, but there does not seem to be any association between the jobID and the viWaitEvent() call -- that is, there does not seem to be a way to wait for a *particular* viMoveAsync() transfer to finish. I must be missing something.

void send_message(Board* this, const unsigned* message)
{
ViUInt16 srcSpace = VI_LOCAL_SPACE;
ViBusAddress srcOffset = (ViBusAddress)message;
ViUInt16 srcWidth = VI_WIDTH_32;
ViUInt16 destSpace = VI_A32_SPACE;
ViBusAddress destOffset = 0;
ViUInt16 destWidth = VI_WIDTH_32;
ViBusSize length = message[0] + 1;
ViEventType outEventType;
ViEvent outEvent;
ViStatus status;

#if 0
// synchronous move
status = viMove(this->session,
srcSpace, srcOffset, srcWidth,
destSpace, destOffset, destWidth,
length);
CHECK_STATUS(status);
#else
// make sure previous async move is complete (we're not really
// going to wait 1s)
if (this->jobId != VI_NULL)
{
status = viWaitOnEvent(this->session, VI_EVENT_IO_COMPLETION, 1000,
&outEventType, &outEvent);
if (status < VI_SUCCESS)
{
// if it hasn't finished by now, it's not gonna, kill it
status = viTerminate(this->session, VI_NULL, this->jobId);
CHECK_STATUS(status);

// now the I/O completion event should exist in the queue
status = viWaitOnEvent(this->session, VI_EVENT_IO_COMPLETION, 0,
&outEventType, &outEvent);
CHECK_STATUS(status);
assert(outEventType == VI_EVENT_IO_COMPLETION);
}
}
VALUE(outEvent, "%x");

// start async move
status = viMoveAsync(this->session,
srcSpace, srcOffset, srcWidth,
destSpace, destOffset, destWidth,
length, &this->jobId);
VALUE(status, "%x");
if (status != VI_SUCCESS && status != VI_SUCCESS_SYNC)
{
assert(status == VI_SUCCESS);
}
VALUE(this->jobId, "%x");
#endif
}

The relevent initialization snippets:

typedef struct Board
{
ViSession session;
ViJobId jobId;
} Board;

this->jobId = VI_NULL;

// open a VISA session to the device at logical address LA
sprintf(resource_string, "VXI0::%d::INSTR", LA);
access_mode = VI_NULL; // use defaults
timeout = VI_NULL; // wait forever (?)
status = viOpen(defaultRM, resource_string, access_mode, timeout,
&this->session);
CHECK_STATUS(status);

// create the async I/O completion event queue
status = viEnableEvent(this->session, VI_EVENT_IO_COMPLETION,
VI_QUEUE, VI_NULL);
CHECK_STATUS(status);

[ScotSalmon]

Mike-

I should have put 2+2 together earlier -- when I originally suggested using viMoveAsync I don't think I knew you were using Linux. Unfortunately NI-VISA on Linux does not currently implement viMoveAsync with a separate thread (only a feature of the Windows version at the moment). Sorry about that; it looks like you'll have to implement your own thread management. On a related note, the underlying driver for VXI doesn't implement the necessary support for viTerminate, so the ability to terminate a running I/O is not really there. The async support would provide you with the ability to run another operation in parallel, but not to terminate an operating in progress.

-Scot

[Mike Percy]

So, I'm not crazy or missing something important. As for viTerminate(), I never expected to have to terminate and was only calling it in the case of what would amount to a severe error -- the documetnation indicated that Terminate() should be called in cases of timeout.

Since maximum throughput is essential, especially for particular message lengths, I guess I will be testing various combinations of vximove vs. viMove vs. viOut32 vs. direct mapping, etc.

Oh, and I think I;ve figured out that I can call viGetAttribute() to get the jobId of the just-completed IO event, so it is possible to fetermine that some particular I/O has finished (if only async was possible in Linux). Plus, it seems that the overhead of doing the various viXXX calls for doing async would not be worth the coding trouble, for the amount of data I need to send, which is too bad but irrelevent since async does not work in Linux.

Thanks for the info, it has been helpful if unfortunately disappointing.