Not uniform length in array

[MiguelIrusta]

Dear all,

I am having one problem while writting an array. I want to pass data to a FIFO that goes to an FPGA, but I would like to all elements be of the same length; some of them are formed of 30bits, others 10...and all should be of 32 bits. The first picture ilustrate that (NoFixedLength).
The idea is after that pass the data to the FPGA module and output data (32 bits every 10ns), but also here data is not of the desired length (which seems logic due to the not fixed length of data entry, but the resulting element read is even shorter than the data in the entry) as it is shown in the second picture (LessDataFPGA).
The idea of the project is to send the data received from the FPGA module, by streams of 32bits every 10ns, I have divided into FIFOs to have more data in memory, but it should be output in order, that is why I have thought in case structures in the FPGA module, but it also did not work properly, only data of FIFO1 is being shown and not well.

Thank you in advance, if more info is need ask me please.

Miguel.

[Guilherme_Correa]

You don't need to have the state machine inside the timed loop, you could read all FIFOs in sequence.

If the source for the FIFO is not unique, you could use most of the FPGA power to run the FIFOs in parallel.

Also, you don't neet to have the outer while loop, in fact if you have a timeout only flag it to a indicator, but it is not required to have a control to stop the Timed Loop since it is being driven by the external clock. The code in FPGA as a good practice should be running all the time.

If your application requires more determinism, you may try to use only two FIFOs, one acting Target to Host, and another one Host to Target. Inside the FPGA code, you may place another FIFO to work your data internally.

[GerdW]

Hi Miguel,

I want to pass data to a FIFO that goes to an FPGA, but I would like to all elements be of the same length; some of them are formed of 30bits, others 10...and all should be of 32 bits. The first picture ilustrate that (NoFixedLength).

Well, if you would set a proper display format in your indicator (like "%032b") you should get each array element shown with 32 bits…

Your array consists of U32 values: each element is stored in 32 bits as the datatype name implies. There's no other possibility!

When the data is shown different (or: "not as expected)" in the front panel then clearly the formatting is wrong!

The idea is after that pass the data to the FPGA module and output data (32 bits every 10ns), but also here data is not of the desired length (which seems logic due to the not fixed length of data entry, but the resulting element read is even shorter than the data in the entry) as it is shown in the second picture (LessDataFPGA).

Here the same problem applies: set the correct formatting in the FP indicator!

Additionally you have a coercion dot at your "FIFO 1" indicator: there is some datatype conversion going on! Did you correctly configure the FIFO?

[MiguelIrusta]

Dear GerdW,

You were right, some of the FIFOs were not well configured, some of them were with U8 type and the data that come is U32. That should solve some of the problems, but the data that arrives to the FIFO still does not fit. I will explain deeper my solution: I read a big file and I split it. The result is a string or text that I need to pass to the FIFO. I convert into an array with the "String to byte array function" in order to pass it to a valid format. Then I do the cast to make it of 32bits instead of U8. Is probably there the mistake, in that cast or in the pass to the Byte array? The result in the indicator has not changed, one element with more bits than the other. (picture attached)

Dear Guilherme_Correa,

I have done it like a state machine because I thought it was the clearest option, but what I need is only FIFOs to host to FPGA and the directly from the FPGA pass it outside by a port (not yet decided which one), so, I do not need to return the data to the Host. I have changed it without the external while loop, but it still does not pass from one state to another. The source of the file is part of the string that enters to the FIFOs, I mean, a string for example of 16MB is divided into 16 strings of 1MB and each string is driven to each FIFO. I have done firstly in parallel but the data maybe wont'b sent in order and thats why I have changed, but if does not matter I will change again. If the parallel option is not available, are you suggesting me to do it with a flat sequence?

Many thanks to both of you.

Kind regards,
Miguel.

[GerdW]

Hi Miguel,

this image is useless as we cannot see which datatypes you using in the typecast!

Please show all needed information, attaching the VI would be best!

[MiguelIrusta]

Thank you again for the response,

Here is the VI!! If some more info is needed let me know.

Many thanks,

Miguel.

[crossrulz]

Why don't you just read U32 directly from the file instead of doing all of the type casting ect.?  Furthermore, you could read all of the file in one shot and then use Decimate 1D Array to create all of the data your FPGAs will need.  1 file read and one set of writes to your FIFOs.

[ZX81]

1:  the data values are getting cropped.  You mentioned changing to U32.  Did you finish that, both for the type cast and for the FIFO configuration?  There should be NO red dots.  The red dots show that some data was lost (or at least changed).  Mixing up U32 and I32 would work, but is bad form; make them all U32.

2:  The state machine is designed so it will only change state after the entire file is done (or similar).  The state number will only increment if there are no values left in the FIFO.  But as it is written, you will put all values into the FIFO at once.  So, FIFO 1 will be written over and over, till the entire FIFO is emptied.  That will be too fast to show up on your display.  Then, it will move to the next state.  The final behaviour may be a little random.  If you send the next block of data before the first one is finished, it may stay in FIFO 1 even longer.  I don't have a good solution for you, because it depends too much on your application.  But it should be addressed.

3:  The type cast from an array of 8-bit values to an array of 32-bit values will add (if I remember right; maybe it's delete) up to 3 bytes at the end, to make up the difference in total byte count.  Depending on your application, that might be very important.

Your data rate seems awefully high.  Usually, I would recommand against all the effort of converting to U32, but for the data rate, maybe you need it.  U64 may also help.

[MiguelIrusta]

Dear Crossrulz,

Thank you so much for the response, I cannot read from a row due to the volume of data I will work with (around 1TB), so I have to read splitting the files. I have changed the data type and instead of Decimal 1D I have used "Split 1D Array Function" and now it arrives always 32bits (I have to check the index,now does not work properly).

Dear ZX81,

Thank you so much for the response as well, as you said I have changed all up to U32 and no red dots appear this time (I have changed the data inputs for the divide blocks also).
According to your second point, that was my idea, but probably I should change in that case the while loop to a timed loop also in the hostVI to be sure the host is "slow" enough to let pass to the following FIFOs previously than writting again the first one. The displays in the FPGA site are only to check the data if I am executing slow (with the light lamp switched on), in the final application will not be there, as you say it is impossible to see 32 bits every 10ns.

I will change it and I will post the results.

Thank you all for the responses.
Best regards,

Miguel.

[MiguelIrusta]

Dear all, 

Here is my last version. Now the blocks are of 32. The program now can divide into little files if the button is activated and in other way is read like a number. I put it here in case someone would like to use it. The index is selected like: ((sizeOfTheFile/N)/NumberOfFPGAs)/4.
N is de number of subfiles obtained dividing the TotalSize/DesideredsubFileSzize.

4 is because the size is in bytes and the components are in U32 (four bytes).

I think that is correct, isn't it?

The only think is that the number send to the FPGA does not fix exactly with the number that should be, do you know what could be the reason? Are similar, but not the same. If you have a file you can check it, I have no reasonable explanation.

Anyway here is the VI, previous to the final one it can unless help somebody with some of the doubts I had.

Thank you again to everybody implied in the discussion.

Kind Regards,
Miguel.