Tranfering data from a CSV file into the SystemLink Server

[craneMac]

I have a collection of CSV files saved to my SystemLink server that contain measurement data for various tests. I want to take the data from these CSVs and create test results on the server.

Is there a way to do this in Python without using TDMS/DataFinder? If so, which API would work best for this task?

Thanks in advance fine people.

[JoshuaP · Accepted solution]

You will need to use a small portion of DataFinder and the Analysis Automation to do this today, however we are working on an alternative that will be available next year.

For now, you will need to create a DataFinder instance for File Ingestion.

Then create a saved search.

Finally, open up Jupyter and click on the fx icon on the left to create a new analysis automation procedure in Jupyter that will run any time a new file arrives that matches your saved search. You can then use the notebook to parse the CSV file and call the appropriate Test Monitor API's to create the result and append the file to it.

[craneMac]

Thanks for this, its been very helpful to see what steps I need to take moving forward. I am stuck on the second screenshot where you say to create a saved search.

It appears that you are in the Data Navigation view and have selected the DataFinder "FileIndex" to create a new saved search.

When I try to follow this step and select my DataFinder, I get the following error:

Forbidden: EXCEPTION: AoException: AO_ACCESS_DENIED (1): RequestorBase(173): ERROR(181): Limit of simultaneous connections reached or no license available.

My DataFinder is indexing correctly and I have verified with my supervisor about the license, so I believe the licensing isn't the issue. 

Besides this, I am confused on what I should be searching for. 

Thank you for shepherding me through this process so far.

[JoshuaP]

In most cases that error will show up if you have a limited number of licenses and they are currently already allocated to other connections, which can be other users who have the web interface open or other users who have DIAdem connections. Maybe wait a few minutes and try again.

[JoshuaP]

The saved search is how you define which files you want to trigger your notebook. For example, if you create a saved search for files with the extension TDMS it will then run your notebook everytime a new file is ingested with that extension.

[JoshuaP]

I also wanted to mention that we know the workflow is a bit disjointed currently and we are currently working on some improvements that will make this a lot more straightforward in the future.