LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Amazon AWS .NET

Hello,

Some years ago, I wrote VI's for connecting to  Amazon AWS S3, which is a storage service provided by Amazon.  They are buggy and have limited function, and so I am now trying to figure out how to connect to S3 service using the .NET SDK provided by Amazon.   I have attached a sampe VI written in LV 2012.  It enables you to put text and then get the same text and write to a file.   I have included a Key/SecretKey combination that will allow reading and writing to one of my sample buckets.  Those keys won't get into any of my other buckets.

 

I have it "sort of" working.  The read and write happen without error about half the time.  But they are extremely slow (probably 5 seconds to do something that should take 1/4 second.)  I'm out of guesses.  If anyone has experience with AWS .NET, I'd appreciate any pointers.  Here's some additional info:

 

 

The SDK is here:

http://aws.amazon.com/sdkfornet/

 

The documentation for the SDK is here:

http://docs.amazonwebservices.com/sdkfornet/latest/apidocs/Index.html

http://awsdocs.s3.amazonaws.com/sdk-net/latest/aws-sdk-net-dg.pdf

0 Kudos
Message 1 of 7
(3,108 Views)

Hello pwalsh2,

 

After taking a look at your VIs, two things struck me as a little odd.  First, you are using a string as the input to you case structure.  You should be using an enumerated value - LabVIEW takes longer to analyze the string input that it does for an enum.  This is because the enum has predetermined possible values, while strings can take any value.  It is a simple operation to drop an enum on your front panel and add two values, one for Get and one for Put.  While the execution time saved by changing the case selector value may be small, it is still saved, and using the enum may simplify development down the line.  

 

Also, I noticed you are using a File Dialog Express VI.  These VIs are designed to be overly simple to use, which results in their code taking longer to execute.  I would recommend using a File Path Control instead; you can set it to either prompt you for a file path or code in the appropriate path.  This will also save you some execution time.

 

While these two changes will improve your program, they will not result in the jump in execution speed that you are hoping for.  After I performed some basic benchmarking on your code, it became clear that most of the execution time is spent in the AmazonS3Client invoke node.  I believe this node is where communication with Amazon's servers takes place, and there is very little we can do to speed that communication.  You could try pinging their server to check the data transfer rates.  Your connection speed has a lot to do with how fast this call will execute.

 

Hopefully you can find a way to increase the performance to a level you are comfortable with.  Thank you.

Patrick
CLA
Message 2 of 7
(3,070 Views)

Hi Patrick,

Sorry I'm late getting back with you.  Thanks for the advice related to enum.  I will change my style on that.

 

On the AmazonS3Client, I agree that's where the time is.  I know the amazon server is not the problem because I have sub-VI's written that do these operations in 1/4-1/2 second.  I don't know why the amazon .NET SDK takes so long.  And sometimes, it fails.  I am guessing it has something to do with the properties I am setting at PutObjectRequest, but I can't figure it beyond that.  I will hunt the amazon forum to see if any of the C programmers have had similar problems.  

Thanks for the help.

Pat

0 Kudos
Message 3 of 7
(3,041 Views)

Patrick,

Another Quick update.  I just set up another sample program, but this time, I made 20 Put requests inside of a loop.  The first one takes 4 seconds, and the next 19 take 0.1 seconds each which is about what I expect. I've attached a screenshot of the sample program.  So maybe this has something to do with the AmazonS3Client Constructor Node??  It just has to get warmed up??  In any case, I can probably live with a one time initialization time of 4 seconds.    

Thanks again,

Pat

Message 4 of 7
(3,034 Views)

Hello Pat,

 

I am glad that you have been able to configure your program in a satisfactory way.  It certainly seems like some point in the connection with the Amazon servers is causing the delay, and once the connection is established you are able to Put data much more quickly.  Unfortunately, I haven't been able to find any resources relating to using LabVIEW with the Amazon S3.  I was able to find a reference to an Amazon S3 Toolkit for LabVIEW, but it has been in beta testing for quite some time, making it difficult to comment on when (or if) that software will be released.

 

You mentioned that you are able to do it much more quickly with your subVIs.  Is there a difference in how you code these VIs?  Could you potentially call them as your Put and Get functions, and if you can, do they still run quickly?  Perhaps there is some way to integrate those subVIs with your larger program to increase the speed of execution.

 

Regards

Patrick
CLA
0 Kudos
Message 5 of 7
(3,005 Views)

Hi guys,

 

I just wanted to let you know, that LabVIEW Interface for Amazon S3 has been finally released and available for install in VIPM package list.

LVTN page is planned to be released on the 15th of December, 2012.

 

Regards,

 

Peter

0 Kudos
Message 6 of 7
(2,961 Views)


AWS provides all the familiar tooling and integrations .NET developers expect, enabling .NET developers to leverage the most innovative cloud with a short learning
curve. AWS supports everything you need to build and run Windows applications such as Active Directory, Microsoft SQL Server, and Windows File Server

After getting your answer i tryied to configure program it work.It seems like some point to connect with the AWS is causing the delay, and once the connection is
established and able to Put data much more quickly and work fastly. But I was facing issue relating LabVIEW while using it with the Amazon S3. What should I do I don't understand.If you have any suggestion then share.

0 Kudos
Message 7 of 7
(2,159 Views)