There are online hash generators that will allow you to check the validity of your algorithm, such as "http://www.tools4noobs.com/online_tools/hash/", which provides several hashing routines.
Next time you upload your VI, ensure that all SubVIs are also uploaded! I can't run your VI since I don't have "rsa encryption.vi" and "hash.vi".