What device are you trying to communicate with? Do you have a link to its manual?
Have you tried connecting to it with the MB Serial Master Query .vi in the Modbus library?
Are you sure you want to use ASCII mode? Try it with RTU mode. I've communicated with a handful of devices with modbus, and they have all used RTU. I have yet to see something that uses ASCII.