Python code fails to run when called by TestStand 2019

[bruno.rodrigues]

I am new to TestStand and getting some experience with Python.

In any case, I'm working on making an already functional code to run from TestStand. That code is suppose to interact with some monitor/display/HIM-device connected through the network interface.

Basically there is a module for each test case that should be writed by user. The module provide instructions for the test case and will call the tool's modules to accomplish it.

I managed to adapt it into a object shape and  it works just like the regular code. Now the code runs through an object with two methods: __init__ and run().

In TestStand there are two steps: create instance and run.

While running over TestStand some issues could be solved at the beginning of __init__ :

Current Work Directory is not the some as the Python file but actually the Sequence file. os.chdir() solved this matter.

There is no argv argument on sys module. This was solved by creating and declaring the value at the start of __init__.

While running create instance step after those changes Python seems to be running as usual on console. When I call the run step the object method run() starts and then it crashes at some point of the run.

I couldn't really debug what's going on because that only occurs at TS but before the crash the console shows "Exception occured in TaskExecLoop() : O sistema nÒo pode encontrar o arquivo especificado." and after that a traceback message in red.

The message translate to "TaskExecLoop() :  The system cannot find the file specified."

By now I could find out that SSH sessions are running fine but when I run a VNC portion of the code it breaks. I believe error is happening after VNC do:

self.p = multiprocessing.Process(target = _start_connection_method)

self.p.start()

After some analysis, as I can see, the exception occurs on p.start() before _start_connection_method begin.

So far I have no idea on how to solve it. If there is a limitation with this process creation on TestStand, or some system variable to be changed in order to run as usual. Any idea will be great.

Thanks on advance!

[bruno.rodrigues]

Hey guys, in case there's someone out there!

After few hours coding I was able to write a simple code that will cause the same issue. Running directly from python should be done without problema although.

I'm still not sure why is that but at least I have something to start and share:

from time import sleep
from multiprocessing import Process
import multiprocessing
import sys

#Simple function that will print out a counting
def func(name, count):
    for n in range(count):
        print("{} say its {}".format(name, n))
        sleep(1)
    print("{} say HERE I GO".format(name))
    sleep(1)
    return

#Object that prepair and two processes with name and count
class proc_call:
    #init set variables in the object as it should
    def __init__(self):
        #path of the running code it's hard coded (you need to adjust to the actual path to this file) to fix a TestStand issue
        file_path = file_path = 'C:/Users/bruno/path/to/file/process.py'
        #create argv argument in sys in case there is none. It happens on TestStand.
        if not hasattr(sys, 'argv'):
            sys.argv = [file_path]

        print(sys.argv)
        #Finally create the processes
        self.p1 = Process(target=func, args=('Marie',4))
        print("p1 created!")
        self.p2 = Process(target=func, args=('Hanzel',12))
        print("p2 created")

    def start(self):
        #Start processes
        self.p1.start()
        self.p2.start()

#This will run in case you call it from python
if __name__ == '__main__':
    a = proc_call()
    a.start()

In case you run the code from python you might i find those results:

"C:\Program Files (x86)\Python35-32\python.exe" "C:/Users/-----/----------/-----/TestStand/Processes_case/process.py"
['C:/Users/------/---------/-----/TestStand/Processes_case/process.py']
p1 created!
p2 created
Marie say its 0
Hanzel say its 0
Marie say its 1
Hanzel say its 1
Marie say its 2
Hanzel say its 2
Marie say its 3
Hanzel say its 3
Marie say HERE I GO
Hanzel say its 4
Hanzel say its 5
Hanzel say its 6
Hanzel say its 7
Hanzel say its 8
Hanzel say its 9
Hanzel say its 10
Hanzel say its 11
Hanzel say HERE I GO

Process finished with exit code 0

 If called from TS with the following steps:

The console will show this:

[Shashidhar]

multiprocessing module in python, by default, use another instance of the same executable to start new process.

So, if you use python.exe to execute your file, multiprocessing.Process will use python.exe. Similarly, if you use TestStand to execute your file, multiprocessing.Process will use TestStand's executable.

Can you try by explicitly setting the option to use python.exe when creating new process from multiprocessing module? You can do that by using the following code before creating Process in the code:

multiprocessing.set_executable(os.path.join(sys.exec_prefix, 'python.exe'))

-Shashidhar

[bruno.rodrigues]

Shashidhar, thank for your reply!

I tried your suggestion and it seems to be part of the solution. I did some minor changes on the dummy code aswell.

This is what it look like by now:

import os
from time import sleep
from multiprocessing import Process
import multiprocessing
import sys


#Object that prepair and two processes with name and count
class proc_call:

    # Simple function that will print out a counting
    def func(self, name, count):
        for n in range(count):
            print("{} say its {}".format(name, n))
            sleep(1)
        print("{} say HERE I GO".format(name))
        sleep(1)
        return

    #init set variables in the object as it should
    def __init__(self):
        #path of the running code it's hard coded (you need to adjust to the actual path to this file) to fix a TestStand issue
        file_path = 'C:\\Users\\------\\TestStand\\Processes_case\\process.py'
        #change the cwd to the usual one when the file is running
        file_dir_path = os.path.dirname(file_path)
        os.chdir(file_dir_path)

        #create argv argument in sys in case there is none. It happens on TestStand.
        if not hasattr(sys, 'argv'):
            sys.argv = [file_path]

        multiprocessing.set_executable(os.path.join(sys.exec_prefix, 'python.exe'))

        print(sys.argv)
        #Finally create the processes
        self.p1 = Process(target=self.func, args=('Marie',4))
        print("p1 created!")
        self.p2 = Process(target=self.func, args=('Hanzel',12))
        print("p2 created")

    def start(self):
        #Start processes
        self.p1.start()
        self.p2.start()

#This will run in case you call it from python
if __name__ == '__main__':
    a = proc_call()
    a.start()

    #Add some sleep to avoid main finishing before the processes
    sleep(15)
    print("Main has finished")

Once I run that code outside TS in run as usual just announcing when main is finished. When launched from TS console shows the following:

['C:\\Users\\------\\TestStand\\Processes_case\\process.py']
p1 created!
p2 created
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Program Files (x86)\Python35-32\lib\multiprocessing\spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "C:\Program Files (x86)\Python35-32\lib\multiprocessing\spawn.py", line 116, in _main
self = pickle.load(from_parent)
ImportError: No module named 'process_test'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Program Files (x86)\Python35-32\lib\multiprocessing\spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "C:\Program Files (x86)\Python35-32\lib\multiprocessing\spawn.py", line 116, in _main
self = pickle.load(from_parent)
ImportError: No module named 'process_test'

 Couldn't figure out what is going on here yet.

Any advices?

[Shashidhar]

Import error will occur when python is unable to find the module it needs to use on creating new process.

One of the solution to ensure python is able to find the file is by updating sys.path. sys.path contains list of search directories for importing module. Use the following code to add the file path to sys.path:

sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))

-Shashidhar

[bruno.rodrigues]

Still having the same problem here. Keeps saying "Import Error: No module named 'process'."

I've checked if the values printing them out on console and the values check. Before the change sys.path didn't have our file directory path, but once I add your suggestion the first value checks the actual directory path to process.py.

Couldn't say why is this happening.

Edit:

I copied my process.py to the directory I set on PYTHONPATH on environment variables and code is running.

Anyway, there must be a better way to do it without duplicating files.

[bruno.rodrigues]

My guess right now is that p1 and p2 doesn't include file folder path, just the initial sys.path from TS execution. In that case it still can find files on PYTHONPATH, site-packages and other.

There must be a way to include it on process instance.

[bruno.rodrigues · Accepted solution]

Shashidhar, found out how to fix the issue. It happens that sys.path turn back to default in each step I guess.

I've printed out sys.path on __init__ after the inserting the path to file folder.

Then I printed out sys.path at the begining of function start() called by second step on TS.

sys.path1:
['c:\\users\\bruno\\path_to_file\\teststand\\processes_case', 'C:\\Program Files (x86)\\National Instruments\\TestStand 2019\\Bin', 'C:\\Program Files (x86)\\National Instruments\\Shared\\NIPythonInterface\\.\\', 'C:\\SVN\\tools', 'C:\\Program Files (x86)\\Python35-32\\python35.zip', 'C:\\Program Files (x86)\\Python35-32\\Lib', 'C:\\Program Files (x86)\\Python35-32\\DLLs', 'C:\\Program Files (x86)\\National Instruments\\Shared\\NIPythonInterface', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages\\win32', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages\\win32\\lib', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages\\Pythonwin', 'C:\\Program Files (x86)\\Python35-32', 'C:\\Program Files (x86)\\Python35-32\\lib\\site-packages', 'C:\\Program Files (x86)\\Python35-32\\lib\\site-packages\\pywinauto-0.6.2-py3.5.egg']


p1 created!
p2 created


sys.path2:
['C:\\Program Files (x86)\\National Instruments\\TestStand 2019\\Bin', 'C:\\Program Files (x86)\\National Instruments\\Shared\\NIPythonInterface\\.\\', 'C:\\SVN\\tools', 'C:\\Program Files (x86)\\Python35-32\\python35.zip', 'C:\\Program Files (x86)\\Python35-32\\Lib', 'C:\\Program Files (x86)\\Python35-32\\DLLs', 'C:\\Program Files (x86)\\National Instruments\\Shared\\NIPythonInterface', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages\\win32', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages\\win32\\lib', 'C:\\Users\\bruno\\AppData\\Roaming\\Python\\Python35\\site-packages\\Pythonwin', 'C:\\Program Files (x86)\\Python35-32', 'C:\\Program Files (x86)\\Python35-32\\lib\\site-packages', 'C:\\Program Files (x86)\\Python35-32\\lib\\site-packages\\pywinauto-0.6.2-py3.5.egg']

As you may notice sys.path did not include file folder path when second step was called. So to fix that you may have to insert the path at the beginning of each step.

Code end up like this:

import os
from time import sleep
from multiprocessing import Process
import multiprocessing
import sys



#Object that prepair and two processes with name and count
class proc_call:

    # Simple function that will print out a counting
    def func(self, name, count):
        for n in range(count):
            print("{} say its {}".format(name, n))
            sleep(1)
        print("{} say HERE I GO".format(name))
        sleep(1)
        return

    #init set variables in the object as it should
    def __init__(self):
        #path of the running code it's hard coded (you need to adjust to the actual path to this file) to fix a TestStand issue


        #Set CWD to python file dir
        os.chdir(os.path.dirname(os.path.realpath(__file__)))

        #create argv argument in sys in case there is none. It happens on TestStand.
        if not hasattr(sys, 'argv'):
            sys.argv = [__file__]

        #Set multiprocessing to python
        multiprocessing.set_executable(os.path.join(sys.exec_prefix, 'python.exe'))

        # include python file path to python path
        sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))

        #Finally create the processes
        self.p1 = Process(target=self.func, args=('Marie',4))
        print("p1 created!")
        self.p2 = Process(target=self.func, args=('Hanzel',12))
        print("p2 created")

    def start(self):
        sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))

        #Start processes
        self.p1.start()
        self.p2.start()

#This will run in case you call it from python
if __name__ == '__main__':
    a = proc_call()
    a.start()
    #Add a sleep time to avoid parent processes finishing before child processes
    sleep(15)
    print("Main has finished")

And console finally end up like this:

p1 created!
p2 created
Marie say its 0
Hanzel say its 0
Marie say its 1
Hanzel say its 1
Marie say its 2
Hanzel say its 2
Marie say its 3
Hanzel say its 3
Marie say HERE I GO
Hanzel say its 4
Hanzel say its 5
Hanzel say its 6
Hanzel say its 7
Hanzel say its 8
Hanzel say its 9
Hanzel say its 10
Hanzel say its 11
Hanzel say HERE I GO  

 Thank you for the help on this issue!