Working With Python Sandbox

Hi,

This topic has been talked about for well over a decade and workarounds have been proposed.  What I think might be useful is dedicated set of functionality to execute a script and get back data though this would be via files and not direct reading of data.  Some of my interest is in handling cross platform issues.

The gist of the idea, is since a python script in reality is just text, it could be loaded into an IP experiment. There could be some defined directories that the script would execute in to facilitate exchanging data between IP and Python and the IP function would include setting that directory before executing independent of platform. The other functionality would be a flag to indicate completion of the script and ready for reading back into IP.  Sort of a sandbox for executing the script in a known location with the added functionality of handling completion flags.

This would allow the minimum scripting necessary within Python and free up the need to manage different paths and the like especially cross platform since the execute function does not guarantee a known location when a script is launched.

Andy

Hi Andy

In the past we have explored adding some integration with Python, and we continue to explore that possibility. There are many different ways that integration could be implemented and so far the customers requesting Python integration have not been requesting the same functionality, so there hasn't been an obvious place for us to start. Thanks for giving us your idea. Others reading this who are interested in some kind of Python support are welcome to give us their own use cases, either on the forums or directly to support@wavemetrics.com. The more details we have the better.

ExecuteScriptText launches using Igor's working directory as the current directory. I believe this is true on both platforms, though I only tested Windows. Igor's working directory is usually the Igor Pro X Folder, however if Igor was launched from the command line it will be the current directory in the command shell when Igor was launched.

Would the following work for you:

1. In Igor, call SetEnvironmentVariable to set IGOR_PYTHON_SCRIPT_DIR (or choose your own name) to the native path you want your Python script to put data in. If necessary, you can use ParseFilePath to convert an Igor path into a native path.

2. In Igor, use ExecuteScriptText/B to start your Python script in the background. Your Python script would need to fetch the value of IGOR_PYTHON_SCRIPT_DIR and use it as appropriate.

At this point your Python script is running in a separate process and you need to figure out a way for it to communicate back to Igor the fact that it has finished (or that there was an error of some kinid). One solution would be to use the ZeroMQ XOP (https://github.com/AllenInstitute/ZeroMQ-XOP). This could be used to send progress information or even the actual data, though it requires serialization of the data so you might get better performance saving the data to a file.

Another approach would be to use a disk file that contains that progress information. If your Python script and Igor agree on the file name (they already both know about IGOR_PYTHON_SCRIPT_DIR, and you could create a similar environment variable for a file name or just use a fixed value), your Python process could write a value into the file to indicate that it has started and a different value to indicate that it has finished. It could even write progress information (0-100 say) that could be displayed in Igor.

After Igor calls ExecuteScriptText to start the Python script, it would then set up a background task to monitor for completion of the process using one of the mechanisms described above. Upon completion Igor would load in the data from the files and do something with it.

Take a look at the Example Experiments->Programming->Watch Folder demo experiment for an example of watching a folder for data created by another process.

From the outside looking in ...

* This will return your python installations on macOS.

Function get_pythons()
    string theBCmd = "do shell script ", theCmd
    string thePython = "which python"
    sprintf theCmd, "%s \"%s\"", theBCmd, thePython
    try
        ExecuteScriptText/B/Z theCmd
        print S_value
    catch
        print "No python"
    endtry
    thePython = "which python3"
    sprintf theCmd, "%s \"%s\"", theBCmd, thePython
    try
        ExecuteScriptText/B/Z theCmd
        print S_value
    catch
        print "No python3"
    endtry
    return 0
end

* On macOS, Igor Pro runs with python3 from /usr/bin. My preferred python3 install is however in /usr/local/bin. Having the ability to have a flag to change the directory is therefore appreciated.

* As a user only at an intermediate level in programming at best, I would prefer not to have to include and invoke an XOP to get return values from a python script running in the background. My preference would be that, should a user wish to query the status of a python process that is invoked by Igor Pro to run in the background, they should be advised to have the python code (periodically as needed) output a variable and string with the current status. The variable and the string should have defined names (and defined locations) e.g. IGOR_PYTHON_STATUS_VAR and IGOR_PYTHON_STATUS_STR. The status variable convention might be -1 (done with error), 0 (running), 1 (done with no error). The string could contain anything in status (not data output).