Working With Python Sandbox

Hi Andy

In the past we have explored adding some integration with Python, and we continue to explore that possibility. There are many different ways that integration could be implemented and so far the customers requesting Python integration have not been requesting the same functionality, so there hasn't been an obvious place for us to start. Thanks for giving us your idea. Others reading this who are interested in some kind of Python support are welcome to give us their own use cases, either on the forums or directly to support@wavemetrics.com. The more details we have the better.

ExecuteScriptText launches using Igor's working directory as the current directory. I believe this is true on both platforms, though I only tested Windows. Igor's working directory is usually the Igor Pro X Folder, however if Igor was launched from the command line it will be the current directory in the command shell when Igor was launched.

Would the following work for you:

1. In Igor, call SetEnvironmentVariable to set IGOR_PYTHON_SCRIPT_DIR (or choose your own name) to the native path you want your Python script to put data in. If necessary, you can use ParseFilePath to convert an Igor path into a native path.

2. In Igor, use ExecuteScriptText/B to start your Python script in the background. Your Python script would need to fetch the value of IGOR_PYTHON_SCRIPT_DIR and use it as appropriate.

At this point your Python script is running in a separate process and you need to figure out a way for it to communicate back to Igor the fact that it has finished (or that there was an error of some kinid). One solution would be to use the ZeroMQ XOP (https://github.com/AllenInstitute/ZeroMQ-XOP). This could be used to send progress information or even the actual data, though it requires serialization of the data so you might get better performance saving the data to a file.

Another approach would be to use a disk file that contains that progress information. If your Python script and Igor agree on the file name (they already both know about IGOR_PYTHON_SCRIPT_DIR, and you could create a similar environment variable for a file name or just use a fixed value), your Python process could write a value into the file to indicate that it has started and a different value to indicate that it has finished. It could even write progress information (0-100 say) that could be displayed in Igor.

After Igor calls ExecuteScriptText to start the Python script, it would then set up a background task to monitor for completion of the process using one of the mechanisms described above. Upon completion Igor would load in the data from the files and do something with it.

Take a look at the Example Experiments->Programming->Watch Folder demo experiment for an example of watching a folder for data created by another process.

From the outside looking in ...

* This will return your python installations on macOS.

​Function get_pythons()
	string theBCmd = "do shell script ", theCmd
	string thePython = "which python"
	sprintf theCmd, "%s \"%s\"", theBCmd, thePython
	try
		ExecuteScriptText/B/Z theCmd
		print S_value
	catch
		print "No python"
	endtry
	thePython = "which python3"
	sprintf theCmd, "%s \"%s\"", theBCmd, thePython
	try
		ExecuteScriptText/B/Z theCmd
		print S_value
	catch
		print "No python3"
	endtry
	return 0
end

* On macOS, Igor Pro runs with python3 from /usr/bin. My preferred python3 install is however in /usr/local/bin. Having the ability to have a flag to change the directory is therefore appreciated.

* As a user only at an intermediate level in programming at best, I would prefer not to have to include and invoke an XOP to get return values from a python script running in the background. My preference would be that, should a user wish to query the status of a python process that is invoked by Igor Pro to run in the background, they should be advised to have the python code (periodically as needed) output a variable and string with the current status. The variable and the string should have defined names (and defined locations) e.g. IGOR_PYTHON_STATUS_VAR and IGOR_PYTHON_STATUS_STR. The status variable convention might be -1 (done with error), 0 (running), 1 (done with no error). The string could contain anything in status (not data output).

For those finding this thread in the future, Igor 10 has added Python integration (Python Overview | Igor Pro). Python scripts can be called from Igor procedure code using the PythonFile and Python operations. These operations support returning Python objects and arrays back to Igor with the var and array keywords.

I've just upgraded to IP10 and my first impressions are favourable. It's good that one can the installation can be configured to use both conda and virtualenv environments.

 However, it would be nice to:

1. be able to use multiple var and array keywords.
2. have an easier way of constructing strings in Igor. At the moment strings are typically constructed via sprintf, or addition (num2str(num1) + " " + str2 + " " + num2str(num3)). It would be nice to have an f-string approach in igor. e.g. f"{num1} {str2} {num3}". The reason this is useful is because the executable strings sent to python/pythonfile can be quite long.

Andy,

Thanks for the feedback, and we hear you on both counts. Both (1) and (2) come down to working within the constraints of Igor's compiler. There are syntaxes that we could adopt for (1) to be allowed, but it would probably have to involve different keywords than 'var'/'array' (maybe just 'vars'/'arrays') since 'var'/'array' are already established. You can of course make multiple, separate calls to 'Python var = {}' to return as many variables as you like, but I understand that isn't quite what you meant.

For (2), there has been some internal discussion about wanting an f-string type of function, which would be very useful for Python as you say, but also for making string construction more composable in general. There is a bit of compiler infrastructure that needs to be developed first to make this kind of function feasible, since it would need to accept an arbitrary number of arguments of various types. Some of that work is already being done, so it is on our radar.
 

I also got a situation with the release version yesterday where the `igorpro` module was not findable (can't remember the exact message). This happened after I nuked and rebuilt the conda environment that the Igor python setting was pointing to (python3.dll). I tried removing and restoring the IP python setting, but that didn't work. The thing that did work was installing the latest IP10 nightly over the top of it.

There were some problems with how Igor was recovering from failed Python initializations that were present in the initial Igor 10 release, but were fixed in a nightly build at some point afterwards. It's possible that's what you ran into. Otherwise, if your updated conda environment happened to be using Python 3.14, this would fail on the initial Igor 10 release since we didn't support 3.14 yet (it hadn't been released officially). Later nightly builds updated our support to include 3.14 though, which maybe is why updating Igor fixed your problem. Glad it's working for you in any case.