[pymex] Or you could just use Jython...

I recently discovered an alternative to pymex: using Jython inside the Matlab interpreter. Jython is an implementation of the Python interpreter in Java. The Matlab interpreter allows the importing of Java objects, so all a programmer needs to do is import the Jython interpreter. Here is an example:

>> javaaddpath('C:\Jython25\jython.jar')
>> import org.python.util.PythonInterpreter
>> import org.python.core.*
>> interp = PythonInterpreter()
 
interp =
 
org.python.util.PythonInterpreter@22ff0470
 
>> interp.exec('import sys')
>> interp.exec('print sys')


>> interp.set('a', 42)
>> interp.exec('print a')
42.0

>> a2 = interp.get('a')
 
a2 =
 
42.0

Jython is really cool, but it does have a drawback. Modules written in C for CPython are not compatible with Jython. That excludes alot of modules, but that may not bother some people. If you do need to use a CPython module, you can use pymex.

[pymex] Threading

Threading didn't really work in the first version of pymex. You could create a Python thread, but it wouldn't actually get any execution time. I recently found out why, and fixed it!

Python has limited support for multi-threading. The Python interpreter is not fully thread-safe, and something called a global interpreter lock (GIL) is used as a sort of mutex on the interpreter. Only the thread which has acquired the GIL can interact with Python objects or use the Python C API. The negative side-effect of the GIL is that Python threads run more-or-less contiguously, instead of concurrently. Thus, Python threads can't take advantage of parallel hardware.

By default, Python runs in one main thread and the GIL is not initialized. Threads and the GIL are automatically initialized when a Python thread is created. Python automatically shares the GIL between threads so each thread can execute. This is accomplished by threads yielding on certain functions like I/O, sleep, etc. This doesn't happen for C extensions, they must explicitly release the GIL using the C API.

The old version of pymex would never release the GIL. The GIL would still get released, but only while the Python interpreter was running (during a call to PyRun_SimpleString). This meant that threads would only get CPU time during calls to pymex, and not during normal Matlab execution. Python scripts which joined their threads before ending didn't have a problem, but daemonized threads didn't work at all.

The fix was actually very simple. All I had to do was release the GIL using the C API before returning from the mex function, and reacquire it on reentry. This was done using PyEval_SaveThread and PyEval_RestoreThread. I also decided to release the GIL inside the matlab module during various (slow) mex calls, like mexPrintf. I did this to hopefully increase Python's threading performance.

But there was one drawback: added complexity. Programmers now need to consider thread-safety. Unfortunately, Matlab's mex interface isn't thread-safe. By extension, the matlab module isn't thread-safe. Don't use the matlab module inside a thread, it is liable to segfault! There is an exception to this rule. It is safe to use the matlab module in a thread as long as it ends before the script ends (i.e. the thread is joined). That way the underlying mex calls only occur during the call to pymex, so memory access violations are avoided.

Check out the pymex website for more details!

[pymex] Source Code Released

I've added a few things since my last post, and everything seems to be working properly, so I'm going to call this version 0.1. To learn more and/or get the source code, visit the pymex website.

http://vader.cse.lehigh.edu/~perkins/pymex.html

In case you didn't know, pymex mixes Python and Matlab by embedding a Python interpreter in Matlab. pymex also allows programmers to transfer data between Python and Matlab, this is accomplished by extending Python with a special "matlab" module.

You can find pymex at Mathwork's File Exchange.

[pymex] Python in Matlab

So I've been pretty busy on a few different things. One of those things happens to be pymex. I often need to extend Matlab to do something

To extend Matlab's capabilities it's necessary to write mex files in C/C++. Python is quick to code and it already has a bunch of available modules. With pymex a programmer can embed Python code inside of Matlab and avoid the whole mex file thing.

As an added benifit I also wrote a python module called 'matlab'. This module does two things. First, it provides a simple print function which prints to Matlab's command window (like mexPrintf). Second, it allows Python variables to be 'pushed' to Matlab.

matlab.mex_print(str, ...)
'''Like Pythons built-in print function, takes a variable
length of objects, converts them to strings using str(),
and prints the result to the Matlab command window.'''

matlab.push(varname, var, ...)
'''Using the Matlab function 'assignin', creates a Matlab
variable in the caller's scope from 'var' named 'varname'.
The 'varname' must be a valid Matlab variable name.
The 'var' must be a scalar; lists, tuples, and objects are
not supported yet.'''

Here's a sample script to show what pymex can do.

pyscript = sprintf([...
'import matlab\n', ...
'x = 7\n', ...
'for i in range(x):\n', ...
'    matlab.mex_print(i)\n', ...
'    matlab.push("i_%%d" %% i, i)\n', ...
'matlab.mex_print("demo complete!")\n', ...
]);

pymex(pyscript);