NumPy on PyPy progress report
Hello.
A lot of things happened in March, like pycon. I was also busy doing other things (pictured), so apologies for the late numpy status update.
However, a lot of things have happened and numpy continues to be one of the main points of entry for hacking on PyPy. Apologies to all the people whose patches I don't review in timely manner, but seriously, you do a lot of work.
This list of changes is definitely not exhaustive, and I might be forgetting important contributions. In a loose order:
-
Matti Picus made out parameter work for a lot of (but not all) functions.
-
We merged record dtypes support. The only missing dtypes left are complex (important), datetime (less important) and object (which will probably never be implemented because it makes very little sense and is a mess with moving GCs).
-
Taavi Burns and others implemented lots of details, including lots of ufuncs. On the completely unscientific measure of "implemented functions" on numpypy status page, we're close to 50% of numpy working. In reality it might be more or less, but after complex dtypes we're getting very close to running real programs.
-
Bool indexing of arrays of the same size should work, leaving only arrays-of-ints indexing as the last missing element of fancy indexing.
-
I did some very early experiments on SSE. This work is seriously preliminary - in fact the only implemented operation is addition of float single-dimension numpy arrays. However, results are encouraging, given that our assembler generator is far from ideal:
Numpy
PyPy SSE
PyPy
GCC non-looped
GCC looped
a+b
0.6s
0.3s
0.4s
0.3s
0.25s
a+b+c
1.9s
0.35s
0.5s
0.7s
0.32s
a+b+c+d+e
3.2s
0.36s
0.8s
1.7s
0.51s
The benchmark repo is available. GCC was run with -O3, no further options specified. PyPy was run with default options, the SSE branch is under backend-vector-ops, but it's not working completely yet.
One might argue that C and Python is not the same code - indeed it is not. It just shows some possible approach to writing numeric code.
Next step would be to just continue implementing missing features such as
- specialised arrays i.e. masked arrays and matrixes
- core modules such as fft, linalg, random.
- numpy's testing framework
The future is hard to predict, but we're not far off!
Cheers,
fijal
UPDATE:Indeed, string and unicode dtypes are not supported yet. They're as important as complex dtype
Comments
I think the string dtype is missing too?
Hello,
May you get a bit more precise on the GCC test ?
For instance, is the GCC code using SSE too ? Is it written in a single loop (x[i] = a[i] + b[i] + c[i]) or in several consecutive loops first a+b then (a+b) + c ?
Just to know :-)
One thing I'll note is that I do from time to time use the object dtype. Occasionally, I've got multidimensional arrays of objects, and the array operations from numpy are useful. I don't really get a speed advantage there, but the interface from numpy is useful. But its not super necessary and certainly not a priority.
Sorry, didn't RTFA completely. I just had a look at the C code.
Still, a question: is PyPy doing the optimization of combining operations in one step ?
A "good" Fortran compiler should be able to do those optimizations, for instance.
You should compare to numpy with a JIT, such as numexpr, it would be interesting to see whether PyPy is able to beat the numexpr JIT.
Very cool!
"busy doing other things (pictured)". Pictured where? :-)
Hi, Numpy masked arrays, matrices and the testing framework are pure Python, so why do you need to implement them?
Ralf, we don't have to implement the pure-python stuff, so much as we need to make sure the features of NumPy's core that they depend on are implemented.
Support for objects is actually quite useful: please reconsider adding it.
Here is a very useful case: the manipulation of arrays of numbers with uncertainties (special uncertainties.UFloat objects). Numbers with uncertainties behave very much like regular numbers: it is very useful to be able to use the regular NumPy syntax for array operations, for calculating matrix inverses when the matrices contain number with uncertainties, etc. I know many people use these features.
It would be *great* (read: irreplaceable :) to have support for the object NumPy dtype.
This sounds really cool!
And it would be awesome if you’d manage to coordinate with numpy, so the projects merge to a single python codebase with two separate backends: One C-Extension based for CPython and one Pure-Python based for pypy.
Any chance comparing with Fortran? There are assumptions about pointers and alignment that Fortran compiler can make.
Nice...but what is the next step?
Numpy alone is not that useful.
"We" need at least scipy and matplotlib.
Are you going to port all these modules? I don't think so.
One way forward could be to have numpy in pypy and at least scipy and matplotlib working with the pypy C api at a decent speed.
What do you think?
What about pickling? I'd love to experiment with hybrid CPython/PyPy execution using some magic from the multiprocessing module or a similar parallel computation framework.
Hello,
This is a very promising result, thank you for sharing it.
Could you give a few more details about the differences wrt to numpy?
What would people have to do to use numpypy with scipy?
I think the numpy.linalg module is pretty important.
How to move efforts into this?
I think the numpy.linalg module is pretty important.
How to move efforts into this?