Community: Framework mailing list archives

Re: PyPy

- 12/11/2014 16:58:58
I'm listing my findings for this evening in here:

- Installing lxml using pip in a virtualenv with PyPy is a PITA on
Fedora 20. Had to use some export CFLAGS="-I ..." for the libxml and
libxslt libraries.
- The guys on IRC #pypy tell me there is a lxml-cffi available (after
bouncing on several segfaults [1] with the default one), which is
preferred over default lxml.
$ pip install -e git+git://
should do the job of installing it.

Thereafter things started to run:
The server started without a lot of problems (except begging for more
modules). It didn't want to initialize the database, I'll investigate
that on another moment.

Then I copied over my production server odoo files and I set up an SSH
tunnel for postgresql.
I can't login on existing databases: [2]

Now it's generating a new database (over SSH, slow!) using the files
copied from the production server. It's saying "You may not believe it,
but the application is actually loading". No issues in my terminal
window so far. I'll keep you posted. As it's too late for coffee ("Take
a minute to get a coffee, because it's still loading..."), I'll call it
a night and post my next findings tomorrow.

EDIT: Wow, almost went to bed. It gives the same error as the
new-install now [3]. Apparently, some XML-id cannot be found. Would this
be a bug in lxml?

Good night!



On 11/12/14 15:57, Raphaël Valyi wrote:
> Hello Ruben,
> some 2 months ago I tried to run Odoo v8 on PyPy again following the hints from Sandy. Basically the old style Postgresql cursor has been changed in v8 and there was no need for any hack here anymore.
> That being said, it was still crashing(segfault I believe) badly in fields_view_get when doing complex things with lxml (C extension). That would work from some trivial views but crash in any complex views.
> In fact, several Python projects using lxml have similar issues on PyPy it seems.
> Amazingly back to 2009 when Oracle didn't had totally dis-moralized the Java world yet and when Jython was still alive (I mean really), I managed to get to the same point getting OpenERP running on Jython (there is even an old Launchpad branch) but that was crashing exactly on the same point.
> Possibly, lxml is using low level C Python API in a way that is not possible with alternative implementations. So for instance, lxml now "officially" supports PyPy but mention that they use a different implementation: weakrefs instead of back pointers: [1]
> This kind of known lxml issues on PyPy may very well be related with our trouble with Odoo [2]
> I'm not sure if the micro-benchmarks benefits will hold true on real world cases like Odoo, but still if anyone manage to makes its way in the lxml portability pitfalls, I'm curious to know.
> Best regards.
> On Thu, Dec 11, 2014 at 7:57 AM, Ruben De Smet < [3] > wrote:
> Hello Odoo developers
> I was reading some things on PyPy, the JIT python interpreter and I
> thought this could be something very interesting for Odoo, to reduce
> memory usage and CPU time.
> According to this [1] (simple) benchmark, PyPy cuts memory usage in half
> and reduces runtime to one third. This counts for memory intensive
> applications and I think Odoo is among those.
> Think of it: cutting memory usage and cpu usage in half means you can
> double the amount of Odoo instances run on one server and therefore
> could cut the TCO in half, especially for large Odoo installations.
> According to this [2] article written by an Odoo developer, OpenERP
> 8-trunk ran in PyPy back then, using some minor changes.
> Caveats included
> 1) Using psycop2cffi instead of psycop2 because it's faster and more
> elegant (and psycop2 doesn't work using pypy). That means that the
> (legacy) psycopg1cursor calls had to be replaced by something else which
> is explained in the article.
> 2) There was a segfault (with regard to unicode) in psycop2cffi which
> was fixed by him [3]
> 3) Another segfault: "After a bit of digging, it would seem the segfault
> happens sometimes when calling the function on line 923 of
> openerp-server/openerp/tools/"
> I haven't tried anything regarding PyPy myself yet, but I'm planning to
> in the near future.
> For 1) I'd suggest I take a look if those call can be replaced by the
> newer (non-legacy) variant. Regarding to the article, "and this should
> serve as note to core developers to drop the psycopg1.cursor and use the
> DictCursor in the psycopg.extras modules which implements a much better
> and faster algorithm."
> 2) Has been fixed already
> For 3) I'd suggest the same. I'll try out Odoo on PyPy, list the caveats
> and try to produce a smooth fix, be it in Odoo, in PyPy or in some
> project else.
> My questions to this list:
> - Are there people, other than me, interested in using Odoo on PyPy?
> - Could, once tested and stable, PyPy become officially supported? I'm
> willing to take the testing and developing on me.
> - Would Odoo accept patches (on github) to make Odoo work on PyPy?
> In any case, I'll report back my findings (being caveats and performance
> statistics, if I can gather those) when trying this out.
> Thank you for reading through! Any feedback is welcome
> Ruben De Smet
> [1]
> [4]
> [2]
> [5]
> [3]
> [6]
> _______________________________________________
> Mailing-List: [7]
> Post to: mailto: [8]
> Unsubscribe: [9]
> --
> Raphaël Valyi Founder and consultant [10]   +55 21 3942-2434 [11]
> _______________________________________________
> Mailing-List:
> Post to:
> Unsubscribe:
> [1]
> [2]
> [3]
> [4]
> [5]
> [6]
> [7]
> [8]
> [9]
> [10]!/rvalyi
> [11]
  • PyPy

    - 12/11/2014 04:54:45 - 0

    6 replies 6 replies