Community: Framework mailing list archives

expert-framework@mail.odoo.com

Re: PyPy (it work's !)

by
n.schmitt
- 06/02/2015 17:20:46
Hi all,

I find your endeavours really interesting, it is strange that it has not
raised more support so far.

It would be nice to have the input of the Odoo S.A. representatives on this
topic, knowing that the growing of Odoo would surely require the switch
at some point.

Regards,
Nicolas Schmitt

On 01/06/15, Nicolas PIGANEAU wrote:
> Hi all,
> I have done a few tests to check the speed of my running Odoo on PyPy server vs Odoo on CPython server.
> Tests were made on two docker instances (built on top of official python and pypy docker images)
> pointing to the same database docker. The same database was duplicated, one for each docker instance.
> Server call duration were measured with Firebug.
> Suprisingly, on most common calls PyPy instance was 2-3 times slower than CPython instance (I did
> make a few requests before to warm up the JIT, but it doesn't seem to bring it to the level of
> CPython).
> But on specific calls where CPython is slow, we have a performance improvement from PyPy version.
> Here is an example: On Odoo v8, go to warehouse->all operations and clicking on "ready" operations of
> receipts (I had ~30 ready receipts) gave the following:
> 
> (First column is CPython, second is PyPy) in ms
> load                 123  159
> stock.picking/fields_view_get     31  186
> stock.picking/fields_view_get	   64  357
> get_filters	           16   27
> stock.picking/fields_get       32   81
> stock.picking.type/name_get      19   44
> stock.picking/fields_view_get     71  234
> search_read             2170  934
> Most notable is the last line which repeatedly gave best performance by 2-3 times for PyPy.
> Any idea on which part of Odoo could be slower on PyPy is very welcome !
> Nicolas PIGANEAU
> -------------------
> NDP Systèmes
> 06 68 93 87 56
> http://www.ndp-systemes.fr
> ----- Mail original -----
> De: "Nicolas PIGANEAU" <nicolas.piganeau@ndp-systemes.fr>
> À: "Odoo S.A. Community: Framework" <expert-framework@mail.odoo.com>
> Envoyé: Dimanche 31 Mai 2015 23:08:07
> Objet: Re: PyPy (it work's !)
> Thank you for your answers, Raphaël and Ruben.
> I've got further today, and managed to have my first Odoo server on PyPy running !
> But there is still some way to go.
> As I said before, I came across 2 issues:
> 1. Database creation does not work. After having rapidly inspected the generated database, only the base module has been loaded, although the logs show that all the modules have been loaded. I suppose that there has been a rollback somewhere, but I'll have to check.
> 2. When using an already created database, I had issues with inherited views, as you mentioned. With the fact that only one every other line of the inherited view was inserted inside the base view, I managed to locate the issue, and applying the following patch solves it:
> ===================================================================
> --- openerp/addons/base/ir/ir_ui_view.py	(revision e9f802425cf8ca0f82e7075774585d4ca897ad08)
> +++ openerp/addons/base/ir/ir_ui_view.py	(revision )
> @@ -449,7 +449,7 @@
> if node.getparent() is None:
> source = copy.deepcopy(spec[0])
> else:
> -            for child in spec:
> +            for child in spec[:]:
> node.addprevious(child)
> node.getparent().remove(node)
> elif pos == 'attributes':
> @@ -461,7 +461,7 @@
> del node.attrib[attribute[0]]
> else:
> sib = node.getnext()
> -          for child in spec:
> +          for child in spec[:]:
> if pos == 'inside':
> node.append(child)
> elif pos == 'after':
> Maybe there are other places in the Odoo code where the same sort of modifications should be made, but this was enough to get it running.
> For the few quick tests I made, I get QWebException complaining about void* not being known and giving a code 500. But if you reload several times, you get it running in the end. And when you get the interface, then no more problems.
> Now, for the big question: is it really faster than Odoo in CPython ?
> Well, I did not have time to make a lot of benchmarking, but:
> - The interface seems the same speed as the CPython version, I'll have to make deeper tests to see if the server calls are faster or not.
> - On one specific stock operation (validating a several hundred lines picking) which took me just above 1 minute on CPython server took only 27 seconds on the PyPy server.
> => So it seems to be significantly faster (~x2), at least on operations with a lot of python calculation like stock operations. Now it needs to be confirmed with full benchmark.
> 
> Nicolas PIGANEAU
> -------------------
> NDP Systèmes
> 06 68 93 87 56
> http://www.ndp-systemes.fr
> ----- Mail original -----
> De: "Raphaël Valyi" <rvalyi@akretion.com>
> À: "Community: Framework" <expert-framework@mail.odoo.com>
> Envoyé: Samedi 30 Mai 2015 15:02:09
> Objet: Re: PyPy
> Hello Nicolas,
> a few months ago I tried again and commented here https://www.odoo.com/groups/community-framework-62/community-framework-10089512
> Basically it seems the Pypy lxml implementation differs from the CPython one and eventually this is what breaks the view inheritance system. Also I pioneered tests on Jython back around 2011 and I had the same issues with the Java based lxml adapter not having the exact same internals of the one of CPython. May be it's not this issue anymore but I bet it is. That's interesting to see efforts here however. Let us know if you make any progress on this.
> Regards.
> On Fri, May 29, 2015 at 9:00 PM, Nicolas PIGANEAU < nicolas.piganeau@ndp-systemes.fr > wrote:
> 
> Hi,
> Has anyone tried to run Odoo on PyPy recently ?
> On my side, I used:
> - lxml-cffi from git:// github.com/amauryfa/lxml.git - latest gevent sources (after merging https://github.com/gevent/gevent/pull/552 )
> - psycopg2cffi and psycopg2cffi-compat (pip install)
> I had also to patch Odoo sources to add pypy opcodes to safe_eval:
> --- ../GitHub/odoo/openerp/tools/safe_eval.py	(date 1432802291000)
> +++ ../GitHub/odoo/openerp/tools/safe_eval.py	(date 1432901508000)
> @@ -69,6 +69,7 @@
> _SAFE_OPCODES = _EXPR_OPCODES.union(set(opmap[x] for x in [
> 'LOAD_NAME', 'CALL_FUNCTION', 'COMPARE_OP', 'LOAD_ATTR',
> +  'CALL_METHOD','LOOKUP_METHOD',
> 'STORE_NAME', 'GET_ITER', 'FOR_ITER', 'LIST_APPEND', 'DELETE_NAME',
> 'JUMP_FORWARD', 'JUMP_IF_TRUE', 'JUMP_IF_FALSE', 'JUMP_ABSOLUTE',
> 'MAKE_FUNCTION', 'SLICE+0', 'SLICE+1', 'SLICE+2', 'SLICE+3', 'BREAK_LOOP',
> 
> And also add this one if I wanted to start in multi-worker mode (for some reason pypy complained they were float instead of int):
> --- ../GitHub/odoo/openerp/service/server.py	(date 1432802291000)
> +++ ../GitHub/odoo/openerp/service/server.py	(date 1432901508000)
> @@ -713,7 +713,7 @@
> raise Exception('CPU time limit exceeded.')
> signal.signal(signal.SIGXCPU, time_expired)
> soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
> -    resource.setrlimit(resource.RLIMIT_CPU, (cpu_time + config['limit_time_cpu'], hard))
> +    resource.setrlimit(resource.RLIMIT_CPU, (int(cpu_time + config['limit_time_cpu']), hard))
> def process_work(self):
> pass
> 
> Then my server starts all right and I am able to go to the database manager page.
> Creating a database seems to work, but it is then impossible to load the created database. Indeed, it seems some imports have been skipped.
> In particular, ir_model_data is only filled with xml_ids of the base module.
> So I created a database from a cpython instance and accessed it through my pypy instance. I could login, but then the loading stopped after
> displaying the top menu. A look in firebug showed that JS complained about "nv" not being defined in graph_widget.js.
> So I checked what had been loaded, and found out that only one every other scripts had been loaded from this module, but also from all
> other modules. Same goes for css files: one loaded, one skipped, one loaded, etc. for each line of the assets_backend extension template
> (web_graph/views/web_graph.xml in this case).
> I suppose this an issue linked with lxml, that sort of misses one line every two lines but I haven't got a clue on where to go now.
> Any idea or experience on this?
> Thanks !
> Nicolas PIGANEAU
> -------------------
> NDP Systèmes
> 06 68 93 87 56 http://www.ndp-systemes.fr
> _______________________________________________
> Mailing-List: https://www.odoo.com/groups/community-framework-62
> Post to: mailto: expert-framework@mail.odoo.com
> Unsubscribe: https://www.odoo.com/groups?unsubscribe
> 
> 
> 
> --
> Raphaël Valyi
> Founder and consultant
> http://twitter.com/rvalyi
> +55 21 3942-2434
> www.akretion.com
> 
> _______________________________________________
> Mailing-List: https://www.odoo.com/groups/community-framework-62
> Post to: mailto:expert-framework@mail.odoo.com
> Unsubscribe: https://www.odoo.com/groups?unsubscribe
> _______________________________________________
> Mailing-List: https://www.odoo.com/groups/community-framework-62
> Post to: mailto:expert-framework@mail.odoo.com
> Unsubscribe: https://www.odoo.com/groups?unsubscribe
>