Community: Framework mailing list archives

expert-framework@mail.odoo.com

Re: PyPy (it work's !)

by
陈帆
- 04/10/2016 05:28:46
recently i try to run odoo8 on pypy 5.0.1, it failed and report  segmentation fault.


install ubuntu 14.0.4 LTS , then install postgresql database and git tools. update odoo 8.0 source code from http://github.com/odoo/odoo.git and save code in /opt/odoo/ . download pypy-5.0.1 for Linux 64, and extract it to /opt/pypy/. download get-pip.py save into /tmp/, and change to /opt/pypy/bin, then run ./pypy /tmp/get-pip.py to install pip tool.

because some python lib need to compile from source, so need to install dev tools, and run command sudo apt-get install libxml2 libxslt-dev libpq-dev libldap2-dev libsasl2-dev

change /opt/odoo/requirements.txt as change psycopg2 to psycopg2cffi and add new line as psycopg2cffi-compact, then install python lib via sudo /opt/pypy/bin/pip install -r /opt/odoo/requirements.txt.

now, pypy for odoo is reday. next, should to create database for odoo, and run odoo daemon.

use command sudo -u postgres createuser --createdb --no-createrole --no-superuser --pwprompt odoo to create postgres user odoo and assign a password, such like 1234. now edit odoo.conf file and save in /opt/odoo/, the content of odoo.conf is

[options]
; This is the password that allows database operations:
; admin_passwd = admin
db_host = 127.0.0.1
db_port = 5432
db_user = odoo
db_password = 1234

change to odoo directory, use command /opt/pypy/bin/pypy odoo.py -c odoo.conf to start the odoo service.

notes.

if lxml-3.3.5 is not working, please upgrade it to lxml-3.6.

if odoo raise opcocde LOOKUP_METHOD is not allowed and/or opcocde CALL_METHOD is not allowed, please edit /opt/odoo/openerp/tools/safe_eval.py to add those two opcode into var _SAFE_OPCODES

_SAFE_OPCODES = _EXPR_OPCODES.union(set(opmap[x] for x in [
    'LOAD_NAME', 'CALL_FUNCTION', 'COMPARE_OP', 'LOAD_ATTR',
    'STORE_NAME', 'GET_ITER', 'FOR_ITER', 'LIST_APPEND', 'DELETE_NAME',
    'JUMP_FORWARD', 'JUMP_IF_TRUE', 'JUMP_IF_FALSE', 'JUMP_ABSOLUTE',
    'MAKE_FUNCTION', 'SLICE+0', 'SLICE+1', 'SLICE+2', 'SLICE+3', 'BREAK_LOOP',
    'CONTINUE_LOOP', 'RAISE_VARARGS', 'YIELD_VALUE',
    # New in Python 2.7 - http://bugs.python.org/issue4715 :
    'JUMP_IF_FALSE_OR_POP', 'JUMP_IF_TRUE_OR_POP', 'POP_JUMP_IF_FALSE',
    'POP_JUMP_IF_TRUE', 'SETUP_EXCEPT', 'END_FINALLY',
    'LOAD_FAST', 'STORE_FAST', 'DELETE_FAST', 'UNPACK_SEQUENCE',
    'LOAD_GLOBAL', 'LOOKUP_METHOD','CALL_METHOD',# Only allows access to restricted globals
    ] if x in opmap))

type url as http://ip_address_of_odoo_box:8069 to openerp odoo application, create a new database with demonstration data.

login odoo, and click inbox, odoo should crash because segmentation fault.


any feedback should be thankful.

jeffery


On Mon, Jun 1, 2015 at 8:03 AM, Nicolas PIGANEAU <nicolas.piganeau@ndp-systemes.fr> wrote:
Thank you for your answers, Raphaël and Ruben.

I've got further today, and managed to have my first Odoo server on PyPy running ! 
But there is still some way to go. 

As I said before, I came across 2 issues: 

1. Database creation does not work. After having rapidly inspected the generated database, only the base module has been loaded, although the logs show that all the modules have been loaded. I suppose that there has been a rollback somewhere, but I'll have to check. 

2. When using an already created database, I had issues with inherited views, as you mentioned. With the fact that only one every other line of the inherited view was inserted inside the base view, I managed to locate the issue, and applying the following patch solves it:

===================================================================
--- openerp/addons/base/ir/ir_ui_view.py	(revision e9f802425cf8ca0f82e7075774585d4ca897ad08)
+++ openerp/addons/base/ir/ir_ui_view.py	(revision )
@@ -449,7 +449,7 @@
                     if node.getparent() is None:
                         source = copy.deepcopy(spec[0])
                     else:
-                        for child in spec:
+                        for child in spec[:]:
                             node.addprevious(child)
                         node.getparent().remove(node)
                 elif pos == 'attributes':
@@ -461,7 +461,7 @@
                             del node.attrib[attribute[0]]
                 else:
                     sib = node.getnext()
-                    for child in spec:
+                    for child in spec[:]:
                         if pos == 'inside':
                             node.append(child)
                         elif pos == 'after':

Maybe there are other places in the Odoo code where the same sort of modifications should be made, but this was enough to get it running. 
For the few quick tests I made, I get QWebException complaining about void* not being known and giving a code 500. But if you reload several times, you get it running in the end. And when you get the interface, then no more problems.

Now, for the big question: is it really faster than Odoo in CPython ?
Well, I did not have time to make a lot of benchmarking, but:
- The interface seems the same speed as the CPython version, I'll have to make deeper tests to see if the server calls are faster or not.
- On one specific stock operation (validating a several hundred lines picking) which took me just above 1 minute on CPython server took only 27 seconds on the PyPy server.
=> So it seems to be significantly faster (~x2), at least on operations with a lot of python calculation like stock operations. Now it needs to be confirmed with full benchmark.


Nicolas PIGANEAU

-------------------
NDP Systèmes
06 68 93 87 56
http://www.ndp-systemes.fr

----- Mail original ----- 
De: "Raphaël Valyi" <rvalyi@akretion.com> 
À: "Community: Framework" <expert-framework@mail.odoo.com> 
Envoyé: Samedi 30 Mai 2015 15:02:09 
Objet: Re: PyPy 

Hello Nicolas, 

a few months ago I tried again and commented here https://www.odoo.com/groups/community-framework-62/community-framework-10089512 

Basically it seems the Pypy lxml implementation differs from the CPython one and eventually this is what breaks the view inheritance system. Also I pioneered tests on Jython back around 2011 and I had the same issues with the Java based lxml adapter not having the exact same internals of the one of CPython. May be it's not this issue anymore but I bet it is. That's interesting to see efforts here however. Let us know if you make any progress on this. 

Regards. 

On Fri, May 29, 2015 at 9:00 PM, Nicolas PIGANEAU < nicolas.piganeau@ndp-systemes.fr > wrote: 



Hi, 

Has anyone tried to run Odoo on PyPy recently ? 

On my side, I used: 
- lxml-cffi from git:// github.com/amauryfa/lxml.git - latest gevent sources (after merging https://github.com/gevent/gevent/pull/552 ) 
- psycopg2cffi and psycopg2cffi-compat (pip install)

I had also to patch Odoo sources to add pypy opcodes to safe_eval: 

--- ../GitHub/odoo/openerp/tools/safe_eval.py	(date 1432802291000)
+++ ../GitHub/odoo/openerp/tools/safe_eval.py	(date 1432901508000)
@@ -69,6 +69,7 @@
 
 _SAFE_OPCODES = _EXPR_OPCODES.union(set(opmap[x] for x in [
     'LOAD_NAME', 'CALL_FUNCTION', 'COMPARE_OP', 'LOAD_ATTR',
+    'CALL_METHOD','LOOKUP_METHOD',
     'STORE_NAME', 'GET_ITER', 'FOR_ITER', 'LIST_APPEND', 'DELETE_NAME',
     'JUMP_FORWARD', 'JUMP_IF_TRUE', 'JUMP_IF_FALSE', 'JUMP_ABSOLUTE',
     'MAKE_FUNCTION', 'SLICE+0', 'SLICE+1', 'SLICE+2', 'SLICE+3', 'BREAK_LOOP',


And also add this one if I wanted to start in multi-worker mode (for some reason pypy complained they were float instead of int): 

--- ../GitHub/odoo/openerp/service/server.py	(date 1432802291000)
+++ ../GitHub/odoo/openerp/service/server.py	(date 1432901508000)
@@ -713,7 +713,7 @@
             raise Exception('CPU time limit exceeded.')
         signal.signal(signal.SIGXCPU, time_expired)
         soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
-        resource.setrlimit(resource.RLIMIT_CPU, (cpu_time + config['limit_time_cpu'], hard))
+        resource.setrlimit(resource.RLIMIT_CPU, (int(cpu_time + config['limit_time_cpu']), hard))
 
     def process_work(self):
         pass


Then my server starts all right and I am able to go to the database manager page.
Creating a database seems to work, but it is then impossible to load the created database. Indeed, it seems some imports have been skipped.
In particular, ir_model_data is only filled with xml_ids of the base module.

So I created a database from a cpython instance and accessed it through my pypy instance. I could login, but then the loading stopped after 
displaying the top menu. A look in firebug showed that JS complained about "nv" not being defined in graph_widget.js. 

So I checked what had been loaded, and found out that only one every other scripts had been loaded from this module, but also from all 
other modules. Same goes for css files: one loaded, one skipped, one loaded, etc. for each line of the assets_backend extension template 
(web_graph/views/web_graph.xml in this case).

I suppose this an issue linked with lxml, that sort of misses one line every two lines but I haven't got a clue on where to go now. 

Any idea or experience on this?

Thanks !

Nicolas PIGANEAU

-------------------
NDP Systèmes
06 68 93 87 56 http://www.ndp-systemes.fr 

_______________________________________________ 
Mailing-List: https://www.odoo.com/groups/community-framework-62 
Post to: mailto: expert-framework@mail.odoo.com 
Unsubscribe: https://www.odoo.com/groups?unsubscribe 






-- 
Raphaël Valyi 
Founder and consultant 
http://twitter.com/rvalyi 
+55 21 3942-2434 
www.akretion.com 



_______________________________________________ 
Mailing-List: https://www.odoo.com/groups/community-framework-62 
Post to: mailto:expert-framework@mail.odoo.com 
Unsubscribe: https://www.odoo.com/groups?unsubscribe

_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-framework-62
Post to: mailto:expert-framework@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe




--
Jeffery        -odoo expert form kunshan, suzhou, china