Server crash when pushing search more button with 70000 products

Odoo is the world's easiest all-in-one management software.
It includes hundreds of business apps:

CRM
e-Commerce
Accounting
Inventory
PoS
Project management
MRP

Take the tour

Posts People Badges

Tags View all

odoo accounting v7 pos invoice

About this forum

Help

Server crash when pushing search more button with 70000 products

5 Replies

6381 Views

server search crash more down

yann

Hi !

I'm on Ubuntu 12.04 and OpenERP 7.0.

I've experiencing an issue when I've done : Quotation / create / add an item / search more

The issue raised is a XmlHttpRequestError, which hide the fact that the server was down...

Can someone explain me how to fix this / how these actions can make the server goes down ?

Thanks all !

(sorry for bad English)

EDIT :

I've opened the web console when the error occurs, and there's this entry :

POST [path]/web/dataset/call_kw 
send
jQuery.extend.ajax
instance.web.JsonRPC.instance.web.Class.extend.rpc_json
instance.web.JsonRPC.instance.web.Class.extend.rpc
instance.web.Session.instance.web.JsonRPC.extend.rpc
prototype.(anonymous function)
instance.web.Model.instance.web.Class.extend.call
instance.web.DataSet.instance.web.Class.extend.name_search
values.push.action
$input.autocomplete.select
$.Widget._trigger
_on.menuselect
handlerProxy
jQuery.event.dispatch
elemData.handle.eventHandle
jQuery.event.trigger
(anonymous function)
jQuery.extend.each
jQuery.fn.jQuery.each
jQuery.fn.extend.trigger
$.Widget._trigger
$.widget.select
(anonymous function)
_on.click .ui-menu-item:has(a)
handlerProxy
jQuery.event.dispatch
elemData.handle.eventHandle

Does anyone know how to fix this ? I tried to go into the core of openerp-server, but I can't find in wich file the event is handled...

EDIT 2: And why I can't see any changes in the log when I use the --debug starting option ?

EDIT 3: this happen with any form, if this button is present, and if I push it.

EDIT 4

I've made things yesterday, and now I think I understand what happen :

created a new base
imported products (csv again) in the new base
deleted products in the old base

Now that there's less products in the old base, the "Search more" button works fine... So, that seems to be a size problem (size of the table products_products, and/or products_template, ....)

But, why OpenERP is so popular, if a problem like the number of products can simply makes your server crash ? And, again, any idea ? How to fix this ? ^^

EDIT 5:

I've now tried to start openERP in front, with the --debug and --log-level, but that dont seem to change anything.. I were not prompted when the server crashed. Just a "Processus stopped"... Plus, no error occurs in this log when the XmlHttpRequestError occurs on the client side...

PS : And I can't post the log, because this site don't allow be to post links (even if there's no links in the text I wan't to post...)

Thierry Godin

How many products do you have in the DB ??

yann

Author

I have around 70000 of them.

Daniel Reis

You should post the server log tail.

Laura Jarvenpaa

This is now just hypothesis BUT problem could be the fact that jQuery can't handle such a big amount of data based on this blog post: gabrieleromanato.name/the-limits-of-jquery-with-large-data-structures/ (I can't post links so removed http part from the beginning) it is said that jQuery crashes when example code described in this blog creates table with 30000 rows and 90000 table cells. And for fetching all your products from the databases jQuery has to create a table that has 70000 rows (even if it uses pagination to divide received data on multiple pages it will fetch all data from your db and how big amount of data it for once dumbs into a table showed on your openERP client depends on the implementation).

But like I said this is just hypothesis that you could use for trying to find solution from google I can't be 100% sure if this is connected to your problem.

Edit: out of curiousity could you test to import your products to runbot and check if it will have the same problem (other alternative if you don't want to put your data to runbot is that you download this 15 days trial if you haven't used it yet and test to import data there)? Then we would know if problem is in your configuration/system or in OpenERP in general.

yann

Author

Thanks for this answer ! I'll try what you say !

Gustavo

Restart your server with the --debug option and see what the log says. You will have detailed information of what caused the crash

yann

Author

Thanks, but the log file is exactly the same before and after clicking on this option (in debug mode)... perhaps this is not the same log file that we are talking about ? I work with the file /var/log/openerp/openerp-server.log, defined in openerp-server.conf (tutorial from The Open Sourcerer).

Daniel Reis

The --debug option doesn't change the logs - it makes the server console stop at the error with a (pdb) prompt. To increase log level use --log-level=debug.

yann

Author

I think I understand, but the log is written in the background, because I run OpenERP this way... So I've to run OpenERP in a console, in the foreground, with these options, to see this (pdb) prompt ?

Marco Di Francesco

I tried this fix and works very well. h_t_t_p_s://code.launchpad.net/~openerp-dev/openerp-web/6.1-opw-574218-xal/+merge/10737

Anton Chepurov

In the UI, there is absolutely no point in fetching all the records, when a user is only interested in finding a handful of them (see @Laura Jarvenpaa's reply on how JS handles it).

You could jump directly to /addons/web/static/src/js/view_form.js:2937 in your server and specify the limit explicitly:

--- addons/web/static/src/js/view_form.js   2013-10-10 09:11:25 +0000
+++ addons/web/static/src/js/view_form.js   2013-12-11 12:09:11 +0000
@@ -2934,7 +2934,7 @@
 values.push({
     label: _t("Search More..."),
     action: function() {
-        dataset.name_search(search_val, self.build_domain(), 'ilike', false).done(function(data) {
+        dataset.name_search(search_val, self.build_domain(), 'ilike', 500).done(function(data) {
             self._search_create_popup("search", data);
         });
     },

A limit of 500 records is more than enough for manual handling - and UI is exactly for this, it's not a batch processing of db data or something.

If you also use web_m2o_enhanced module, then the very same change will have to be introduced to the dataset.name_search() function invocation in web_m2o_enhanced/static/src/js/form.js (search for 'Search More...' in the file).

Thierry Godin

Hello, Maybe you could start by taking a look at the openerp-server.log file You should find some useful informations in there.

Regards.

yann

Author

I've already done this. And this crash doesn't leave any log entry... The log is exactly the same before I click on "Search more" and after... but thanks for the answer.

This question has been flagged