Odoo Help

Welcome!

This community is for beginners and experts willing to share their Odoo knowledge. It's not a forum to discuss ideas, but a knowledge base of questions and their answers.

1

Search method causes 1GB memory usage and slow performance

By
Martin Pishpecki
on 7/30/15, 8:53 AM 430 views

Sending emails from Openerp 7 causes 1GB memory spike. This is caused by  this part of the code:

fol_obj = self.pool.get("mail.followers") 
fol_ids = fol_obj.search(cr, SUPERUSER_ID, [
('res_model', '=', message.model),
('res_id', '=', message.res_id),
('subtype_ids', 'in', message.subtype_id.id)
], context=context)

in /openerp-base/src/addons/mail/mail_message.py (def _notify method line 881).

Our mail_folowers table contains around 3 000 000 rows. The search method, in converting our code example to sql statement, at the point where it creates a where clause, it does a string supstitution something like - "where s%, s%, s%...." with 3 000 000 %s characters, and then substitutes a list of 3 million id elements to create a single string. Than it executes that string. This causes a memory error on our system in /openerp-base/src/server/openerp/tools/misc.py,  flatten() method (line 226).

A workaround is to use cr.execute() insted of search method, but I am afraid that this points to a bigger problem. When using tables with more than a couple of million of rows, search is a big problem.


+1 for the detailed testing and numbers. I'm curious for reactions about this. Have you tried other search methods to improve this speed?

Yenthe
on 7/30/15, 8:56 AM

No just this one. Now I use direct query to database. But I have other big tables like account move lines with even bigger number of lines. This is a good candidate why I need a lot of ram to run easy reconcile (or any reconciliation for that matter). I will be fixing this in the near future. I will post more on this problem at that time.

Martin Pishpecki
on 7/30/15, 9:35 AM

Your Answer

Please try to give a substantial answer. If you wanted to comment on the question or answer, just use the commenting tool. Please remember that you can always revise your answers - no need to answer the same question twice. Also, please don't forget to vote - it really helps to select the best questions and answers!

About This Community

This community is for professionals and enthusiasts of our products and services. Read Guidelines

Question tools

1 follower(s)

Stats

Asked: 7/30/15, 8:53 AM
Seen: 430 times
Last updated: 7/30/15, 9:35 AM