Community mailing list archives

Re: sales order confirmation v9: speed issue

XOE Corp. SAS, David Arnold
- 02/01/2016 23:24:21
Thanks everyone warmheartedly for sharing!

El lun., 1 feb. 2016 a las 22:26, Gaspard Dessy (<>) escribió:
Hello and thanks for all your answers.

So our best solution so far has been to tell users to open another tab while the first is being confirmed. We configured well the server and it scale pretty well (if you confirm 10 SO at the same time, each SO still takes almost the same time).

For information we also removed the functionality that checks availability on-the-fly when we add SO line. Btw, I also think it could be an optional functionality in Odoo by default to uncheck this feature but it is another story. This query was very slow and not really needed in our case, so has just been dropped (we have over 500 Warehouses, the query built by Odoo with parent left-right was becoming crazy).

We have also tested the solution proposed by Nhomar and Fabien, upgrade Odoo and postgres to 9.5

Postgres 9.5 and latest odoo9 comunity :
test 1 : 162 line 117s
test 2 : 162 line 113s

9.3 and the old : before >120 second.

So no significant improvement on that side.

Christophe Combelles also suggested the idea to optimize the code by using direct SQL. 
We have done some prototypes and evaluated the risk but as our first solution was finally good enough, 
we set this development "pending" as the customer want to go live soon and we don't want to rush/take risks. 

On Mon, Feb 1, 2016 at 10:22 PM, Stefan Rijnhart <> wrote:
On 01-02-16 12:03, Fabien Pinckaers wrote:
<blockquote cite="" type="cite">
Details about the two latest performance issues:

1) The main issue was after gathering data from the DB with float field. They were always rounded in convert_to_cache even if it comes from the DB (validated = False). The problem is the round function asks for a new cursor and when there are a lot of floats (i.e., while prefetching), that is an issue. (# lines * nbr floats in table since all floats are prefetched). The fix, was to skip the rounding when it comes from the database using a lazy cursor (v8) and a better implementation (master).

2) Also, we improved the request in _product_available with a cr.execute in order to reduce the time of the function from 1s/1K products (after convert_to_cache improvement) to 0.01s/1K products.

I don't have the rev number here, but search LazyCursor in recent commits, in both v8 and v9.

Thank you Fabien, this really gives a boost!

Just some anekdotal numbers from one of my cases:

On stock.picking, there is an unstored related field on the stock moves' destination location. When the picking is displayed in the list view, Odoo's prefetching strategy causes to up to 200 rows from stock_move to be read just to fetch the first row's destination location. This is on a database with several tens of thousands of pickings. My test concerned displaying the first 80 of the incoming pickings. These 80 pickings have a total of 979 stock moves. Nothing too outrageous you'd think. Still, displaying this list nearly took 18 seconds.

Here is the average of three tries of the search_read call from the client to read these pickings comparing the situation before and after the LazyCursor change, as well as a change of mine manipulating the prefetching strategy when fetching this particular related field:

- Without any optimization: 17568 ms
- With LazyCursor: 6060 ms
- Fetching the related field with prefetch_fields=False in the context: 529 ms
- With both LazyCursor and fetching the related field with prefetch_fields=False: 388 ms

While my prefetching hack still drastically improves the performance over LazyCursor when displaying the list view, it is a very adhoc and dirty hack for a specific bottleneck that can easily lead to deteriorations when applied unwise. The LazyCursor change however gives an equally dramatic improvement to these bottlenecks all over Odoo in an elegant and generic manner.

So, great work! According to the git log Raphael Collet was responsible for the change, so thank you Raphael and everyone else who may have been involved.

Opener B.V. - Business solutions driven by open source collaboration

Stefan Rijnhart - Consultant/developer

tel: +31 (0) 20 3090 139


Gaspard Dessy
CEO Southeast Asia
 +62-813-2970-6795 - Skype gaspard.dessy

IT solutions for International Trade, Manufacturing and e-Commerce

Your trusted partner in Asia

Post to: