Community mailing list archives

Re: sales order confirmation v9: speed issue

Stefan Rijnhart
- 02/01/2016 10:21:27
On 01-02-16 12:03, Fabien Pinckaers wrote:
<blockquote cite="" type="cite">
Details about the two latest performance issues:

1) The main issue was after gathering data from the DB with float field. They were always rounded in convert_to_cache even if it comes from the DB (validated = False). The problem is the round function asks for a new cursor and when there are a lot of floats (i.e., while prefetching), that is an issue. (# lines * nbr floats in table since all floats are prefetched). The fix, was to skip the rounding when it comes from the database using a lazy cursor (v8) and a better implementation (master).

2) Also, we improved the request in _product_available with a cr.execute in order to reduce the time of the function from 1s/1K products (after convert_to_cache improvement) to 0.01s/1K products.

I don't have the rev number here, but search LazyCursor in recent commits, in both v8 and v9.

Thank you Fabien, this really gives a boost!

Just some anekdotal numbers from one of my cases:

On stock.picking, there is an unstored related field on the stock moves' destination location. When the picking is displayed in the list view, Odoo's prefetching strategy causes to up to 200 rows from stock_move to be read just to fetch the first row's destination location. This is on a database with several tens of thousands of pickings. My test concerned displaying the first 80 of the incoming pickings. These 80 pickings have a total of 979 stock moves. Nothing too outrageous you'd think. Still, displaying this list nearly took 18 seconds.

Here is the average of three tries of the search_read call from the client to read these pickings comparing the situation before and after the LazyCursor change, as well as a change of mine manipulating the prefetching strategy when fetching this particular related field:

- Without any optimization: 17568 ms
- With LazyCursor: 6060 ms
- Fetching the related field with prefetch_fields=False in the context: 529 ms
- With both LazyCursor and fetching the related field with prefetch_fields=False: 388 ms

While my prefetching hack still drastically improves the performance over LazyCursor when displaying the list view, it is a very adhoc and dirty hack for a specific bottleneck that can easily lead to deteriorations when applied unwise. The LazyCursor change however gives an equally dramatic improvement to these bottlenecks all over Odoo in an elegant and generic manner.

So, great work! According to the git log Raphael Collet was responsible for the change, so thank you Raphael and everyone else who may have been involved.

Opener B.V. - Business solutions driven by open source collaboration

Stefan Rijnhart - Consultant/developer

tel: +31 (0) 20 3090 139